Preheat the oven to 350 degrees F. Using your electric mixer, cream the butter, sugar, and vanilla together until it is combined.
Place the lemongrass and 1 tablespoon water together in a small bowl. Heat in the microwave for 1 minute to loosen the flavors. Divide into two equal portions.
Turn the mixer on low, then add half the lemongrass, salt and flour a little at a time, until the dough comes together.
Dump the dough onto a floured work surface. Roll the dough out until it is 1/3 of an inch thick. Using a floured 2-inch cookie cutter, cut out cookies and place on a parchment paper lined cookie sheet. Try to cut out as many as possible from the first sheet of dough. Each time you gather and re-roll the dough the cookies will get a little tougher.
Bake for 20 minutes—until barely golden. Allow the cookies to cool completely.
Pour 3 tablespoons of milk in a medium bowl. Add the remaining lemongrass and microwave for another minutes to allow the lemongrass to infuse into the milk. Whisk in the powdered sugar until smooth. Add a drop of yellow and a tiny drop of green food coloring if desired.
Dip the tops of each cookie in the glaze. Use a spoon to scrape the excess off the sides, then place back on the parchment paper to dry.