Preheat oven to 350°F. Line a 12-cup muffin pan with paper liners, and spray liners with cooking spray.
In a bowl using an electric mixer, combine flour, Splenda Sugar Blend, milk powder, baking powder, baking soda, and salt on low speed. Add margarine and yogurt and beat on medium speed for about 2 minutes.
In a glass measuring cup, whisk together egg, water, and lemon extract. Add to batter and mix just until smooth.
Spoon batter into prepared cups.
Bake for 20–25 minutes, or until cupcakes spring back when lightly pressed. Remove to wire racks and cool completely.
To make the glaze:
In a bowl, combine powdered sugar and lemon juice; whisk until smooth. Whisk in water a bit at a time (you may not need all of it) to make a thick but pourable glaze.