Preheat the oven to 375ºF. Grease a donut pan with cooking spray.
In a small pot, heat milk over medium high heat until just shy of boiling. It should be hot, steamy, and small bubbles will appear around the edge. Remove from the heat and submerge the chai tea bags in the milk. Allow to steep for 5 minutes. Remove the tea bags, using the back of a spoon to squeeze out all the milk that may have been absorbed by them.
Meanwhile, whisk together flour, granulated sugar, baking powder, cinnamon, and allspice in a large bowl. Add the egg, vegetable oil, 3 tablespoons of the chai infused milk, yogurt, and vanilla. Refrigerate the remaining chai milk until you’re ready to make the glaze. Stir the batter until all the dry ingredients are incorporated.
Transfer the batter to a large zip top bag and snip off one of the corners. Squeeze batter evenly into the donut pan. Alternatively, you could use a couple of spoons to spoon the batter into the pan.
Bake 10-12 minutes until donuts are springy and lightly browned. Remove from the oven and allow to cool 5 minutes before turning out onto a cooling rack to cool completely.
Once the donuts are cool, prepare the glaze by adding powdered sugar to a small bowl along with 2 teaspoons vanilla extract and about 1 tablespoon of the chai infused milk. Stir until smooth and shiny, adding a few drops of chai milk at a time until the desired consistency is reached. You will probably not need all the chai milk. I like my glaze thick but runny enough to drip down the sides a bit. If it gets too runny just add a little more powdered sugar. If it is too thick and you run out of chai milk you can use regular milk to thin it out.
Dip one side of the donuts in the glaze and return to the cooling rack to let it set. Sprinkle with additional cinnamon if desired.