Bring milk to a boil or heat in the microwave for 60-90 seconds. Pour over tea bag and steep for 20-30 minutes. Allow to cool to room temperature.
Preheat the oven to 350ºF degrees. Line a 12-cup muffin pan with cupcake liners.
In a mixing bowl, whisk the flour, baking powder, baking soda, salt, and spice blend together until combined; set aside.
In the bowl of a stand mixer fitted with a paddle attachment, beat the butter and both sugars together on medium speed until lightened and creamy. Beat in the eggs, one at a time, until combined, scraping down the sides and bottom of the bowl as needed. Beat in the sour cream and vanilla on medium speed.
Pour the dry ingredients into the wet ingredients and mix until combined. Add the chai milk, then mix until just combined, scraping down the sides and bottom of the bowl as needed.
Scoop the batter into the liners. Fill only 2/3 full to avoid baking over the sides. Bake for 18-22 minutes, or until a toothpick inserted in the center comes out clean. Allow to cool in the pan for 10 minutes before transferring to a wire rack to cool completely.