Heat your oven to 375 F. In a medium bowl, whisk together flour, cocoa powder, baking soda, and salt; set aside.

In the bowl of an electric mixer fitted with the paddle attachment, cream together butter and sugar until light and fluffy, about 2 minutes. Add egg, milk, and vanilla, and beat until well combined. Add reserved flour mixture; mix on low speed until combined.

Using a tablespoon or 1 ¾-inch ice cream scoop, drop dough onto ungreased baking sheets, about 2 inches apart. Bake until cookies begin to spread and become firm, 8 minutes.

Remove baking sheets from oven, and place a marshmallow, cut-side down, in the center of each cookie, pressing down slightly. Return to oven, and continue baking until marshmallows begin to melt, 2 to 2 ½ minutes. Transfer cookies to a wire rack to cool completely before frosting.

For the frosting: Place confectioners’ sugar in a medium bowl and whisk in butter and cocoa powder. Add milk and vanilla, and whisk until well combined.

Spread about 1 tablespoon of frosting over each marshmallow, starting in the center and continuing outward until marshmallow is covered.