In a medium saucepan, combine the milk and butter and warm over low heat until about 125 degrees F, and the butter has melted.
In the bowl of a stand mixer, combine 1 1/2 cups flour, sugar, yeast and 1 teaspoon salt. Add the warmed milk and melted butter, bananas, eggs, yolks and vanilla extract. With the dough hook, mix for about 2 minutes. Add in 3 1/2 cups flour, half cup at a time, and stir until it comes together. Dough will be sticky, but that's okay. Transfer to a greased bowl, and cover loosely with plastic wrap and a damp kitchen towel. Let rest in a warm spot until doubled in size, about 1 hour.
Punch dough down, and transfer to a well floured work surface. Knead a few times and then roll out to a large rectangle about 3/8 of an inch thick. Cut into circles using a 3-inch cookie doughnut cutter. Place cutouts on a baking sheet lined with parchment paper and cover with plastic wrap. Let rise for about 30 minutes or until doubled in size.
Fill a dutch oven halfway up with oil and bring up to 365 degrees F. Fry the doughnuts in batches, for about 2 minutes or until golden brown, turning once halfway through. Remove with slotted spoon, draining off any excess oil and transfer to a wire rack. Continue frying the remaining doughnuts.
In a large bowl, combine the sugar, cinnamon and nutmeg until evenly combined. Roll the warm doughnuts around in the sugar and serve with caramel for dipping. Enjoy!