Your imagination is not serving you well. Ponchos do catch the wind and are very much not aero, but they generally have loops or something for your hands to hold them in place, and are never long enough to go down to your wheels, that would be lethal.

Their biggest downsides is they don't protect your knees and below, and they flap around in the wind, and may obscure lights and cameras on your handlebars if you're not careful. They're also worthless unless you have full fenders/mudguards.

But they're great in warm weather because ventilation is excellent, even if they're made of cheap fully waterproof material. Indeed there's little point in making them out of breathable material because it'll only make a fractional improvement, and you generally only wear them when it's actively raining, and when it's wet, breathable material isn't breathable any more anyway.

They're also easy to put on and take off over clothes so for utilitarian/transportation use when you don't change clothes at either end they're extra convenient.