Indeed. To avoid any discussion that may clutter this thread, the following is text from a very educated gentleman on the topic of higher impedance drivers.
Nice article, but one *very important* thing I didn't see mentioned was power compression. In the end, the actual resistive component of the 8 ohm and 4 ohm speaker probably wouldn't be terribly far off from each other in most drivers, assuming substantial power levels (a safe assumption, because if we're talking about low volume levels, the point is moot anyway). That theoretical doubling of power you get when you halve the impedance quickly goes away when you factor in amplifier losses and power compression. And that's before you even get into the loudspeaker sensitivity stuff. [For those who don't know, power compression is the phenomenon whereby the resistive component of the driver increases with temperature -- naturally, more power dissipation means higher resistance which, for a voltage amplifier, means a reduction in power dissipation. It's a sort of negative feedback mechanism].
Anyway, you're right to say that the 4 ohm vs 8 ohm issue is all a tradeoff. But you can also look at it from a different perspective: to deliver the same amount of *power* into the 4 ohm speaker requires less voltage. This doesn't matter much until the signal starts to approach the onset of clipping. The problem is that with an 8 ohm driver you'll reach clipping earlier than you will with a 4 ohm driver (again, assuming somewhat similar power dissipation!).
The obvious response to that is: so what? If we've already established that the difference between 4 ohm and 8 ohm output isn't that great because of power compression and amplifier losses, then the difference in clipping shouldn't be all that great either, right? Maybe not. For some musical content, the difference between the peak and average output (usually specified as the "crest factor") can be substantial. So the transients at even moderate volume levels can sometimes reach the rail voltage. That translates into distortion...whether or not it's enough to be perceptible is the question. We can typically tolerate a pretty hefty amount of clipping before it becomes noticable.
Personally, I like to use amplifiers that have enough guts to them to be able to run the 2 or 4 ohm load they claim to be able to run and not have to worry about it. But I also realize that if the speaker I want to use is a higher impedance, like a lot of the 8 ohm stuff that people tout, it's definitely not the end of the world...