Heres a more comprehensive answer
5.21 What's worse for a speaker, too much or too little power? [iDB]
======================================================================
Problems occur (in everyday operation) when distortion is fed to a
speaker. This occurs MUCH more often when you are dealing with an
underpowered system - typically the owner will turn up the volume too
much or set the amplifier gains too high to try and get more volume
from the system. These introduce distortion to the signal - this will
destroy *any* speaker. ((see Section 5.22).)
When a speaker is overpowered, however, it is not nearly as common to
have these kind of problems, so speakers aren't blown as much. Of
course, it is certainly possible to destroy a speaker (thermally) by
overpowering it, but you'll have a pretty hard time doing this on your
own, especially with standard car audio amplifiers.
5.22 Why is distortion harmful to my speakers? [RK]
=====================================================
Distortion is hard on speakers for two reasons.
Reason 1: Distortion causes the power spectrum to shift upwards in
frequency. A bass note, when distorted, will have lots of high
frequency energy. This will cause mid-ranges and tweeters to fry, if
the amplifier is operating full range. It doesn't harm woofers,
necessarily.
Reason 2: Distortion causes the average power to be much higher.
Typically, a music signal that never clips has an average power level of
1/4 the peak power level for even the most compressed speed metal or
pop. More dynamic music will be 1/8 the peak level or less on average.
When you clip the amp hard, the average output moves up to the
full-rated output of the amp or more. The peak to average ratio can be
less than 2 to 1, with the peaks being at double the rated power of the
amp, and the average being at the rated power of the amp or higher.
Thermally, the speaker can handle the average power being 1/4 the rated
power of the amp (little to no clipping), but it will have a much harder
time with the average power being the amp's rated power or more (massive
clipping). As you might expect, this is pretty hard on the amp, too.
For transients, most speakers can handle a ton of power. But for long
term signals, the power handling is much less.