There are two ways to damage a speaker; Mechanically and thermally.
Power the subwoofer beyond it's mechanical limitations; Overexcursion. Results in bottoming out the driver and physically damaging it. Torn tinsel leads, wrecked VC former, broken joints, torn spider, etc etc.
Powering the subwoofer beyond it's thermal limitations. Results in the voice coil frying.
Well....then, ofcourse, there is actual physical damage......like dropping the woofer and offsetting the voice coil, that kind of stuff.
I bought some used and blown IDMax a while back that were obviously overdriven. Bottom the drivers out so hard that the VC former was rammed into the dustcap and left a nice big circular dent in the dustcap. Other one had same problem, aswell as probably thermally overdriven as the voice coil was stuck in the gap.