a lot of the surrounds are messed up because idiots don't understand the mounting style and stab the surround with a screwdriver. the cone is somewhat easy to break, but again, it's easy to break if you're a dumbass. jl is a big name, so a lot of clueless people gravitate towards them. if you read the instructions, it's pretty simple. to answer your question, no they don't just break on their own.
as far as affecting performance... it depends on the extent of the damage. generally small holes aren't a big problem.