Any and all loud music will affect your hearing. However, 1. you can withstand much much louder at 20-60hz than you can at 1-10khz. 2. Loss can be very frequency dependent. If you listen to tons of 20khz material really loud, you'll stop being able to hear it as well. Same goes for bass. Remember the first few times you heard 'loud' systems that were probably in the 120-130 range? Seemed pretty loud huh, now mid 140's doesn't sound very loud to your ears. Happens every time I step up. My ears hurt for a while when I listen to it, and then that goes away and it doesn't sound as ridiculously loud anymore.
So why doesn't it matter at these hearing tests that you can't hear bass as well anymore? Because they never test you on bass...
Why doesn't it affect your life to have poor and damaged 20-100hz hearing? Because nobody speaks that low (even the lowest voice you've ever heard in your life only has peaks down into the 300-500hz range most likely). Not much of the content we hear in our daily lives is sub-bass, so it has no real effect on them, other than being able to sit in a car doing 150's, where someone who's never been exposed to something that loud will be plugging their ears in pain.