hey, your 4 channel is not 500 watts. There is no way, what components are you running? Even if you are adding another 150-200 watts, that will not affect your subs output much. If you have a voltage problem, it is because your alternator is strained or your battery is dying or not completely charged.
Didn't i mention i have had this same problem? I have been down the same road as you, i still don't know why my system got quieter either. It could be so many things.
Oh, and what size wire is running to your battery?