Um...wrong!!! The SQ will NOT remain the same. There is radiated noise throughout the car audio envronment. Ignition tic, alternator whine, hum, buzz, pop, etc...Some of it is picked up through the RCA's. What noise is picked up will be amplified by your amp's pre-amp section. The more you have to turn up the gain on the amp, the more the noise will be amplified....along with the clean signal. No, it might not be 80 db of noise at min volume, but there WILL BE noise. A higher voltage from the output of your HU will allow you to keep you amp gains low. Thus minimizing the radiated noise picked up in your RCA's. Now, how much of that noise bothers you IS ANOTHER STORY. BUT, it will be there.
More important, to me at least, is that the output from a "2V" HU is not 2V. It's more like 1.7V, with .3V of clipped signal at full volume. 2V outputs are derived from the 12V charging system. 4V, and up use switching power supplies to provide an unclipped signal all the way to full volume. BTW, can't clipped signals cause damage to your speakers?? Ohhh, that's right THEY DO!