Higher signal voltage does one thing only: help suppress noise induced into the signal circuit. That's it, nothing else. So if you have no noise problems now, you dont need a higher signal level.
Gains are adjustments for input sensitivity. The amp maker has no idea if you will hook the amp to an 8v h/u, or some old .25v deck. So to allow maximum power output no matter what h/u is used, they have to add an adjustable input sensitivity knob. The higher the signal strength, the less sensitive the input stage is adjusted to. The weaker the signal, the higher the sensitivity is set. So, no matter what signal voltage your deck is capable of, your amp will put out the same maximum output (in watts), assuming the gains on the amp are set properly in the comparison. No output change on power or perceived loudness of the stereo, the only difference being a higher signal level, which increases the distance between signal level and the noise floor. As I said initially, there is no advantage to higher signal voltage other than noise suppression.
And to be honest, 2.5v is about where I draw the line at if you need more signal voltage to rise above your noise floor, the noise problem is then large enough to fix it instead of just increasing signal level. 2.5-4v is the sweet spot for signal voltage, anything higher is a waste, and for those people who like 'oldschool' amplifiers, it ends up not even being used. In other words, if 2.5v of signal wont overcome the noise, the noise problem is big enough that it should be addressed directly.