The gains are nothing more than input sensitivity adjustments. An amplifier should work virtually identical no matter the gain position its set to, so long as the gain is adjusted according to the input signal strength. In other words, if you experience no induced noise running a 2v deck and have your gains set properly, you will experience no performance gain (or loss) by changing it out for a 4v deck and re-adjusting your gains lower.
Don't get caught in the trap of thinking a gain is a volume know that should always be kept as low as possible. There is an optimal gain setting for your h/u's signal output potential, anything above that will cause clipping, and anything below will limit output artificially. Limiting output via the gains may be desireable, but that's another discussion.
Cheers.