Thanks Squeak for the read. I believe I get it.
In other words, set the gain on the x-over as high as possible and the gain on the amp as low as possible. This way only the signal noise of the x-over and HU is amplified and not the signal noise of all three, HU, x-over and amp are amplified...