Exactly, it's like any other type of electrical signal, just because it's rated at a certain amount doesn't mean it's putting it out all the time. IIRC it's logrithmic or something, so that for most of your volume knob your input voltage is minimal, only on the last few turns of the knob does it really scale up. (assuming you set your gains normally) So for normal volume listening, your probably not using more than the first couple volts anyway.
