To set the gains, you have to define the goals. there are 3 common goals:
1.) SPL -- get the most power without distortion and without damage to equipment.
2.) Street -- maximize the volume of the system, limiting distortion to an "acceptable" level and allowing for "adequate" tonal balence.
3.) SQ/HiFi -- minimize distortion and give good tonal balence while achiving a volume deemed "loud enough". SQ is a more subject goal where you might choose to add distortion or change the tonal balence because it just sounds better to you. but the goal isn't to get as loud as possible.
#2 is the most common goal. in all cases, damage to equipment is undesired.
only goal #1 can be acheived using a DMM or Oscope.
An Oscope or DMM is a tool and should be used in all 3 cases.
My General method for #2, #3.
1.) turn HU to 3/4 max volume. disconnect speakers. turn off effects.
2.) increase gains on all amps. use DMM to find an output voltage that is either the rated output of the amplifier, or the rated power of the speaker. MARK THIS GAIN. a cell phone camera or dot of ink/paint can be used. or a drawing.
3.) decrease all gains to minimum.
4.) reconnect a set of speakers to an amp. (apply filtering as normal)
5.) increase the gains until the speakers begin distorting or the value found in #2 is found, or the speaker becomes too loud. decrease gain slightly.
6.) disconnect the speaker. repeat #4,#5 for all amps
7.) reconnect all speakers. now all speakers are playing at the "loud" setting.
8.) leave the gain setting for at least 1 amplifier as it is. the speakers attached should be the least loud with the gain setting you found.
9.) lower the gain setting on the other amplifiers as needed to acheive an acceptable tonal balence.
10.) at this point you should have a fairly clean sounding system. it is now that you can optimize the system for your goals. if the subwoofers can handel the extra power, you can boost the gains on the subwoofers to give extra power (and minor clipping).
the DMM and Oscope are tools. if you set the gains using just these tools you will end up correcting yourself by adjusting the balence/fade/ect... on the HU to control the volume. at that point its only marginally better then cranking gains to maximum (and setting the volume to 5/30). *it is fine to set all gains using the DMM/OScope and then using the HU to control the actual volumes. i don't like this as much because if the battery gets disconnected, you usually lose your settings! so long as you realize that the DMM is just a tool you will be fine.
if you follow a method above, you should be able to achevie a goal of a system that is as loud as you want it (for SQ/HiFi) while limiting distortion to acceptable limits (HiFi/Street), and giving acceptable tonal balence (HiFi,SQ,Street).
edit --
if the amp was overrated, then you should have heard distortion and stopped increaseing the gains (inf steps 4,5). if the amp was underated you are still fine. Only for goal #1 is this an issue. for this goal, the amp should have its gain set using an Oscope or creatively with a DMM.