Yes, Power=V*I But this is true for DC signals and AC signals with a pure resistive load.
Once you throw a speaker in there, (An inductive piece of equipment), You'll get some inductance which will then give a slight phase shift between the voltage and current.
When you have a phase shift, (caused by either capacitance or inductance) your power equation becomes V*I*cos(theta), where theta is the angle between V and I.
So the only accurate way to measure an amp's output is to use a power meter, or get some high power load resistors.