First off, I don't completely understand the technical aspects behind it, but basically the sound and signal is not as stable as 4 or 8ohms. My understanding is that as we play music through our amps,it does not always send a "1ohm"(if that is what your speakers are wired to) load to the amp. This causes the signal to not be even and essentially for audiophiles, it causes breakup and inconsistencies.
Now I could be wrong, so please do not take what I say as gospel.. I ask all my electronics questions to my buddy that works at SLAC, so I am by no means an expert on this subject. But basically it is an audiophile thing..