Yeah, I would definitely get a new alternator, and maybe even a capacitor. If the lights are dimming get a new alternator or a capacitor. The cap will help the dimming lights, but if you are running below 12v you will need to replace the alternator so that you have enough juice to power everything.
I agree the new alt is your best first step. I was running a fosgate 500watt and soundstream 300 watt with HO alt, and it took care of my headlight dimming and voltage drops. Next best step is a 1-2farad cap, if the alt doesnt work by itself. Good luck