Any particular reason you have them mounted on the pillar?
I'd try mounted down in the kickpanel area aimed off-axis. Should help with the issues you're having.
Or, as was mentioned, buy a whole new set if you don't mind spending the money. The Infinity's really aren't that good in any aspect.