That's assuming that you're using the same brand and model of sub. In my case the comparison is between an '04 XXX, which is on the higher end of the price spectrum and the Assassin 8 (x4). The Assassins are much cheaper and take up less space. Looking back I wish I had gotten the 10 XXX two years ago. I didn't need the extra output that the 12 could potentially provide and I would have saved a ton of space in my trunk and probably wouldn't be doing the rebuild that I'll be doing when I get home.
Also depending on your priorities and the goals of your system, spending more money to save a bunch of space might be totally worth it. If all you care about is getting loud and are willing to sacrifice a lot of space to do it, then you don't need to worry about small subs, but if you are building your system for you, and you actually use the car and need to conserve space and maybe retain your spare tire and the use of some of you trunk, the huge enclosure requirement of larger subs just eliminates them as an option.
Another thing to consider is the physical size of the larger drivers themselves, enclosure notwithstanding. In the first big system I ever had, there were 6 10's. I couldn't have fit 1 12 in the same space. The sub itself wouldhave been too tall for the room that was there. The size of the 12 was a huge design obstacle in my current system as well.
The only thing you lose going to a smaller driver is output. With the vehicle cabin gain factored in, most car subs in a sealed box can play down to sub-audible frequencies. The larger subs will simply be louder all other things being equal. If there was anything wrong with 10s or 8s no one would buy them so no one would make them. Fact is for a lot of people, smaller subs just make more sense.