i think you should start with YOUR system first.
buy some subs, an amp or 2, some MDF, and some components.
im assuming you know almost nothing about car audio right now (not making fun, but its the impression i get) buy some equipment, and try and figure everything out for yourself, with your car. get a feel for it, you may not like it. it doesnt always work (i like cars, i like music, why not open a car audio shop?) without hands on. schooling is good yes, but you cant learn some things from a book. you have to learn to troubleshoot.
the "custom" part would really come from your boxes more than anything imo. you have to learn to cut and make them look pretty, but thats not even the half of it. you have to take dimensions, mathematical formulas to calculate volumes and port areas to get the right hz and all that bs. its not easy. i dont even know how to do it, i ask people on here to design for me (something you cant do everytime you have a customer come in wanting a box). then you gotta learn wiring, ohms, impedances, and every car wiring is different. sometimes if you wire the radio in wrong, the car will think its being stolen, and not start until you go to the dealer to get codes lifted. (i work in an autobody repair shop) some cars, if you cross 2 wires under the dash, the fuel pump wont pump gas..etc. things like that. cars a high tech now. its not simple plug and play anymore sometimes.
aanndd you wouldnt just have an audio shop more than likely. to make the real money, youd need a full custom shop. rims/tires/suspension/custom interior/engine mods.etc. and all that stuff = lots of overhead = thousands of dollars worth of stuff laying around for customers to look at. they just dont wanna walk into a customization shop thats a solid white wall room with a magazine on the counter to look through and wait for stuff to be ordered..comein..blah blah blah
so get some stuff, even if its cheap, and start playin.