For me its anything under 8 degrees F and i hear a noticeable difference between 10 degrees and 20 degrees. At 35 the bass hits nice, anything under that and you start to get less output because of stiffer surround hard glue etc.
I live in MN too. I broke the spider of an old sub by playing it in the cold, the cabin was warm but i didnt let the sub play at low level first so i just turned it up and started listening, couldnt hear anything wrong till i opened the trunk and heard an awful noise, then i pushed on the cone and heard a horrible noise, look on the side and the spider tore away from the cone.
**** winter!