Funny thing about that is, I also have a Scion (2008 xB). I have a feeling it's a faulty unit, I already did an RMA with crutchfield and I'm waiting on the replacement. Hopefully that solves my problem.
Will update this post if it doesn't. Thanks to everyone for the suggestions.
-AnarchE