No. Some people try to make it that way, but mostly they are black people looking for something to get offended about. Before anyone says I'm racist my brother is black so stfu.
Many southern people think of it as a pride from where they're from, not some hateful reminder of breaking off from the states. Tons of people say "I love X team because I'm from there", well a lot of people like SC or wherever because they're from there.
Wearing a white robe around swinging a noose is racist, the flag is not.
Hell, there are soooo many organizations for the advancement of "african americans" (most of whom have ancestors from islands or places OTHER than africa) while there is not one channel, magazine, scholarship, or organization for white people, because it would be racist. People look for something to b!ch about.