Well, I mean as far as this discussion goes. Calling someone a wigger isn't calling them a white nigger by today's slang terms, though it may look that way. Wigger has no hateful connotation to it, unless saying someone may look silly is hateful. I can't see where you can think it is just as bad as calling someone a nigger. It is somewhat as ludicrous as someone saying "saying darn or dang instead of damn is just as bad", when obviously both take on different feelings/meanings.
But your missing the point that it is DERAGOTORY and implies that a person looks like a black person.
1) Using that is stereotyping black people
2) the connotation of it implies that being like black people is not good.
Even if you don't think its the same as calling them a white ni**er, it does have negative connotations that can be related to the idea that there is something wrong with being black.