Have you inspected the pin in the center of the leaf springs? You should be able to see it through the hole in the bottom of the plate that clamps the rear to the leafs and also serves as the lower shock mount. Check both sides. It is very possible this is not lined up, if so your rear is not sitting square on the leafs.
Yes, I rechecked it, and I also had this problem before I did the rear end swap which even then the the pin was lined up properly, thats why I trying to figure out what could be wrong. I can only hope that it is a bad quarter panel install that is throwing it off.