If that plate is not flat with your pads or the axle you will put all the loads where you don't want them. I would try to get the pads flush with the plate/axle. As mentioned everything needs to relax a bit, then try to tighten it down some more, and keep trying to get it flush. I would be very concerned with safety if they are not flush.