The lateral load would be greatest when turning and braking, which I assume you do when you turn into a driveway. I suspect the bearings are not adjusted correctly, which would be a bigger issue when turning. Make sure the spindles are not trashed and carefully set the bearings to spec.