fgf's did not come with passenger side mirrors, so location mounted is likely the issue. If you can't swivel the mirror to a useful location, you are just stuck with the condition unless you are willing to re-mount the mirror (sounds like further forward in your case) to a better location. My '67 had that issue but mine was oposite. As cars got closer to me, they could not be seen. (was mounted too far back).

I simply sold the car and bought another one without the passenger mirror.