My other 9 had a vanity mirror, a base 350, w/o ac, m/d p/s, and remote mirror. But the visos, ash tray, and glov box lid were thrown away because, I guess, a sfb coulnd't stand it because I was in my second new company truck.
It's not the mirror causing the problem. The plastic tips on the end help, but the main problem is the adjustment screw, the slotted screw. Hogey it down some. If it continues to loosen, try removing them and clean out the internals.
Over the 34 years of non-stop owership, I have found this to be part of vehicle pm, needing a tweak however often.