Just for clarification...I'm not an originality guy...honestly not sure how many ground straps and cables were there originally or where they should be.

But if you're willing to toss originality aside, one of the best things anyone can do for any vintage vehicle electrical system is to securely ground EVERYTHING. Otherwise if you think about the path the electricity needs to take from your headlights and switches back to the battery...through rubber subframe mounts? Rubber padded engine mounts? Rubber radiator support mounts? I'm a firm believer in chaining ALL of these items together with quality ground straps, whether Pontiac put them there originally or not. Usually they can be mostly hidden. Often that alone brightens everything up and eliminates a whole lotta gremlins...

I'm also not promising you this is your actual problem. It might be something else. But this is where I'd start if it were me, because it's a valuable improvement to the car whether it fixes the immediate problem or not. But often if a high electrical load causes a large voltage drop as you describe, that can also be an indication of an overloaded/high resistance connection or wire somewhere on the hot side (dirty/rusted/corroded fuse or plug or nearly-broken or drastically undersized wire, or partially burned fusible link maybe?). These connections will carry electricity up to a point...but when asked to carry MORE electricity that the weak link can handle, the resistance in that location gets too great, heat builds, heat causes resistance to increase yet more, lights flicker...etc. Just like it would if trying to push all that current thru the only available ground, which if lacking a proper strap might be a rusted clip attached to a rusted brake line attached to a rusted subframe...

You can temporarily trouble-shoot a ground problem with a long jumper wire too (not too thin...think maybe 12-ish gauge wire). Attach one end securely to negative at the battery, now with the lights on touch the other end firmly to clean metal on various places...dash/floor/subframe/engine, wherever...if suddenly your lights brighten up when you touch that wire to one of these locations...you've absolutely confirmed your problem is a lack of an appropriate ground path.