Yeah, unfortunately, it's a PITA to remove. I just did the same harness on a 68 and I can't remember if I actually had to remove the kick panel. The connector is way at the top, so I may have been able to get at it. I do remember having to remove the AC vent, move the hose out of the way and reach in with one hand.

If it's never been removed, the sealant that seals the vent duct to the sheetmetal might be still holding it. Removing the parking brake isn't always absolutely necessary unless there is a screw on the top that it's blocking.

Sorry I'm not much help. Just be patient and gentle when tugging on it.