I too recently did a 4-speed swap on my 67 LeMans with only 54k miles. I dorve the car for a couple of months with the 326, 2-speed auto, 2.56 open rear. I too was reluctant to do the 4-speed conv. However, having only 2 speeds, slow and slower became old really quick!
So, I made the change and I am glad I did. That little 326 is a whole new car now! If we keep the car it will get a 3.55 posi rear next.
As for originality, if you keep the old original parts, how can it possibly be a bad thing to a new potential owner? Also, be sure to use OEM/correct parts vs. a 4-speed conv. hack job which I have seen plenty of. Now that will definately have negative impact down the road if you go to sell the car. It's about the same as hacking up a dash and installing some horrendous looking radio etc. A hack job that will later come back to haunt the person who did it etc....