You asked for opinions so here is mine. For the same 20k you find a car that is already done, paint and all, ready to drive.
The down side to working with what you have is, honest opinion here, you will end up putting more into the car than what it will be worth in the end. I know many will not agree with me as I always look at the investment aspect. No matter how nice the car is when it's done, it will not be worth what it will cost to get it there due to the non matching number etc.
Think of this, let's say your car that you have now is worth 5k, sell it & put 10-15k with it and buy a nice driver. Most likely the person who buys your bird will be a body/paint person and they will do the body/paint and not have as much into it etc.
The estimate the guy gave you might sound high, but once you start removing all those rusty panels, 20k will rack up REAL QUICK...