I don't see the loss in HP by just changing to a 4bb. Yes it would be a miss match and not be the best combo in the world but as long as you are using a Q-jet and not a Edelbrock carb you would still feel a performace diffrence. I have change 2bb to 4bbs and thought it was a great change.
One of the magazines did a engine dyno where they did incremental changes with the intake, then the cam and then the exhaust maybe not in that order but it showed small increases until the last restriction for the motor was replaced and then you saw the HP really jump.
Getting a Q-jet to work well with a restricted 350 should just be a matter of increasing the tension on the seconday spring to avoid bogging all the time.