First thing to check would be vacuum at the suspect port at idle. If you have no vacuum until you start to go OFF idle, then yes you should move it to manifold vacuum.
Teeing it into another shouldn't be an issue.
Can you snap a pic of the location of the hose on the Holley?