Yes, the glove box must come out. I took the glove box door off, too.

Did you get your new heater core yet? Be sure to match it up to your old one. Make sure the tubes are the same, otherwise, you will not be able to install it.