In the end, it's your car, you can (and should) do what you want.
But like 68 Blackbird said, It's probably better to keep the bloodlines pure. Right or wrong, the Chevy guys will think you're just a Camaro wanna-be, and the Pontiac guys will label you a traitor.
The other factor that you'll have to decide is important, is the value. Even a non-original Pontiac block (like a 428 or 455) in the car holds value more than a Chevy swap. A big-block Camaro goes for $$$. A Chevy big block Firebird ain't worth a darn. This is based on what I saw in the marketplace when I was buying, not opinion.
I saw a couple that were in excellent shape, and the price was reasonable had it been just a non-original Pontiac 350, but with a 454 they weren't moving (I'm talking show car and engine with all the good stuff). I think if they pulled the motor/tranny and sold the car seperately, they would have sold and got the money.
You may want to at least keep the old engine/trans in case you sell.