I'm not a stereo expert but I am currently replacing the system on my '68 convertible.
You say the original AM radio, though not functioning, is connected to the (original?) antenna. Does that mean your Secret system is NOT hooked up to an antenna? If not you should hook up an antenna and see if that makes things better.
Do you have 2 speaker outs (left/right) from your Secret system? If you have only one out then you can't have stereo (meaning the left and right speaker play different sounds at same time). I'm not sure what system that is but if you can play a speaker test it can help you tell if you are getting stereo sound from your speakers.
Removing the original speaker looked intimidating but wasn't terrible. To get to the dash speaker: Remove walnut trim, then the ashtray holder, then the radio, then loosen (but don't need to remove) the heater controls and that should give you access to the speaker. I was able to easily fit a new 4" multicone speaker to replace my original speaker (at least what was left of it). I might try to rig up a new bracket and fit a 2nd 4" in there too. Right now I am running a 2nd speaker to a custom arm rest/console to get "stereo" sound.
I ordered a modern radio that looks very similar to stock and fits in dash with no mods. It hasn't arrived yet but others have had good results with them.