The caller will hear a tone at the end of the "ring" indicating the person being called is already on the phone. This only works if both callers are Verizon users - so if your callers are on another carrier, they will not hear the tone. Also, if you use ringback tones, they hear your chosen music, but no tones.
By any chance, do you or any of your callers have or use the Advanced Calling feature? I have no idea if this would change anything, but it does seem to affect some other calling features. I'm just trying to come up with ideas as to why your callers would not hear the tone.