THIS IS MY PRESENT SETUP
HOPE IT HELPS IN YOUR CASE
Tom, I looked over your diagram and it doesn't seem to be analgous to the situation the OP describes.
To the OP: I have the same setup you describe. I feed one TV from my one STB using HDMI and others (a small TV and a computer tuner card) using the coax RF output of the STB (channel 4 in my case). (I also feed another TV using component cables, not your case.) All TVs receive video and audio even when the others (e.g. the one via HDMI) are off. So what you see isn't the way it can work. It all works fine for me.
The only thing I can think is that your TV is somehow shutting down the STB audio signal when you turn it off. Some TV's have the ability to control up-stream HDMI funtionality, and your may be doing this. (This is sometimes refered to on a Panasonic TV as "Viera Link" sttings, for example, and can be turned on or off. Other manufacturers use other terms for this generic HDMI function. For example, if enabled if I turn the something other input source connected via HDMI.)
If HDMI control is not the case, than something else is wrong. It could be something in the STB audio settings, but I don't see any obvious one that would cause what you're seeing, but I may have missed something. I would contact Verizon support directly (they're great via Twitter or the DSL Reports Verizon Direct forum by the way) for help.
Hope you can get your problem solved.