My experience with Note 4 speaker function and blue tooth headsets is they malfunction as often as they work properly during phone calls. When the speaker or blue tooth is activated during a call there is dead air between myself and the other party. I am on my fourth Note 4 in a month due to this problem, that is I bought a new phone and Verizon has sent me three replacements in an effort to fix the issue. Each phone has malfunctioned in exactly the same way. Verizon and Samsung are unable to discuss this with me other than to go through all the diagnostic steps every time I call to ask for help. I am at the point of either living with the problem or changing phones altogether.
Something I find amazing is when I talk to Verizon or Samsung they say this is not a reported issue. Maybe I am the only person who had this experience. Maybe I got the only four Note 4's (in a row) that malfunction like this. Maybe.
This leaves holding the phone to my ear, which is rather old school and does not work with my state's driving laws. Any fixes or reality checks are welcome.