If you have a smartwatch turn off the remote Bluetooth settings. It appears to be waiting for you to speak into your smartwatch microphone. This is why Active edge only worked when I was holding the phone with my LEFT hand, which is the arm where I wear my S3 Frontier. I was too dumb to notice!
settings/ Google/ settings/ Google Assistant/ Phone/ other voice settings/ hands free
Google couldn't have buried this menu better if they tried!
Thanks! I too have a Gear S3 Frontier paired with the Pixel 3a and was having the same problem. I could activate active edge on the 3a but it would never hear what I said. After turning the bluetooth recording setting off, it worked as expected. Much appreciated!