Packet loss / timeouts to many Amazon AWS and CDN sites
RetroTekLLC
Newbie

I have 150/150 Verizon FIOS Business Service with a static IP block in northern NJ (upgraded about two years ago from BPON 50/50 to GPON 150/150. ) It's been working great, except for the last month or so, where almost all traffic directed at sites or services hosted on Amazon AWS, CDN or Cloudfront (for example, 13.33.0.0/16 subnet) is dropping/blocking packets and either takes forever to get a web page, page looks totally broken, or it just times out completely. 

Sometimes the website itself is hosted elsewhere, but has links to ads or other resources that are hosted on AWS/CDN, and those hold up the main site as well.

To note, the issue does periodically clear up and sites work fine for a time before the issue reoccurs and sites are broken again.


I've seen some posts on other sites recommending lowering the MTU to something under 1500 on the WAN interface. I've tried all that, but it doesn't fix the issue. 

Packet captures with Wireshark shows three-way handshake with the affected sites fail with SYN packets being retransmitted until the handshake finally completes or times out. Ping is fine, no ICMP loss, and traceroutes (where possible) look good as well.

I run PFSense as my main router/firewall (no Actiontec or other Verizon CPE) which I've been using since the day I got FIOS, without issue. Currently on 2.4.4 stable public release.

I also have a secondary Internet WAN connection to OOL on the same PFSense device, and when I switch to using that as my outbound Internet, it works fine to all AWS sites without issue.

Anyone have a similar experience? Which Verizon department/phone# would be the best place to start with troubleshooting?

Thanks.

0 Likes