alter.net issues continue for months - verizon doesn't care

[tzm]

I have had multiple reoccurring issues with Fios and alter.net here in NYC for the past 6 months. Upon searching the web I see it's a reported problem by other users. I have 150/65 and going to multiple sites that pass traffic through alter can be extremely slow.

Talked with Verizon and they won't do anything about it. They will ask you to run a speed test through the Verizon Fios site. Of course everything checks out fine since it is one hop directly to their servers. They will say that anything else is third party and won't help.

My suspicion, though I have no facts to back it up, is they know about it but perhaps use it as a way of limiting your bandwidth. Primarily this happens on the weekends and from 11am to 1am week days. Anything that doesn't route through alter.net works at full speed.

Perhaps it is also that other ISP's pay for only a limited amount of bandwidth which jams up the highway.

I don't have enough technical knowledge to know but my traceroutes show the problem clearly.

For those users thinking about switching to Fios you should do traceroutes to the sites you most often use. If they hop through alter.net…. find another provider…….or stay where you are if it works well……

[Hubrisnxs]

https://www.linode.com/docs/networking/diagnosing-network-issues-with-mtr#analyzing-mtr-reports

That is a tutorial on how to interpret Traceroutes/MTR reports.  It's very good.   I would recommend you download WinMTR and do your tests again.   Traceroutes don't give you a lot of visibility into where a problem might be.  

Your first trace to LAtimes is quite good with round trip MS under 10MS

Your 2nd one to Bing, is similiarly good, under 10MS

your 3rd to Slashdot isn't too bad, it's hits mild traffic at around 40ms, but nj to Chicago, it's about right. 

[seitzbg]

Verizon's backbone and peering are great when everything is working however it seems they play the netflix game far too often with peers. (Don't care about capacity, want payment to upgrade it, which causes it's users to suffer)

I have 500/500 and the best case i've seen lately is maybe 250Mbps from my colocation in Ashburn, VA (one of the largest peering points in the world) to my FIOS in Delaware (~100 miles).  Depending on the route/peering sometimes it slows down to a few Mbps.  I don't think there is anything we can do as it relates to their core business practices and engineering decisions that are not made by people who frequent this forum. 

If anyone does ever manage to find a backbone engineer for FIOS I'd love to talk to them 🙂

[nick_d]

It has gotten a miniscule bit better but that's being generous. I pay for 300 yet can barely watch an Amazon Prime or Netflix show without some buffering pauses during playback. All support is ever willing to do is send a new router -- as if that is even close to the issue.  If they want to just waste time and money with things they know aren't the problem I'd be happy to take a free month every time I called instead of a "new" router. At least then they can waste their money while I enjoy the benefit of not paying for sub-par service.  

[luxedesign]

I am so **bleep** about this problem I am on the phone with liquidweb.com getting all this info right now for months I have been on freakout and verizon is just STUPID, I want a credit for my internet for 3 mos minimally for the frustration dealing with Verizon. Today I had to reluctantly pay the bill - because the person in retentions {please keep your posts courteous} can't accept that it's not me or my server or my connection or whatever else - the time I've spent circumventing this is worth a LOT of money to me and it's embarassing to have to make any kind of excuse to anyone about your 'internet connection' Verizon - when Obamacare called to see if I wais available to fix a healthcare site, I didn't say my internet might be on the rocks.... I can't handle your Business TV price, for TV I barely watch so I have residential. You need a support option for residential that ISNT premium nor is it designed for grandma not knowing where the start button is. Or now orb or whatever else. I am not nuts obviously as I heard you guys BOUGHT OUT alter.net this AM - but I AM STILL HAVING THE PROBLEM EVEN THOUGH ON THE VERIZON END THE ISSUE WAS SUPPOSED TO HAVE BEEN FOUND AS SOLVED TOTALLY. Next post is my latest trace.

{edited for privacy}

[luxedesign]

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\system32>tracert limopostmaster.com

Tracing route to limopostmaster.com [69.16.208.193]
over a maximum of 30 hops:

  1     2 ms     1 ms     1 ms  192.168.1.1
  2     7 ms     6 ms     7 ms  L100.NWRKNJ-VFTTP-168.verizon-gni.net [108.35.18
7.1]
  3    11 ms    13 ms    15 ms  G0-3-5-2.NWRKNJ-LCR-21.verizon-gni.net [130.81.1
82.196]
  4    38 ms     8 ms     7 ms  ae0-0.NWRK-BB-RTR1.verizon-gni.net [130.81.209.1
54]
  5     *        *        *     Request timed out.
  6     *        *        *     Request timed out.
  7     *        *        *     Request timed out.
  8    60 ms    55 ms    93 ms  0.ae1.XL3.CHI13.ALTER.NET [140.222.225.179]
  9    34 ms    27 ms    28 ms  TenGigE0-6-0-6.GW5.CHI13.ALTER.NET [152.63.64.21
0]
 10    21 ms    24 ms    23 ms  giglinx-gw.customer.alter.net [152.179.92.62]
 11    32 ms    31 ms    31 ms  lw-dc2-core6-te9-2.rtr.liquidweb.com [209.59.157
.226]
 12    31 ms    33 ms    39 ms  lw-dc3-dist15.rtr.liquidweb.com [69.167.128.205]

 13    32 ms    33 ms    32 ms  limopostmaster.com [69.16.208.193]

Trace complete.

C:\Windows\system32>.

[Hubrisnxs]

That tracert really doesn't show what your ISP is talking about.

When someone at a high level is diagnosing traceroutes, you need both path's.   Do you BY Chance have or can get a trace from liquidweb back to you.

This is a a good indepth ts guide  - If liquidweb never did a trace back to you, then the guy you're talkling to might not know how to diagnose connections issues like that.

https://www.nanog.org/meetings/nanog45/presentations/Sunday/RAS_traceroute_N45.pdf

---

Few people are skilled at interpreting traceroute

•

Most ISP NOCs and even most mid-level engineering staff are

not able to correctly interpret complex traceroutes.

•

This leads to a significant number of misdiagnosed issues and

“false reports”, which flood the NOCs of networks worldwide.

•

In many cases the problem of false reports is so bad, it is all

but impossible for a knowledgeable outside party to submit a

traceroute related ticket about a real issue

---

If you can get that tracert, to and from we could probably nail it down or pinpoint it for you. In general though, you can always email NOC@verizon.com if you feel it's really an issue on their end.

[luxedesign]

LW is the only company that DOES know what they are doing TBH.

I know that we both did traces - that's how we both discovered the problem frankly, LW has their own ticket in w/ Verizon - apparently all LW's tech support has been made aware of the issue now because it's affecting literally all their customers. 

I believe the TR back from them is somewhere in a ticket that I have open, I'll search for it.

According to LW's latest update, again Verizon has quietly PURCHASED alter.net to get rid of the roadblockage... I'd love to see if the issue is even fixed but I can't - right now apparently the FIOS line has exploded and it's a full on thing all day with cops blocking off the street now due to downed lines. Not that there was a storm or any reason for it to be downed... 

[Hubrisnxs]

BTW, have you tried getting a new route?

If you can access your router, do the following, and you'll hopefully be working in 5 minutes.

Actiontec MI424-WR - RELEASE DHCP
Click on MY NETWORK icon at the top.
Select NETWORK CONNECTIONS from the menu on the left.
Select BROADBAND CONNECTION (coax or ethernet) depending on your connection to the ONT.
Click SETTINGS
Click RELEASE
Click APPLY
Disconnect the router immediately to prevent it from re-requesting a DHCP lease ---- wait at least 7-10 minutes NO LESS, and turn it all back on and try again

[luxedesign]

This seems like the steps to release your DHCP lease? Yeah that doesn't help much, I could do it five times get five completely varied new IP's and have the same problems. 

[luxedesign]

Sad to say the tech at verizon SUCKS. It was easier to go to Google, and find a CSV file of all the VZ federal division people's cell phone than it has been to alert VZ of any kind of problems - I coulid literally call the guy at NASA, Homeland Security, or the White House - faster than I could get to tech support.... 

[kwgiberson]

Been having issues since July, no resolution yet from Verizon. The below failure @ ALTER.NET is the same for most East Coast individuals after 8PM EST, anyone who plays FFXIV Online knows that with a VPN the game becomes unplayable. 

Tracing route to 199.91.189.36 over a maximum of 30 hops

5 * * * Request timed out.
6 12 ms 13 ms 10 ms 2.ae0.XT2.NYC4.ALTER.NET [140.222.227.27]
7 14 ms 22 ms 23 ms TenGigE0-7-0-9.GW8.NYC4.ALTER.NET [152.63.18.206]
8 29 ms 26 ms 26 ms tinet-gw.customer.alter.net [152.179.72.122]
9 33 ms * 38 ms xe-7-3-0.mtl10.ip4.gtt.net [141.136.106.106]

Any word from verizon would be nice, I don't want to switch ISPs over something as paultry as an online game but this is getting ridiculous.  

[uidzer0]

I'm experiencing the same issue as far as timeframe goes. I'm East Coast (VA) and have both fios residential (150/150) and fios business (75/75) here at the house. In the morning I get speeds that I would expect to see and have no problems at all. At nearly 7:30pm on the dot my speed and reliability drop to nearly nothing. Residential and business speeds both drop to less than 10M up and down. Sometimes all the way down to 2M. I'm on the third night in a row of dealing with this. Calls to verizon only results in them attempting to blame my equipment (even though it happens on both residential and business which have 2 separate routers and aren't connected to each other at all) and doesn't matter if a system is hard wired or wireless, the results are the same) and finally end up with either them wanting to send a tech out or they just tell me they'll fix it offline. Apparently my current ticket is due to be 'fixed' by tomorrow 01/30/2015. We'll see what happens. I've sent them outputs of mtr with over 7000 packets sent showing exactly where the congestion is. It's verizon. I get the feeling that they're playing chicken again with level3, netflix, etc, etc trying to get content providers to pay them for a fast lane. It's getting really irritating.

[biter1]

Did they ever fix this for you?  I'm dropping out between Verizon and Alter.net:

3  g0-14-4-4.tampfl-lcr-21.verizon-gni.net (100.41.193.126)  7.758 ms  8.138 ms  7.446 ms

4  * * *

5  0.ae3.gw4.tpa2.alter.net (140.222.227.199)  7.003 ms  6.143 ms  6.642 ms

This is for any site and I can't even use a proxy to bypass the hop.

Traceroute has started…

traceroute: Warning: google.com has multiple addresses; using 65.196.188.59

traceroute to google.com (65.196.188.59), 64 hops max, 72 byte packets

1  wireless_broadband_router (192.168.1.1)  0.629 ms  0.373 ms  0.373 ms

2  l100.tampfl-vfttp-39.verizon-gni.net (72.64.153.1)  2.708 ms  1.269 ms  7.494 ms

3  g0-14-4-4.tampfl-lcr-21.verizon-gni.net (100.41.193.126)  7.758 ms  8.138 ms  7.446 ms

4  * * *

5  0.ae3.gw4.tpa2.alter.net (140.222.227.199)  7.003 ms  6.143 ms  6.642 ms

6  google-gw.customer.alter.net (152.179.251.98)  6.804 ms  6.313 ms  6.461 ms

7  59.188.196.65.tampa.google-ggc.verizon.com (65.196.188.59)  6.567 ms  6.619 ms  6.513 ms

[bobnkneel]

I too am being bottlenecked at Verizon's beautiful 0.ae2.XT3.DFW9.ALTER.NET server. I mean, who wouldn't I want my packets pinged around a bunch to have my favorite internet game be unplayable. Yay net neutrality....

[tns2]

---

@biter wrote:

Did they ever fix this for you?  I'm dropping out between Verizon and Alter.net:

3  g0-14-4-4.tampfl-lcr-21.verizon-gni.net (100.41.193.126)  7.758 ms  8.138 ms  7.446 ms

4  * * *

5  0.ae3.gw4.tpa2.alter.net (140.222.227.199)  7.003 ms  6.143 ms  6.642 ms

This is for any site and I can't even use a proxy to bypass the hop.

Traceroute has started…

traceroute: Warning: google.com has multiple addresses; using 65.196.188.59

traceroute to google.com (65.196.188.59), 64 hops max, 72 byte packets

1  wireless_broadband_router (192.168.1.1)  0.629 ms  0.373 ms  0.373 ms

2  l100.tampfl-vfttp-39.verizon-gni.net (72.64.153.1)  2.708 ms  1.269 ms  7.494 ms

3  g0-14-4-4.tampfl-lcr-21.verizon-gni.net (100.41.193.126)  7.758 ms  8.138 ms  7.446 ms

4  * * *

5  0.ae3.gw4.tpa2.alter.net (140.222.227.199)  7.003 ms  6.143 ms  6.642 ms

6  google-gw.customer.alter.net (152.179.251.98)  6.804 ms  6.313 ms  6.461 ms

7  59.188.196.65.tampa.google-ggc.verizon.com (65.196.188.59)  6.567 ms  6.619 ms  6.513 ms

---

What you are pointing to is NOT a problem.  Many of these servers are set to not to respond.  Others are set to respond at a lower speed.  If it was a problem later servers would also not respond or go up drasticly in response time.

As said Alter.net belongs to Verizon and represents most of its backbone.  Verizon will not claim any real problems in it aren't their problems.  The points where they do so (correctly or incorrectly) are where they pass off to some other company.

[cjvnyc]

I'm glad I found this thread.

It's criminal fraud for Verizon to sell these higher-speed plans knowing they will use a throttled routing system. I was sold on Verizon as the opposite, but find the Timie of Day slowing to be counter the sales pitch I was given.

My tracert:

Tracing route to gw1.ibllc.com [208.245.107.3]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 5 ms 4 ms 3 ms lo0-100.NWRKNJ-VFTTP-333.verizon-gni.net [*]
3 7 ms 6 ms 7 ms T1-12-0-1.NWRKNJ-LCR-21.verizon-gni.net [100.41.130.206]
4 * * * Request timed out.
5 8 ms 6 ms 6 ms 0.ae13.GW13.NYC1.ALTER.NET [140.222.234.193]
6 6 ms 6 ms 6 ms interactivebrokers-gw.customer.alter.net [157.130.249.134]
7 7 ms 6 ms 6 ms gw1.ibllc.com [208.245.107.3]

Trace complete.

[CRobGauth]

Verizon (as well as most other ISPs) only guarantees service to connect to their network.

That's why you always hear and see ads that say up to X.

And I don't see where you issue is.

Looks like good ping times to me.

[cartographer1]

Same problem here out of Lakeville Massachusetts.  

25/25 Mbps connection, and the speed test says 29/27!

 3  g0-15-2-5.bstnma-lcr-21.verizon-gni.net (100.41.195.224)  12.596 ms  12.389 ms  12.775 ms

 4  * * *

 5  * * *

 6  0.ae3.gw9.ewr6.alter.net (140.222.231.237)  18.603 ms  17.620 ms  17.240 ms

Packet loss in the same location here.    

Trying to download a 6.21 GB file from Apple which should take 31 minutes is taking nearly 2 days!

Direct ethernet connection to the Actiontek modem makes no difference.

[CRobGauth]

The * does not necessarily mean packet loss.

Some devices don't respond to ping for trace route.

If it is always the same hops and no other hops have * then likely just that.

[Ordellus]

They know about it.

Verizon actually owns the alter.net domain

http://ip-lookup.net/index.php

Just search any alter.net IP and look at the domain owner info.  -> Verizon business LLC.

Verizon is simply using a different face of it's company to intentionally route traffic through congested crappy servers / routers / whatever..... then they claim it's "not verizons network" and charge you more money.

I don't know about you but I'm currently shopping with comcast. since verizon would rather lie to it's customters than deliver what it's paid to deliver.

[KChappy1]

I realize this is a really old chat... but I, and others in the Maryland/Virginia area can not get to my business website.  I have tried changing the nameservers and IP address for my website, I have changed the DNS for my router, which works momentarily and then reverts back to a Verizon route.  Clearly Verizon is the issue and they refuse to admit it. Did anyone ever have any satisfaction with getting this fixed?  Trace route runs through alternet too.  I am not experiencing it going to any other site but my own small business ecommerce site.  People on other services do not have this issue.  

Have you reported ot the FTC?