Routing / Latency issues

If you could supply one of the traceroutes you're seeing issues with to us that would be appreciated.

Verizon really doesn't have Tier II or Tier III tech's that talk to customers.

Those tech's get their reports from tier I only, and then they investigate their issue.

Those tech's are designed to talk with other Tech's and other departments. So their network tech's do talk to customers occasionally, but that is after a report has been sent to them, and they are done looking at it, and they need more info. They may call you personally to get that info. other wise they are fielding calls from Field technicians in the street that are on trouble reports or installs, or they are working with Internal Departments to resolve different issues.

So you have to go through tier I.

The only kindasorta way you can get around that is by doing the twitter support. They are pretty exceptional, but otherwise you can post here, and there are a lot of very experienced users of varying backgrounds that can help you, so feel free to post your traceroutes here. XXX out your actual IP address for safety, and let us know if you are hardwired or wirless when doing your tracert and what bandwidth or kind of traffic you are pushing when you do your tests.

Keep in mind gaming/movie or media streaming and bittorrent traffic when replying.

thanks for the information everyone. I know how these things can go (I myself used to work for a broadband ISP). This type of issue is difficult to get any help with being the issue isn't really with verizons network. However, where verizon routes this traffic to can possibly be controlled. All of my issues appear be to on alter.net which is where verizon routes this specific traffic.

- Used to ping around 20-30ms

- Now my ping times are around 110-150ms

- The latency goes from 20ms to 110-150 within 1 hop of verizons network

- Connection is ethernet based (not wireless)

I have included 2 different traceroutes. One is to a server located in VA and the other in DC. It is clear the hop that my latency starts to occur is on alter.net's network. To add to this, I did also change the MAC of my verizon router to get a new DHCP lease in hopes that the routing might change, but it took the same path off verizons network.

******** VA *********

tracert 64.34.186.94

Tracing route to . [64.34.186.94]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home
2 6 ms 6 ms 6 ms lo0-100.PRVDRI-VFTTP-303.verizon-gni.net
3 24 ms 7 ms 6 ms G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
4 62 ms 15 ms 7 ms ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
5 39 ms 14 ms 14 ms ge-7-1-0-0.NY5030-BB-RTR2.verizon-gni.net [130.81.19.33]
6 18 ms 18 ms 17 ms 0.so-0-0-0.XT2.NYC4.ALTER.NET [152.63.17.21]
7 17 ms 22 ms 20 ms TenGigE0-6-2-0.GW8.NYC4.ALTER.NET [152.63.21.117]
8 115 ms 118 ms 119 ms tinet-gw.customer.alter.net [152.179.72.122]
9 125 ms 127 ms 132 ms xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]
10 130 ms 134 ms 134 ms peer1-gw.ip4.tinet.net [173.241.128.102]
11 136 ms 137 ms 134 ms 10ge-xe-1-0-1.wdc-sp225-sbdis-1.peer1.net [216.187.120.114]
12 135 ms 134 ms 132 ms . [64.34.186.94]

Trace complete.

******** DC **********

tracert 50.97.59.34

Tracing route to 50.97.59.34-static.reverse.softlayer.com [50.97.59.34]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home
2 6 ms 5 ms 5 ms lo0-100.PRVDRI-VFTTP-303.verizon-gni.net
3 6 ms 8 ms 6 ms G0-3-4-6.PRVDRI-LCR-21.verizon-gni.net [130.81.191.240]
4 11 ms 12 ms 11 ms ae4-0.NY5030-BB-RTR1.verizon-gni.net [130.81.209.54]
5 13 ms 12 ms 14 ms 0.xe-10-1-0.BR1.NYC1.ALTER.NET [152.63.18.225]
6 13 ms 12 ms 12 ms ae11.edge2.NewYork.Level3.net [4.68.62.41]
7 12 ms 13 ms 13 ms vlan51.ebr1.NewYork2.Level3.net [4.69.138.222]
8 12 ms 13 ms 13 ms ae-4-4.ebr1.NewYork1.Level3.net [4.69.141.17]
9 13 ms 21 ms 24 ms ae-91-91.csw4.NewYork1.Level3.net [4.69.134.78]
10 20 ms 13 ms 13 ms ae-4-90.edge3.NewYork1.Level3.net [4.69.155.209]
11 15 ms 15 ms 15 ms te2-5.bbr01.tl01.nyc01.networklayer.com [4.26.34.22]
12 13 ms 12 ms 13 ms ae7.bbr01.tl01.nyc01.networklayer.com [173.192.18.176]
13 25 ms 20 ms 21 ms ae1.bbr01.eq01.wdc02.networklayer.com [173.192.18.156]
14 24 ms 25 ms 23 ms ae0.dar02.sr01.wdc01.networklayer.com [173.192.18.203]
15 24 ms 26 ms 24 ms po2.fcr03.sr02.wdc01.networklayer.com [208.43.118.159]
16 27 ms 22 ms 21 ms 50.97.59.34-static.reverse.softlayer.com [50.97.59.34]

I have tried to call in and get assistance but the tech couldn't even provide me an email address to send my traceroutes to. If there is absolutely no way to resolve this issue because it is off verizon's network what course of action can I take? The reason I have the services is for gaming and the connection to my specified servers are unstable making it quite difficult to play.

I would like to note, though I am having these issues - my download speeds, streaming, etc are all perfect and I couldn't be happier with that.

Optimal Actiontec MI424WR router set up for Xbox Black Ops 2

Alter.net is MCI's old backbone network, and Verizon Bought them a year or two ago, so they may be able to help still.

Try the twitter guys, that might be more helpful, or you can do a smart who is look up on the routers that you are having trouble with, and email their NOC or technical contact.

Additionally you should look at these QOS settings - that might help you as well.

I have tried playing with QoS but it doesn't make much of a difference. I also know for a fact that is effecting more than just fios customers in my area. There is a few forum posts around this on certain gaming sites I am on, all reporting similar latency issues for about 2 weeks now to the same servers.

Took a few calls into Verizon to actually get someone that understood what was going on. Most of the tier 1 techs didn't know what traceroute or latency was and were not very helpful in allowing me to provide the information. I had to get a supervisor who was super helpful and assisted by remoting to my machine, running the same traceroutes that I provided here and forwarded them to Tier2 (at least I hope so). I'll respond here once I have more information.

I did want to add that this issue is even happening from some servers I have out of TX and even free ones I have tested from

Trace to tinet.net not going through tinet-gw.customer.alter.net - sourcing off verizon network

Hop

(ms)

IP Address

Host name

8.9.232.73

xe-5-3-0.edge3.dallas1.level3.net

4.69.145.205

ae-4-90.edge4.dallas3.level3.net

77.67.71.221

xe-8-1-2.dal33.ip4.tinet.net

89.149.183.70

xe-1-2-0.nyc41.ip4.tinet.net

141.136.111.14

xe-0-0-0.was14.ip4.tinet.net

Trace to tinet.net going through tinet-gw.customer.alter.net - sourced from my machine on the fios network

2 6 ms 6 ms 6 ms lo0-100.PRVDRIxxxxxxx
3 6 ms 6 ms 6 ms G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
4 8 ms 42 ms 38 ms ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
5 65 ms 128 ms 29 ms ge-1-2-0-0.NY325-BB-RTR2.verizon-gni.net [130.81.17.24]
6 17 ms 17 ms 17 ms 0.so-4-0-2.XT2.NYC4.ALTER.NET [152.63.1.57]
7 19 ms 19 ms 17 ms TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129]
8 111 ms 107 ms 109 ms tinet-gw.customer.alter.net [152.179.72.122]
9 117 ms 129 ms 108 ms xe-4-1-0.nyc40.ip4.tinet.net [89.149.183.113]
10 175 ms 107 ms 106 ms xe-0-0-0.nyc41.ip4.tinet.net [89.149.180.158]
11 125 ms 163 ms 120 ms xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]

Trace complete.

These traceroutes were done from different locations to kinda pinpoint the issue, but you can clearly see that as long as it doesn't touch the tinet-gw.customer.alter.net router there is no latency present.

tinet-gw.customer.alter.net

So the red here indicates the company name

the blue tells us that it's a customer circuit

and then the orange tells us it belongs to MCI/Alter.net now owned by Verizon

So with that info, I think if you follow this with Verizon - you will find that Tinet-gw just needs to buy more bandwidth.

I think tinet is just a transport provider and I am not sure how they relate with your gaming servers. Maybe they are just a transport company contracted, or maybe that is the backbone provider for your game server and their peering point with Verizon.

I am going to be very interested to see how verizon's Tier 2 responds to this. If they have anything they can do other than tell the provider to buy more bandwidth or if there is an actual problem they can adjust and fix.

@chzoom wrote:
Took a few calls into Verizon to actually get someone that understood what was going on. Most of the tier 1 techs didn't know what traceroute or latency was and were not very helpful in allowing me to provide the information. I had to get a supervisor who was super helpful and assisted by remoting to my machine, running the same traceroutes that I provided here and forwarded them to Tier2 (at least I hope so). I'll respond here once I have more information.
I did want to add that this issue is even happening from some servers I have out of TX and even free ones I have tested from
Trace to tinet.net not going through tinet-gw.customer.alter.net - sourcing off verizon network
Hop (ms) (ms) (ms) IP Address Host name
1 1 0 0 8.9.232.73 xe-5-3-0.edge3.dallas1.level3.net
2 0 0 0 4.69.145.205 ae-4-90.edge4.dallas3.level3.net
3 3 3 3 77.67.71.221 xe-8-1-2.dal33.ip4.tinet.net
4 39 39 39 89.149.183.70 xe-1-2-0.nyc41.ip4.tinet.net
5 42 39 39 141.136.111.14 xe-0-0-0.was14.ip4.tinet.net
Trace to tinet.net going through tinet-gw.customer.alter.net - sourced from my machine on the fios network
2 6 ms 6 ms 6 ms lo0-100.PRVDRIxxxxxxx
3 6 ms 6 ms 6 ms G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
4 8 ms 42 ms 38 ms ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
5 65 ms 128 ms 29 ms ge-1-2-0-0.NY325-BB-RTR2.verizon-gni.net [130.81.17.24]
6 17 ms 17 ms 17 ms 0.so-4-0-2.XT2.NYC4.ALTER.NET [152.63.1.57]
7 19 ms 19 ms 17 ms TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129]
8 111 ms 107 ms 109 ms tinet-gw.customer.alter.net [152.179.72.122]
9 117 ms 129 ms 108 ms xe-4-1-0.nyc40.ip4.tinet.net [89.149.183.113]
10 175 ms 107 ms 106 ms xe-0-0-0.nyc41.ip4.tinet.net [89.149.180.158]
11 125 ms 163 ms 120 ms xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]
Trace complete.
These traceroutes were done from different locations to kinda pinpoint the issue, but you can clearly see that as long as it doesn't touch the tinet-gw.customer.alter.net router there is no latency present.

photonslowgame

I am experiencing the identical issue (as well as another friend with Fios in the NJ/Philly area). Game server in question for me is in Toronto, only ~300 miles away, and this routing hardly seems ideal to begin with, but this extra 100 MS at " tinet-gw.customer.alter.net" has been the real killer since Tuesday 15th. I can only hope it is being caused by a substantial upgrade or repair that is in progress.

During business hours on monday I will surely attempt some phone calls as well, but I am less than optimistic. I would hate to be forced to move to comcast over one bad hop on one bad network; I have never had any problem with my fios last longer than a few hours before.

C:\Users\******>tracert 70.42.29.65

Tracing route to 70.42.29.65 over a maximum of 30 hops

1 1 ms 1 ms 1 ms Wireless_Broadband_Router.home xxxxxxx
2 8 ms 7 ms 5 ms L100.PITBPAxxxxx

3 8 ms 9 ms 8 ms G0-1-1-6.PITBPA-LCR-22.verizon-gni.net [130.81.212.58]
4 15 ms 18 ms 43 ms xe-0-1-6-0.PHIL-BB-RTR2.verizon-gni.net [130.81.209.76]
5 19 ms 18 ms 19 ms xe-9-1-4-0.NY5030-BB-RTR2.verizon-gni.net [130.81.23.70]
6 20 ms 18 ms 20 ms 0.so-0-0-0.XT2.NYC4.ALTER.NET [152.63.17.21]
7 21 ms 22 ms 23 ms TenGigE0-5-0-0.GW8.NYC4.ALTER.NET [152.63.21.65]

8 115 ms 111 ms 136 ms tinet-gw.customer.alter.net [152.179.72.122]

9 153 ms 150 ms 150 ms xe-4-3-0.tor10.ip4.tinet.net [89.149.184.110]
10 141 ms 139 ms 144 ms internap-gw.ip4.tinet.net [77.67.70.94]
11 134 ms 136 ms 232 ms border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.196]
12 145 ms 136 ms 130 ms 70.42.29.65

Trace complete.

I agree,

I have now placed 4 calls into support (last 2 being updates on my ticket)

I am not very happy right now as my ticket was 'closed' due to missing 1 call from a tier 1 tech asking if I was still having trouble. However, he indicated (while i was on the phone for 30 minutes) that he spoke with a tier 2 tech and they have from what he says recognized this as a widescale issue and they are working on it. He also let me know that I would receive a call back in the next 24 hours.

I basically purchased a 1 month proxy to redirect my traffic to induce some different routing which will hold me over for now. This is obviously not reasonable as I already pay a decent premium for my Verizon service. I have faith in their network engineers to resolve the issue, but I have already been having this issue for a month =\

photonslowgame

What proxy service did you select? I am also considering doing that. I very much appreciate that you keep posting here chzoom, your dedication to the phone is admirable, I'm not sure I have the patience for that.

I only have patience because I do this for a living myself. I know they can't just snap their fingers and fix it, especially if its a bandwidth or throughput issue with a core circuit or equipment.

If you want to private message me I can give you the service I use right now which works perfect(not exactly sure on the TOS of the boards, probably not a good idea to post the link for public view 🙂 )

gyochum1

I just wanted to bump this with my own new latency issue.
In the NJ area, on FiOS and have had a recent jump.

My specific test case is a game called Mechwarrior Online. The game servers are located in eastern Canada.

Average ping for the past 6 months has been steady between 40-60ms.

Now average ping is consistenly 120-130. This is within the week around Christmas as many other reported.

This was tested with two machines inside the home one via WiFi one via CAT6.

Happy to conduct and post tracert's but It seems that further verifying the issue is not whats needed at this point.

if anyone wants me run one to a specific server as another datapoint let me know.

Thanks, and happy to be of assistance to this cause.

Unfortunately I doubt we are going to get any further help with the issue since of course this is a community driven board. Getting Verizon to recognize the issue and try and assist their customers is like pulling teeth. I explained the issue to probably 5+ different people and spent countless hours on the phone to be called and given a 15 second answer that basically told me to pound sand 🙂

I assume one day it will just magically go away and start working as it used to around xmas time.

gyochum1

While you're probably right I figured i'd toss up one trace, just for the heck of it.

Look familiar? 🙂

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]

2 8 ms 6 ms 14 ms L100.NWRKNJ-VFTTP-94.verizon-gni.net [98.109.1291] 3 11 ms 12 ms 13 ms 130.81.185.116

4 10 ms 9 ms 9 ms ae0-0.NWRK-BB-RTR2.verizon-gni.net [130.81.209.12] 5 10 ms 9 ms 12 ms xe-6-1-1-0.NY325-BB-RTR2.verizon-gni.net [130.8123.230] 6 12 ms 12 ms 11 ms 0.so-4-0-0.XT2.NYC4.ALTER.NET [152.63.9.249]

7 15 ms 16 ms 13 ms TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129 8 106 ms 108 ms 109 ms tinet-gw.customer.alter.net [152.179.72.122]

9 164 ms 131 ms 127 ms xe-4-3-0.tor10.ip4.tinet.net [89.149.184.110]

10 146 ms 142 ms 136 ms internap-gw.ip4.tinet.net [77.67.71.62]

11 137 ms 136 ms 142 ms border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.16] 12 * * * Request timed out. 13 * * * Request timed out. 14 * * * Request timed out. 15 * * * Request timed out.

honestly, i think when we see 'tinet-gw.customer' that infers that they themselves are a customer of mci business or verizon business and bought that connection. that means they probably bought X amount of bandwidth and are getting hammered. they likely need to upgrade that connection to something beefier. I used to work in a NOC and we saw that all the time. If a customer of ours was being saturated with bandwidth, the best we could do was call them, notify them and ask if they wanted to buy more bandwidth. If it's nearing 100% useage, there is only a few ways to go, get more bandwidth, employ better load balancing among their routers, or live with it.

So double check with them if that;s their ckt they are leasing, if it is see if they can tell if it;s being saturated. If they are leasing it, then they are verizon's customer on that connection, and they will have to get that info. for anti competitive reasons I couldn't share that kind of intimate detail with anyone but the customer.

There was another user here named smith, that shared that the GW-customer, meant that it was a leased line from Verizon.

gkgough

I am having the same issue, with the same stop in the Trace..see below.

I was wise enough to pay the extra five dollars a month in order to avoid signing a two year contract, so if verizon cant figure out how to resolve this seeming simple issue that is exclusive to Fios, then I will just switch to a competing service.

Are there Fios Reps. contributing/monitoring these threads? If so, i would expect a prompt response.

4 39 ms 18 ms 18 ms xe-2-1-3-0.NY325-BB-RTR2.verizon-gni.net [130.81
.209.10]
5 * 51 ms 49 ms 0.so-4-0-1.XT2.NYC4.ALTER.NET [152.63.10.29]
6 21 ms 67 ms 74 ms TenGigE0-5-0-0.GW8.NYC4.ALTER.NET [152.63.21.65]

7 123 ms 127 ms 121 ms tinet-gw.customer.alter.net [152.179.72.122]

treykaws

"01-23-2013 01:00 PM - edited 01-23-2013 01:08 PM

I just wanted to bump this with my own new latency issue.
In the NJ area, on FiOS and have had a recent jump.

My specific test case is a game called Mechwarrior Online. The game servers are located in eastern Canada.

Average ping for the past 6 months has been steady between 40-60ms.

Now average ping is consistenly 120-130. This is within the week around Christmas as many other reported.

This was tested with two machines inside the home one via WiFi one via CAT6.

Happy to conduct and post tracert's but It seems that further verifying the issue is not whats needed at this point.

if anyone wants me run one to a specific server as another datapoint let me know.

Thanks, and happy to be of assistance to this cause."

I'm from NJ (South Jersey) and I have the exact same problemas GYOCHUM. By chance did you figure out what is causing this? I'm assuming it's 100% on Verizon's end.

How would I go about running a tracert so I can help provide some sort of statistic as well?

gkgough

if you can find the name of the server your game is hosted on, open command prompt and type "tracert 'server name'"

treykaws

Right now, 2:30am EST, pings are normal and as they should be. But throughout the day and evening they increase to about 35-45% higher than what they should be. It only recently (a month ago) started doing this.

What do you mean find the name of the server my game is hosted on?

gyochum1

Trey,

Go to your start button (assuming windows) and type in "command" and hit enter.

If you are now at a command prompt type in:

tracert www.mwomercs.com

Look for the server that first causes a major change in the response time.

Bsylvia1

Having the same issues here. I have Verizon DSL, but my routing has not been changed. I just ran a tracert and I am still getting the put through "tinet-gw.customer.alter.net". Not sure if resetting my modem will make a difference or not and I can not find the live support option on the website. I'm at a loss on what can be done at this point.

Bsylvia1

I'm still having this issue. I called tech support and all they attempted to do was switch the channel I was on at the central office. While this did improve my ping slightly, I still am not back to where I was 3 weeks ago. Sometime around Jan 15th my pings more than doubled and never recovered. Time of day has no effect on this issue either. I went from pings between 60-80ms to 160-220ms. Now after calling tech support I'm seeing 130-150ms, however the server I'm connecting to is in Toronto and I live in Massachusetts.

As a comparison I've run a tracert to the same server from my work computer which is less than a mile from my house. Here is the results:

C:\Users\Bryan>tracert 70.42.29.65

Tracing route to 70.42.29.65 over a maximum of 30 hops

1 1 ms 1 ms 1 ms static-72-248-212-9.ct.onecommunications.net [72.248.212.9]
2 9 ms 9 ms 9 ms lo0.wrcsma05-gsr-03.onecommunications.net [64.69.99.131]
3 12 ms 11 ms 12 ms ge-0-1-0-5.wrcmankgs04.onecommunications.net [64.69.98.50]
4 149 ms 150 ms 16 ms te-3-1.nycnyzrr101.onecommunications.net [64.69.99.238]
5 16 ms 16 ms 16 ms 165.121.238.178
6 16 ms 16 ms 22 ms ge-5-0-1.ar5.jfk1.gblx.net [64.214.141.253]
7 30 ms 30 ms 39 ms te8-1-10G.asr1.YYZ1.gblx.net [67.16.131.186]
8 29 ms 29 ms 29 ms 64.210.12.62
9 29 ms 30 ms 30 ms border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.196]
10 32 ms 31 ms 33 ms 70.42.29.65

Trace complete.

I'm pretty sure the service at work is Earthlink and I can ping the same server in Toronto at 30ms??? Verizon really needs to take a look at "tinet-gw.customer.alter.net" as this is where the horrific latency starts. I'll post a tracert from my home pc later today. Please fix this Verizon!

clampdown26

Also having major ping issues since mid-January. The primary game I play has seen pings more than double from its normal 72ms to 180-200ms.

It returns to normal, slowly, after 10pm EST. All other hours has this ping issue, and many others have had this issue, with the same node. for example, my tracert:

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\> tracert www.mwomercs.com

Tracing route to www.mwomercs.com [70.42.29.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms dslrouter.westell.com [192.168.1.1]
2 40 ms 40 ms 41 ms 10.29.29.1
3 42 ms 42 ms 41 ms P15-1.SCTNPA-LCR-01.verizon-gni.net [130.81.197.
64]
4 46 ms 46 ms 46 ms xe-4-1-8-0.NWRK-BB-RTR1.verizon-gni.net [130.81.
209.194]
5 86 ms 56 ms 56 ms xe-1-0-2-0.NY5030-BB-RTR1.verizon-gni.net [130.8
1.23.184]
6 59 ms 60 ms 59 ms 0.so-7-0-0.XT1.NYC4.ALTER.NET [152.63.17.89]
7 60 ms 59 ms 63 ms TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129
]
8 151 ms 152 ms 152 ms tinet-gw.customer.alter.net [152.179.72.122]
9 186 ms 185 ms 178 ms xe-4-3-0.tor10.ip4.tinet.net [89.149.184.110]
10 181 ms 185 ms 187 ms internap-gw.ip4.tinet.net [77.67.70.94]
11 217 ms 209 ms 205 ms border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.1
96]

Look at the jump from 7 to 8. Just look at it. That tinet-gw.customer.alter.net is really horrendous and the pingsj ust get worse from there. This is the same node everyone's been complaining about, and alter.net is Verizon's backbone. Why is this happening, and is there anything we can do?

matt_g

I'm in the Dallas/FW, TX area and having similar issues. Tracerts with loss starting 3-4 hops out, high ping times...extremely frustrating. Also started around mid-january.

keopi

I'm sure a lot of us who are experiencing this problem are frustrated. Verizon FiOS at its best in 2013 with high latency issues at all times while we pay in full monthly with no compensations.

helloterence

It has been over two weeks since this problem has arised and there has been no feedback from the department that should be handling this. Bit ridiculous if you ask me.

@clampdown26 wrote:
Also having major ping issues since mid-January. The primary game I play has seen pings more than double from its normal 72ms to 180-200ms.
It returns to normal, slowly, after 10pm EST. All other hours has this ping issue, and many others have had this issue, with the same node. for example, my tracert:
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation. All rights reserved.

C:\Users\> tracert www.mwomercs.com

Tracing route to www.mwomercs.com [70.42.29.74]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms dslrouter.westell.com [192.168.1.1]
2 40 ms 40 ms 41 ms 10.29.29.1
3 42 ms 42 ms 41 ms P15-1.SCTNPA-LCR-01.verizon-gni.net [130.81.197.
64]
4 46 ms 46 ms 46 ms xe-4-1-8-0.NWRK-BB-RTR1.verizon-gni.net [130.81.
209.194]
5 86 ms 56 ms 56 ms xe-1-0-2-0.NY5030-BB-RTR1.verizon-gni.net [130.8
1.23.184]
6 59 ms 60 ms 59 ms 0.so-7-0-0.XT1.NYC4.ALTER.NET [152.63.17.89]
7 60 ms 59 ms 63 ms TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129
]
8 151 ms 152 ms 152 ms tinet-gw.customer.alter.net [152.179.72.122]
9 186 ms 185 ms 178 ms xe-4-3-0.tor10.ip4.tinet.net [89.149.184.110]
10 181 ms 185 ms 187 ms internap-gw.ip4.tinet.net [77.67.70.94]
11 217 ms 209 ms 205 ms border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.1
96]

Look at the jump from 7 to 8. Just look at it. That tinet-gw.customer.alter.net is really horrendous and the pingsj ust get worse from there. This is the same node everyone's been complaining about, and alter.net is Verizon's backbone. Why is this happening, and is there anything we can do?

While you are having the tinet routing issue, as you're on DSL we can also cut down on some of the latency by getting your line moved from Interleaved to FastPath if it is running clean enough. That can get your second hop latency from where you're at now, down to about 15ms to even as low as 6ms.

clampdown26

Any pointers you can give me Smith to making that happen? I talked with one of the agents in chat and they seemed to have no idea what I was talking about!

Darn2

Its funny stumbling upon threads like these because verizon really is not going to do anything about it. I started complaining about this problem 3 years ago. I'm pretty sure they flagged my account so tech support ignores me now. I can pretty consistently go to pingtest.net and get 100ms+ latency with 10-20ms jitter. You all are crying on deaf ears. Online gaming probably makes up about 10% of their users so they dont really care. I can go to to the same server 3 days in a row and ill get a different ping each time, usually varying degrees of horrible. Here is a pingtest from yesterday:

and a traceroute

@Darn wrote:
Its funny stumbling upon threads like these because verizon really is not going to do anything about it. I started complaining about this problem 3 years ago. I'm pretty sure they flagged my account so tech support ignores me now. I can pretty consistently go to pingtest.net and get 100ms+ latency with 10-20ms jitter. You all are crying on deaf ears. Online gaming probably makes up about 10% of their users so they dont really care. I can go to to the same server 3 days in a row and ill get a different ping each time, usually varying degrees of horrible. Here is a pingtest from yesterday:
and a traceroute

Ouch.

To argue to that, I know Verizon routinely touts in their marketing that their service is THE BEST for gaming simply because it's Fiber. I suppose if they wanted to live up to that they would take care to fix the latency and jitter issues. But granted as you've stated, gamers tend to be the unfortunate minority who pay more money than usual for higher capacity service (Of course, more capacity !=faster in some cases) and Verizon also has no motivation to fix it since they are consumer, not business grade circuits.

Anways as to the question as to what sort of pointers can be given, I've got someone currently looking into it and i'm waiting to hear back from them as to what they can find. A quick look by them isn't turning up anything but they'll investigate further on Monday. They were trying to see where I am routing through entirely, however Verizon's system keeps complaining about something and it wants to auto-fix something that will genuinely hose my service up if applied.

szymiszymi

If it stays like this I'm gonna switch my provider. Why should I stick with company that doesn't care at all ?

click4dylan

I've had this problem myself ever since FIOS was ever installed back in 2007. I can't connect to any server hosted in the state of Florida because it always routes through georgia. The screenshots below are actually on a particularly good night. It's usually much worse. And alter.net always has ping spikes basically 24/7 and i've even recorded it going up to 1000+ms before. I spent countless hours on the phone dealing with tier 1 support and never receiving call backs. They say it's not their problem since Alter.net is not owned by verizon which it clearly is. Every single connection I make to any server whether it's in antarctica or not will route through alter.net in atlanta, georgia before it goes anywhere else. This adds 30ms to every connection I make.

Here's an example to a server that should give me less than 5ms latency. It's RIDICULOUS! Not only does it route through tampa and then atlanta first, it then goes back down to MIAMI, and then BACK to tampa! :

JamesCav

There is definitely something going on at Verizon's back end. I've been running a ping test over the past four weeks, and you can clearly see that the issue started about two weeks ago.

In the charts below, week 00 corresponds to the first week of January. Look at Week 03 onwards and you'll see the disaster. All ping tests are from a server in Manhattan. The problem appears to be concentrated in the East Coast, but it may be more widespread. I've included ping test reports for other FiOS users as well.

New York, NY

Needham, MA

Camden, NJ

Carnegie, PA

There's definititely some some oversubscription going on backbone. The problem seems to be primarily in the East Coast. I did some investigating and haven't seen these issues on the West Coast.

Come on Verizon! You've touted your superior FiOS service, and now this? Maybe it's for the better. A few days ago RCN called and offered me a 50/6 package for $30 per month, with no equipment fees or contract. With the FiOS network going downhill, it makes my decision to switch that much easier.

click4dylan

You guys make it seem like your routing issues have lasted forever, but us in FL have had pitiful routing and a laggy backbone since it was put in the ground! When other people from FL connect to me in-game they ask me why their ping is so ridiculously high when they're connected to someone who lives less than 100 miles from them. Fix the routing in florida, or at MINIMUM fix the Atlanta backbone because I'm really sick and tired of websites randomly not loading for up to 30 seconds at a time (including verizon.com!), and ping to everything spiking up to 1000+ every couple of seconds for a split second

http://forums.verizon.com/t5/FiOS-Internet/Many-websites-load-painfully-slow/td-p/536875

@click4dylan wrote:
You guys make it seem like your routing issues have lasted forever, but us in FL have had pitiful routing and a laggy backbone since it was put in the ground! When other people from FL connect to me in-game they ask me why their ping is so ridiculously high when they're connected to someone who lives less than 100 miles from them. Fix the routing in florida, or at MINIMUM fix the Atlanta backbone because I'm really sick and tired of websites randomly not loading for up to 30 seconds at a time (including verizon.com!), and ping to everything spiking up to 1000+ every couple of seconds for a split second

I've unfortunately have complained about the lack of local routing to servers for a while. I'm in the same boat as you but I have to route through New York for everything. Too many eggs in one backet. Meanwhile, it's convenient to go from your local area and take any one of 5 different routes to a nearby server, maybe not always cheap in a network architect view, but it sure cuts down congestion from a common route.

helloterence

Does anyone here use Twitter? If what he says is true, I say a number of us should start spamming this thread over to @VerizonSupport. It's been almost a month with no signs of this issue being fixed, let alone Verizon even knowing about it beyond these community boards.

Johnny_Utah

Situation is again horrible here tonight again. SPeeds cut down to 12Mb/s however the latency and packet loss is so bad, I cannot do anything but comment on forums. I have HAD it with this sub-par service.

Long Beach, CA 75/35 FIOS

jk1112

I tweeted them a message tonight. And will continue to do so every day until it is addressed.

cdegroat82

I myself am having the same issues. Tonight is particularly bad, as I cant even carry a basic voice chat conversation.

Here are some of my traces:

traceroute to 173.2.198.87 (173.2.198.87), 30 hops max, 60 byte packets
1 10.58.1.1 (10.58.1.1) 1.493 ms 2.177 ms 2.835 ms
2 10.144.1.1 (10.144.1.1) 0.415 ms 0.428 ms 0.411 ms
3 Wireless_Broadband_Router.home (192.168.1.1) 2.142 ms 2.701 ms 4.812 ms
4 L100.NYCMNY-VFTTP-48.verizon-gni.net (71.167.106.1) 11.406 ms 11.358 ms 11.886 ms
5 G0-1-1-3.NYCMNY-LCR-22.verizon-gni.net (130.81.188.72) 13.924 ms 13.605 ms 13.582 ms
6 ae0-0.NY5030-BB-RTR1.verizon-gni.net (130.81.209.118) 34.274 ms * 20.606 ms
7 0.xe-3-3-0.BR2.NYC4.ALTER.NET (152.63.23.133) 95.534 ms 95.497 ms 95.499 ms
8 204.255.168.90 (204.255.168.90) 13.373 ms 13.368 ms 13.302 ms
9 cr1-te-0-7-3-0.newyork.savvis.net (204.70.224.254) 56.858 ms 56.864 ms 56.850 ms
10 msr1-tengig-0-0-0-0.NewYork.savvis.net (204.70.196.110) 19.184 ms 19.121 ms 21.417 ms
11 cr2-tengig-0-15-2-0.NewYork.savvis.net (204.70.196.113) 91.327 ms 96.139 ms 87.757 ms
12 208.175.101.66 (208.175.101.66) 37.672 ms 37.696 ms 46.210 ms
13 * * *
14 * * *
15 dstswr2-ge3-16.rh.stjmny.cv.net (167.206.39.134) 16.235 ms dstswr1-ge3-16.rh.stjmny.cv.net (167.206.39.130) 90.036 ms 89.664 ms
16 * * *

And a reverse trace from the far end:

Tracing route to {edited for privacy}

over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms HomePC [192.168.1.1]

2 * * * Request timed out.

3 7 ms 5 ms 5 ms 67.59.238.133

4 8 ms 8 ms 9 ms rtr4-ge1-9.mhe.hcvlny.cv.net [167.206.39.133]

5 11 ms 9 ms 9 ms 451be0b9.cst.lightpath.net [65.19.99.185]

6 * * * Request timed out.

7 10 ms 9 ms 9 ms xe-9-1-3.edge2.Newark1.Level3.net [4.30.130.237]

8 11 ms 9 ms 9 ms ae-31-51.ebr1.Newark1.Level3.net [4.69.156.30]

9 11 ms 10 ms 10 ms ae-2-2.ebr1.NewYork1.Level3.net [4.69.132.97]

10 9 ms 9 ms 9 ms ae-4-4.ebr1.NewYork2.Level3.net [4.69.141.18]

11 * * * Request timed out.

12 20 ms * 24 ms po6-20G.ar6.NYC1.gblx.net [208.51.134.113]

13 10 ms 12 ms 12 ms 0.ae11.BR2.NYC4.ALTER.NET [204.255.168.189]

14 * 10 ms 12 ms 0.so-5-0-0.NY5030-BB-RTR2.ALTER.NET [152.63.10.4

15 * * * Request timed out.

16 25 ms * 18 ms pool-96-232-85-72.nycmny.fios.verizon.net

Trace complete.

Trace to a different destination:

trace silicon.typefrag.com
traceroute to silicon.typefrag.com (74.217.195.66), 30 hops max, 60 byte packets
1 10.58.1.1 (10.58.1.1) 7.625 ms 8.067 ms 8.635 ms
2 10.144.1.1 (10.144.1.1) 0.655 ms 0.674 ms 0.668 ms
3 Wireless_Broadband_Router.home (192.168.1.1) 2.331 ms 2.368 ms 2.350 ms
4 L100.NYCMNY-VFTTP-48.verizon-gni.net (71.167.106.1) 14.061 ms 9.365 ms 9.375 ms
5 G0-1-1-4.NYCMNY-LCR-21.verizon-gni.net (130.81.146.230) 18.201 ms 17.538 ms 20.217 ms
6 130.81.151.228 (130.81.151.228) 95.311 ms 93.620 ms 93.605 ms
7 * * *
8 204.255.168.90 (204.255.168.90) 52.169 ms 48.335 ms 50.653 ms
9 cr1-te-0-7-3-0.newyork.savvis.net (204.70.224.254) 50.066 ms 50.092 ms 50.085 ms
10 cr2-pos0-0-0-0.sanfrancisco.savvis.net (204.70.192.90) 98.690 ms 98.703 ms 98.695 ms
11 er2-xe-3-1-0.SanJoseEquinix.savvis.net (204.70.200.197) 152.797 ms 152.790 ms 152.759 ms
12 er1-te-1-3.SanJoseEquinix.savvis.net (204.70.198.141) 99.004 ms 206.28.98.69 (206.28.98.69) 117.008 ms 117.026 ms
13 208.175.172.10 (208.175.172.10) 139.097 ms 139.104 ms 139.086 ms
14 border1.te7-1-bbnet1.sfo002.pnap.net (63.251.63.1) 105.070 ms * border1.te8-1-bbnet2.sfo002.pnap.net (63.251.63.65) 120.573 ms
15 74.217.195.66 (74.217.195.66) 133.900 ms 103.103 ms 103.169 ms

This problem also seems to come and go. I also have packet loss pinging some hops directly. And one my trace from my desktop vs my unix machine, behind the same network.

Tracing route to silicon.typefrag.com [74.217.195.66]
over a maximum of 30 hops:

1 * * * Request timed out.
2 <1 ms <1 ms <1 ms 10.144.1.1
3 <1 ms <1 ms <1 ms 192.168.1.1
4 7 ms 9 ms 9 ms L100.NYCMNY-VFTTP-48.verizon-gni.net [71.167.106
.1]
5 11 ms 15 ms 14 ms G0-1-1-4.NYCMNY-LCR-21.verizon-gni.net [130.81.1
46.230]
6 35 ms 33 ms 18 ms 130.81.151.228
7 * * 114 ms 0.xe-3-2-0.BR2.NYC4.ALTER.NET [152.63.20.213]
8 49 ms 50 ms 51 ms 204.255.168.90
9 45 ms 48 ms 50 ms cr1-te-0-7-3-0.newyork.savvis.net [204.70.224.25
4]
10 101 ms 95 ms 96 ms cr2-pos0-0-0-0.sanfrancisco.savvis.net [204.70.1
92.90]
11 129 ms 93 ms 92 ms er2-xe-3-1-0.SanJoseEquinix.savvis.net [204.70.2
00.197]
12 102 ms 103 ms 100 ms er1-te-1-4.SanJoseEquinix.savvis.net [204.70.198
.145]
13 136 ms * 118 ms 208.175.172.10
14 97 ms 99 ms 116 ms border1.te7-1-bbnet1.sfo002.pnap.net [63.251.63.
1]
15 * 131 ms 129 ms 74.217.195.66

Trace complete.

gmanhq

Just found this forum thread. Same problem I just posted about here:

I did tweet verizonsupport today to for whatever that is worth.

Looks like, for me at least, it could be a level3.net problem but still..our provider ( verizon) should be helping it's paying customers!

thor11

Anyone get an answer on:

8 173 ms 129 ms 128 ms tinet-gw.customer.alter.net [152.179.72.122]

This hop is always the slowest on my tracert. Before January 2013 there was no 173-128 ms problem. It was more like 50ms.