Routing / Latency issues

[chzoom]

Is there anyone I can speak to about routing/latency issues? I recently switched to fios and since have had nothing but terrible lag issues when gaming. The issues occur just a hop or 2 off the verizon network. I can provide traceroutes, but primarily I am looking to speak with someone other than the tier1 folks answering the phones.

[AnnieS1]

Just a friendly reminder, this is a forum where users help other users.

If you'd like to get the ball rolling with help on your issue, this is a good place to start:

http://www.verizon.com/contactus

Choose “Live Chat.”

If a chat agent is available to assist you, the chat link will become live after the page is fully loaded.

Take care!

AnnieS

[Hubrisnxs]

Verizon really doesn't have Tier II or Tier III tech's that talk to customers.

Those tech's get their reports from tier I only, and then they investigate their issue.

Those tech's are designed to talk with other Tech's and other departments.  So their network tech's do talk to customers occasionally, but that is after a report has been sent to them, and they are done looking at it, and they need more info.  They may call you personally to get that info.  other wise they are fielding calls from Field technicians in the street that are on trouble reports or installs, or they are working with Internal Departments to resolve different issues. 

So you have to go through tier I.

The only kindasorta way you can get around that is by doing the twitter support.  They are pretty exceptional, but otherwise you can post here, and there are a lot of very experienced users of varying backgrounds that can help you, so feel free to post your traceroutes here.  XXX out your actual IP address for safety, and let us know if you are hardwired or wirless when doing your tracert and what bandwidth or kind of traffic you are pushing when you do your tests.

Keep in mind gaming/movie or media streaming and bittorrent traffic when replying. 

[chzoom]

thanks for the information everyone. I know how these things can go (I myself used to work for a broadband ISP). This type of issue is difficult to get any help with being the issue isn't really with verizons network. However, where verizon routes this traffic to can possibly be controlled. All of my issues appear be to on alter.net which is where verizon routes this specific traffic.

- Used to ping around 20-30ms

- Now my ping times are around 110-150ms

- The latency goes from 20ms to 110-150 within 1 hop of verizons network

- Connection is ethernet based (not wireless)

I have included 2 different traceroutes. One is to a server located in VA and the other in DC. It is clear the hop that my latency starts to occur is on alter.net's network. To add to this, I did also change the MAC of my verizon router to get a new DHCP lease in hopes that the routing might change, but it took the same path off verizons network.

******** VA *********

tracert 64.34.186.94

Tracing route to . [64.34.186.94]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home
2 6 ms 6 ms 6 ms lo0-100.PRVDRI-VFTTP-303.verizon-gni.net
3 24 ms 7 ms 6 ms G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
4 62 ms 15 ms 7 ms ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
5 39 ms 14 ms 14 ms ge-7-1-0-0.NY5030-BB-RTR2.verizon-gni.net [130.81.19.33]
6 18 ms 18 ms 17 ms 0.so-0-0-0.XT2.NYC4.ALTER.NET [152.63.17.21]
7 17 ms 22 ms 20 ms TenGigE0-6-2-0.GW8.NYC4.ALTER.NET [152.63.21.117]
8 115 ms 118 ms 119 ms tinet-gw.customer.alter.net [152.179.72.122]
9 125 ms 127 ms 132 ms xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]
10 130 ms 134 ms 134 ms peer1-gw.ip4.tinet.net [173.241.128.102]
11 136 ms 137 ms 134 ms 10ge-xe-1-0-1.wdc-sp225-sbdis-1.peer1.net [216.187.120.114]
12 135 ms 134 ms 132 ms . [64.34.186.94]

Trace complete.

******** DC **********

tracert 50.97.59.34

Tracing route to 50.97.59.34-static.reverse.softlayer.com [50.97.59.34]
over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home
2 6 ms 5 ms 5 ms lo0-100.PRVDRI-VFTTP-303.verizon-gni.net
3 6 ms 8 ms 6 ms G0-3-4-6.PRVDRI-LCR-21.verizon-gni.net [130.81.191.240]
4 11 ms 12 ms 11 ms ae4-0.NY5030-BB-RTR1.verizon-gni.net [130.81.209.54]
5 13 ms 12 ms 14 ms 0.xe-10-1-0.BR1.NYC1.ALTER.NET [152.63.18.225]
6 13 ms 12 ms 12 ms ae11.edge2.NewYork.Level3.net [4.68.62.41]
7 12 ms 13 ms 13 ms vlan51.ebr1.NewYork2.Level3.net [4.69.138.222]
8 12 ms 13 ms 13 ms ae-4-4.ebr1.NewYork1.Level3.net [4.69.141.17]
9 13 ms 21 ms 24 ms ae-91-91.csw4.NewYork1.Level3.net [4.69.134.78]
10 20 ms 13 ms 13 ms ae-4-90.edge3.NewYork1.Level3.net [4.69.155.209]
11 15 ms 15 ms 15 ms te2-5.bbr01.tl01.nyc01.networklayer.com [4.26.34.22]
12 13 ms 12 ms 13 ms ae7.bbr01.tl01.nyc01.networklayer.com [173.192.18.176]
13 25 ms 20 ms 21 ms ae1.bbr01.eq01.wdc02.networklayer.com [173.192.18.156]
14 24 ms 25 ms 23 ms ae0.dar02.sr01.wdc01.networklayer.com [173.192.18.203]
15 24 ms 26 ms 24 ms po2.fcr03.sr02.wdc01.networklayer.com [208.43.118.159]
16 27 ms 22 ms 21 ms 50.97.59.34-static.reverse.softlayer.com [50.97.59.34]

I have tried to call in and get assistance but the tech couldn't even provide me an email address to send my traceroutes to. If there is absolutely no way to resolve this issue because it is off verizon's network what course of action can I take? The reason I have the services is for gaming and the connection to my specified servers are unstable making it quite difficult to play.

I would like to note, though I am having these issues - my download speeds, streaming, etc are all perfect and I couldn't be happier with that.

[Hubrisnxs]

Alter.net is MCI's old backbone network, and Verizon Bought them a year or two ago, so they may be able to help still.

Try the twitter guys, that might be more helpful, or you can do a smart who is look up on the routers that you are having trouble with, and email their NOC or technical contact. 

Additionally you should look at these QOS settings - that might help you as well. 

Optimal Actiontec MI424WR router set up for Xbox Black Ops 2

[chzoom]

I have tried playing with QoS but it doesn't make much of a difference. I also know for a fact that is effecting more than just fios customers in my area. There is a few forum posts around this on certain gaming sites I am on, all reporting similar latency issues for about 2 weeks now to the same servers.

[chzoom]

Took a few calls into Verizon to actually get someone that understood what was going on. Most of the tier 1 techs didn't know what traceroute or latency was and were not very helpful in allowing me to provide the information. I had to get a supervisor who was super helpful and assisted by remoting to my machine, running the same traceroutes that I provided here and forwarded them to Tier2 (at least I hope so). I'll respond here once I have more information.

I did want to add that this issue is even happening from some servers I have out of TX and even free ones I have tested from

Trace to tinet.net not going through tinet-gw.customer.alter.net - sourcing off verizon network

Hop

(ms)

(ms)

(ms)

IP Address

Host name

1

1

0

0

8.9.232.73

xe-5-3-0.edge3.dallas1.level3.net

2

0

0

0

4.69.145.205

ae-4-90.edge4.dallas3.level3.net

3

3

3

3

77.67.71.221

xe-8-1-2.dal33.ip4.tinet.net

4

39

39

39

89.149.183.70

xe-1-2-0.nyc41.ip4.tinet.net

5

42

39

39

141.136.111.14

xe-0-0-0.was14.ip4.tinet.net

Trace to tinet.net going through tinet-gw.customer.alter.net - sourced from my machine on the fios network

  2     6 ms     6 ms     6 ms  lo0-100.PRVDRIxxxxxxx
  3     6 ms     6 ms     6 ms  G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
  4     8 ms    42 ms    38 ms  ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
  5    65 ms   128 ms    29 ms  ge-1-2-0-0.NY325-BB-RTR2.verizon-gni.net [130.81.17.24]
  6    17 ms    17 ms    17 ms  0.so-4-0-2.XT2.NYC4.ALTER.NET [152.63.1.57]
  7    19 ms    19 ms    17 ms  TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129]
  8   111 ms   107 ms   109 ms  tinet-gw.customer.alter.net [152.179.72.122]
  9   117 ms   129 ms   108 ms  xe-4-1-0.nyc40.ip4.tinet.net [89.149.183.113]
 10   175 ms   107 ms   106 ms  xe-0-0-0.nyc41.ip4.tinet.net [89.149.180.158]
 11   125 ms   163 ms   120 ms  xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]

Trace complete.

These traceroutes were done from different locations to kinda pinpoint the issue, but you can clearly see that as long as it doesn't touch the tinet-gw.customer.alter.net router there is no latency present.

[Hubrisnxs]

 tinet-gw.customer.alter.net    

So the red here indicates the company name

the blue tells us that it's a customer circuit

and then the orange tells us it belongs to MCI/Alter.net now owned by Verizon

So with that info, I think if you follow this with Verizon  - you will find that Tinet-gw just needs to buy more bandwidth.

I think tinet is just a transport provider and I am not sure how they relate with your gaming servers.  Maybe they are just a transport company contracted, or maybe that is the backbone provider for your game server and their peering point with Verizon.   

I am going to be very interested to see how verizon's Tier 2 responds to this.  If they have anything they can do other than tell the provider to buy more bandwidth or if there is an actual problem they can adjust and fix.

---

@chzoom wrote:

Took a few calls into Verizon to actually get someone that understood what was going on. Most of the tier 1 techs didn't know what traceroute or latency was and were not very helpful in allowing me to provide the information. I had to get a supervisor who was super helpful and assisted by remoting to my machine, running the same traceroutes that I provided here and forwarded them to Tier2 (at least I hope so). I'll respond here once I have more information.

I did want to add that this issue is even happening from some servers I have out of TX and even free ones I have tested from

Trace to tinet.net not going through tinet-gw.customer.alter.net - sourcing off verizon network

Hop

(ms)

(ms)

(ms)

IP Address

Host name

1

1

0

0

8.9.232.73

xe-5-3-0.edge3.dallas1.level3.net

2

0

0

0

4.69.145.205

ae-4-90.edge4.dallas3.level3.net

3

3

3

3

77.67.71.221

xe-8-1-2.dal33.ip4.tinet.net

4

39

39

39

89.149.183.70

xe-1-2-0.nyc41.ip4.tinet.net

5

42

39

39

141.136.111.14

xe-0-0-0.was14.ip4.tinet.net

Trace to tinet.net going through tinet-gw.customer.alter.net - sourced from my machine on the fios network

  2     6 ms     6 ms     6 ms  lo0-100.PRVDRIxxxxxxx
  3     6 ms     6 ms     6 ms  G0-3-4-6.PRVDRI-LCR-22.verizon-gni.net [130.81.191.242]
  4     8 ms    42 ms    38 ms  ae2-0.BOS-BB-RTR2.verizon-gni.net [130.81.209.60]
  5    65 ms   128 ms    29 ms  ge-1-2-0-0.NY325-BB-RTR2.verizon-gni.net [130.81.17.24]
  6    17 ms    17 ms    17 ms  0.so-4-0-2.XT2.NYC4.ALTER.NET [152.63.1.57]
  7    19 ms    19 ms    17 ms  TenGigE0-7-2-0.GW8.NYC4.ALTER.NET [152.63.21.129]
  8   111 ms   107 ms   109 ms  tinet-gw.customer.alter.net [152.179.72.122]
  9   117 ms   129 ms   108 ms  xe-4-1-0.nyc40.ip4.tinet.net [89.149.183.113]
 10   175 ms   107 ms   106 ms  xe-0-0-0.nyc41.ip4.tinet.net [89.149.180.158]
 11   125 ms   163 ms   120 ms  xe-0-0-0.was14.ip4.tinet.net [141.136.111.14]

Trace complete.

These traceroutes were done from different locations to kinda pinpoint the issue, but you can clearly see that as long as it doesn't touch the tinet-gw.customer.alter.net router there is no latency present.

---

[photonslowgame]

I am experiencing the identical issue (as well as another friend with Fios in the NJ/Philly area). Game server in question for me is in Toronto, only ~300 miles away, and this routing hardly seems ideal to begin with, but this extra 100 MS at " tinet-gw.customer.alter.net" has been the real killer since Tuesday 15th. I can only hope it is being caused by a substantial upgrade or repair that is in progress.

During business hours on monday I will surely attempt some phone calls as well, but I am less than optimistic. I would hate to be forced to move to comcast over one bad hop on one bad network; I have never had any problem with my fios last longer than a few hours before.

C:\Users\******>tracert 70.42.29.65

Tracing route to 70.42.29.65 over a maximum of 30 hops

  1     1 ms     1 ms     1 ms  Wireless_Broadband_Router.home xxxxxxx
  2     8 ms     7 ms     5 ms  L100.PITBPAxxxxx

  3     8 ms     9 ms     8 ms  G0-1-1-6.PITBPA-LCR-22.verizon-gni.net [130.81.212.58]
  4    15 ms    18 ms    43 ms  xe-0-1-6-0.PHIL-BB-RTR2.verizon-gni.net [130.81.209.76]
  5    19 ms    18 ms    19 ms  xe-9-1-4-0.NY5030-BB-RTR2.verizon-gni.net [130.81.23.70]
  6    20 ms    18 ms    20 ms  0.so-0-0-0.XT2.NYC4.ALTER.NET [152.63.17.21]
  7    21 ms    22 ms    23 ms  TenGigE0-5-0-0.GW8.NYC4.ALTER.NET [152.63.21.65]

  8   115 ms   111 ms   136 ms  tinet-gw.customer.alter.net [152.179.72.122]

  9   153 ms   150 ms   150 ms  xe-4-3-0.tor10.ip4.tinet.net [89.149.184.110]
 10   141 ms   139 ms   144 ms  internap-gw.ip4.tinet.net [77.67.70.94]
 11   134 ms   136 ms   232 ms  border1.te9-1-bbnet2.tor001.pnap.net [70.42.24.196]
 12   145 ms   136 ms   130 ms  70.42.29.65

Trace complete.

[chzoom]

I agree,

I have now placed 4 calls into support (last 2 being updates on my ticket)

I am not very happy right now as my ticket was 'closed' due to missing 1 call from a tier 1 tech asking if I was still having trouble. However, he indicated (while i was on the phone for 30 minutes) that he spoke with a tier 2 tech and they have from what he says recognized this as a widescale issue and they are working on it. He also let me know that I would receive a call back in the next 24 hours. 

I basically purchased a 1 month proxy to redirect my traffic to induce some different routing which will hold me over for now. This is obviously not reasonable as I already pay a decent premium for my Verizon service. I have faith in their network engineers to resolve the issue, but I have already been having this issue for a month =\

[photonslowgame]

What proxy service did you select? I am also considering doing that. I very much appreciate that you keep posting here chzoom,  your dedication to the phone is admirable, I'm not sure I have the patience for that.

[chzoom]

I only have patience because I do this for a living myself. I know they can't just snap their fingers and fix it, especially if its a bandwidth or throughput issue with a core circuit or equipment.

If you want to private message me I can give you the service I use right now which works perfect(not exactly sure on the TOS of the boards, probably not a good idea to post the link for public view 🙂 )

[rabid_animal]

Having the same issue as well for the last month now, about a week before Christmas. Makes online gaming to and thru DC/Virginia almost unplayable. I got so fed up I also looked into a tunneling/ proxy service. I got a month's subscription which I hope I don't need to continue to use. Sometimes the spike on hop 8 goes to 175+ Thanks for the posts.

1 <1 ms <1 ms <1 ms Wireless_Broadband_Router.home [192.168.1.1]
2 5 ms 5 ms 5 ms L100.BLTMMD-VFTTP-61.verizon-gni.net [71.179.183.1]
3 4 ms 4 ms 4 ms G11-0-5-461.BLTMMD-LCR-04.verizon-gni.net [130.81.146.32]
4 8 ms 8 ms 8 ms so-6-0-3-0.PHIL-BB-RTR2.verizon-gni.net [130.81.28.82]
5 12 ms 11 ms 12 ms xe-9-1-4-0.NY5030-BB-RTR2.verizon-gni.net [130.81.23.70]
6 12 ms 12 ms 13 ms 0.so-0-0-0.XT2.NYC4.ALTER.NET [152.63.17.21]
7 12 ms 16 ms 15 ms TenGigE0-7-0-0.GW8.NYC4.ALTER.NET [152.63.22.45]
8 92 ms 124 ms 90 ms tinet-gw.customer.alter.net [152.179.72.122]
9 96 ms 95 ms 96 ms xe-1-3-1.was14.ip4.tinet.net [141.136.108.154]
10 105 ms 105 ms 104 ms peer1-gw.ip4.tinet.net [173.241.128.102]
11 102 ms 99 ms 95 ms 10ge-xe-1-0-1.wdc-sp225-sbdis-1.peer1.net [216.187.120.114]
12 107 ms 97 ms 94 ms wgn53.wolfservers.com [64.34.179.229]

Trace complete.

[chzoom]

well I got a call from a supervisor - Long story short they claim the router isn't owned by Verizon and can provide me no additional assistance. As much as I hate to, I now need to find a way out of my contract because I can't continue to figure out ways around their issue. This is a wide scale issue that more than just myself or a couple people are experiencing 😐

[photonslowgame]

I'm really sorry to hear that; especially if the server is actually owned by Verizon as an earlier post indicates.

[photonslowgame]

I attempted to speak with technical support over the online chat earlier, but that was unproductive. After explaining my issue this was the response I recieved:

"I am running a test on the line and is coming good. Your issues is beyond our support 
I apologize but we do not provide support for this issue. In an effort to educate you on what is supported, please visit http://support.verizon.com/boundary to see what support options are available. Links to the available support options are provided alongside the type of issue making it easy to use this resource.

However, we do have a group that can help with both hardware and software upgrades and issues. For any assistance with your personal equipment you can sign up for our Expert Care http://surround.verizon.net/Shop/Expertcare/default.aspx or contact your computer/device manufacturer. To add this feature by phone, please call Sales at 1-800-VERIZON (1-800-837-4966) and they will be able to add this feature to your account."

Of course I balked at being told that I would need an additional -paid- service to even talk to someone about the issue, so I asked for a supervisor. His response was

" She explained to me your issue and unfortunately that is out of our scope, here at the Technical Support Department. Indeed, we do have the PREMIUM TECH Support that handles this type of issues. Please give them a call they have a one time free assistance for this matter."

I doubt its worth my time to call and use my "one time free assistance" after you've done all this leg work.

[helloterence]

Does anyone here use Twitter? If what he says is true, I say a number of us should start spamming this thread over to @VerizonSupport. It's been almost a month with no signs of this issue being fixed, let alone Verizon even knowing about it beyond these community boards.

[Johnny_Utah]

Situation is again horrible here tonight again.  SPeeds cut down to 12Mb/s however the latency and packet loss is so bad, I cannot do anything but comment on forums.  I have HAD it with this sub-par service.

Long Beach, CA 75/35 FIOS

[jk1112]

I tweeted them a message tonight.  And will continue to do so every day until it is addressed.

[Johnny_Utah]

I will do the same then!

[smith6612]

If you could supply one of the traceroutes you're seeing issues with to us that would be appreciated.