DNS issues in SoCal

shayne571
Newbie

Since about Monday I'm unable to access various sites. I have confirmed they are up because I can access them from a nearby Starbucks and a neighbor's connection, neither of which are on FIOS.

This is one site I can't access that I confirmed is online:

http://store.steampowered.com/

When I tried to access thiis site using IE 9 (from my Windows 7 PC), I get this message:

Internet Explorer cannot display the webpage

When I click on the Diagnose Connection Problems button, I get this:

website is online but isn't responding to connection attempts

This is happening on multiple sites.

0 Likes
187 Replies
Ftracy3
Newbie

I've been having issues for two weeks now. Started with access to Steam forums and/or their store. Reset the router, released the IPaddress  and powered everything down for a few hours finally cleared things up. Problem came back  later and switched to OpenDNS and I thought that had fixed it.

Now today i can't access speedtest.net or speakeasy.net. Can ping them both. Have flushed dns through ipconfig, rebooted..I may try the IP release, power off router for a few hours again. Verizon, what's going on? I can access these sites from work or through my phone but not through Verizon.

0 Likes
Martymcfli
Newbie
Been having the same issue for almost two days. I just swapped the default Verizon DNS to Google's DNS. Fixed the issues for me.
0 Likes
frnkblk
Enthusiast - Level 3

@Martymcfi:

Which issues? ๐Ÿ˜‰  Your statement makes it sounds like everything is fixed, but you haven't shared which sites weren't working for you.

Can you access these sites, too (they have been noted by some as problematic)

Frank

0 Likes
Scottage
Enthusiast - Level 1

I left my router unplugged overnight. Plugged it back in this morning and got a new IP.

Now it appears I can get to the sites I had trouble with before.

Of course I "fixed" it once before this way and it eventually started to go bad again.

And if it is tied to the IP address as it appears, this means someone else may now have that bad IP.

0 Likes
PJL
Master - Level 3

@Scottage wrote:

I left my router unplugged overnight. Plugged it back in this morning and got a new IP.

Now it appears I can get to the sites I had trouble with before.

Of course I "fixed" it once before this way and it eventually started to go bad again.

And if it is tied to the IP address as it appears, this means someone else may now have that bad IP.


It's tied to a Verizon router cache (not the one in your house) for the IP, which in theory would be cleared when someone else was assigned the IP.  Yours if "fixed" because the new IP got a new cache on the Verizon router.  It may again become corrupt.  Did your FiOS TV Remote DVR and Caller ID break when you were assigned the new IP?

0 Likes
JVMarron
Newbie

Just got off the phone with Verizon FIOS support.  Absolutely no help. He tried to break/renew my IP and that did not help and then he had me use command prompt and trace the route of one of the sites I am having problems with (zune.net) and then told me, it isn't verizon's issue because it fails on the 11th hop and not within the first 6.  I told him that I was able to reach this site with my phone when I turned off the wifi and used 3G and also when I used a proxy server on this pc.  He didn't have an answer.  So has anyone actually found a fix? I tried using google DNS with no success.  I can't sync my music player, so I'm not happy right now.  Any suggestions would be greatly appreciated

0 Likes
BenM1
Newbie

I've been experiencing similar problems in Santa Monica. I originally noticed the problem with techmeme.com, speedtest.net, and store.steampowered.com being inaccessible over my FiOS connection unless I was connected to my work VPN.

Changing DNS servers makes no difference in my case. Leaving DNS out of the picture, directly pinging an affected IP (such as techmeme's 173.192.129.24, which does happen to be returned by both Verizon's and Google's DNS servers) shows 100% packet loss. This is compared to 0% packet loss for pings to the same IP when on the VPN.

I just checked the other sites mentioned in this thread:

costco.com (down)โ€จ

albertsons.com (down)

โ€จclub.nintendo.com (up)

โ€จportforward.com (up)

โ€จeternalmoonwalk.com (down)

โ€จnfggames.com (down)

โ€จfoodily.com (down)

โ€จthinkwithportals.com (up)

โ€จrunicgames.com (up)โ€จ

tcrf.net (up)โ€จ

wiki.teamfortress.com (down)

steamcommunity.com (up)
steampowered.com (down)โ€จ
store.steampowered.com (down)
www.bankofamerica.com (up)
www.alaskaair.com (up)
www.wpcentral.com (up)
www.sdccu.org (up)
www.sdccu.net (up)
www.mysonicwall.com (up)
idp.mrmls.safemls.net/idp/Authn/UserPassword (up)

techmeme.com (down)
speedtest.net (down)

0 Likes
Ftracy3
Newbie

I can ping steamcommunity.com but can't get in--I cannnot ping speedtest.net, timed out, 100% loss. Can get to both from my phone and through proxy sites. This is getting old... Released my IP and rebooted the router after a few hours..that worked last week but not this morning.

0 Likes
frnkblk
Enthusiast - Level 3

@BenM1 wrote:

I've been experiencing similar problems in Santa Monica. I originally noticed the problem with techmeme.com, speedtest.net, and store.steampowered.com being inaccessible over my FiOS connection unless I was connected to my work VPN.

Changing DNS servers makes no difference in my case. Leaving DNS out of the picture, directly pinging an affected IP (such as techmeme's 173.192.129.24, which does happen to be returned by both Verizon's and Google's DNS servers) shows 100% packet loss. This is compared to 0% packet loss for pings to the same IP when on the VPN.

I just checked the other sites mentioned in this thread:


@BenM1: When you checked each of those sites, which DNS server(s) did you use, and what were the returned A values for each of those domains?  The ability to ping a site from the VPN and not from FiOS is not necessarily indicative that there's an issue with HTTP access, as ICMP could be blocked at one or more hops from certain directions/routes and those blocking hops aren't in the other path.

Perhaps for those domains that have their content hosted on multiple A records (either round-robin, SRV, or geo-based DNS) we need to build a hosts file that can be used by SoCal FiOS subscribers to get around those IPs that don't work.

0 Likes
BenM1
Newbie

@frnkblk wrote:

@BenM1: When you checked each of those sites, which DNS server(s) did you use, and what were the returned A values for each of those domains?  The ability to ping a site from the VPN and not from FiOS is not necessarily indicative that there's an issue with HTTP access, as ICMP could be blocked at one or more hops from certain directions/routes and those blocking hops aren't in the other path.

Perhaps for those domains that have their content hosted on multiple A records (either round-robin, SRV, or geo-based DNS) we need to build a hosts file that can be used by SoCal FiOS subscribers to get around those IPs that don't work.


Looks like only one A record per domain, and similar results from all servers except that the Verizon server didn't return as much info for costco.com.


On VPN using private DNS server:

$ host -a techmeme.com
Trying "techmeme.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57789
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 86400 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 86400 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 86400 IN MX 1 aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 86400 IN MX 10 alt2.aspmx.l.google.com.
techmeme.com. 86400 IN A 173.192.129.24
techmeme.com. 86400 IN SOA ns1.softlayer.com. root.techmeme.com. 2011010901 3600 300 604800 3600
techmeme.com. 86400 IN NS ns1.softlayer.com.
techmeme.com. 86400 IN NS ns2.softlayer.com.

Received 309 bytes from 172.16.255.1#53 in 3723 ms


$ host -a costco.com
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Trying "costco.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36880
;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 86387 IN A 170.167.8.1
costco.com. 14387 IN TXT "v=spf1 a:mx1.costco.com a:mx1a.costco.com a:mx1b.costco.com a:mx1c.costco.com a:mx1d.costco.com a:mx1f.costco.com ip4:170.167.3.96/28 ip4:170.167.5.0/24 ip4:170.167.21.1 include:_netblocks.google.com -all"
costco.com. 287 IN MX 10 costco.com.s10b1.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10b2.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10a1.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10a2.psmtp.com.
costco.com. 86387 IN SOA dbru.br.ns.els-gms.att.net. rm-hostmaster.ems.att.com. 123 10800 3600 604800 86400
costco.com. 86387 IN NS dmtu.mt.ns.els-gms.att.net.
costco.com. 86387 IN NS dbru.br.ns.els-gms.att.net.

Received 519 bytes from 172.16.255.1#53 in 31 ms

Using Google public DNS:

$ host -a techmeme.com 8.8.8.8
Trying "techmeme.com"
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8027
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 85848 IN SOA ns1.softlayer.com. root.techmeme.com. 2011010901 3600 300 604800 3600
techmeme.com. 85848 IN NS ns2.softlayer.com.
techmeme.com. 85848 IN NS ns1.softlayer.com.
techmeme.com. 85848 IN A 173.192.129.24
techmeme.com. 85848 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 85848 IN MX 1 aspmx.l.google.com.
techmeme.com. 85848 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 85848 IN MX 10 alt2.aspmx.l.google.com.

Received 309 bytes from 8.8.8.8#53 in 102 ms

$ host -a costco.com 8.8.8.8
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21878
;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 86400 IN SOA dbru.br.ns.els-gms.att.net. rm-hostmaster.ems.att.com. 123 10800 3600 604800 86400
costco.com. 300 IN MX 10 costco.com.s10a1.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10a2.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10b1.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10b2.psmtp.com.
costco.com. 14400 IN TXT "v=spf1 a:mx1.costco.com a:mx1a.costco.com a:mx1b.costco.com a:mx1c.costco.com a:mx1d.costco.com a:mx1f.costco.com ip4:170.167.3.96/28 ip4:170.167.5.0/24 ip4:170.167.21.1 include:_netblocks.google.com -all"
costco.com. 86400 IN A 170.167.8.1
costco.com. 86400 IN NS dmtu.mt.ns.els-gms.att.net.
costco.com. 86400 IN NS dbru.br.ns.els-gms.att.net.

Received 519 bytes from 8.8.8.8#53 in 102 ms

Using Verizon DNS:

$ host -a techmeme.com 68.238.64.14
Trying "techmeme.com"
Using domain server:
Name: 68.238.64.14
Address: 68.238.64.14#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23316
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 4

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 86400 IN NS ns1.softlayer.com.
techmeme.com. 86400 IN NS ns2.softlayer.com.
techmeme.com. 86400 IN SOA ns1.softlayer.com. root.softlayer.com. 2011010901 7200 600 1728000 43200
techmeme.com. 86400 IN MX 1 aspmx.l.google.com.
techmeme.com. 86400 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 alt2.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 86400 IN A 173.192.129.24

;; ADDITIONAL SECTION:
ns1.softlayer.com. 84120 IN A 67.228.254.4
aspmx3.googlemail.com. 2471 IN A 74.125.127.27
aspmx4.googlemail.com. 1262 IN A 209.85.229.27
aspmx5.googlemail.com. 1905 IN A 74.125.157.27

Received 373 bytes from 68.238.64.14#53 in 86 ms

$ host -a costco.com 68.238.64.14
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Using domain server:
Name: 68.238.64.14
Address: 68.238.64.14#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35578
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 84415 IN A 170.167.8.1

Received 44 bytes from 68.238.64.14#53 in 26 ms

0 Likes
frnkblk
Enthusiast - Level 3

@BenM1 wrote:

@frnkblk wrote:

@BenM1: When you checked each of those sites, which DNS server(s) did you use, and what were the returned A values for each of those domains?  The ability to ping a site from the VPN and not from FiOS is not necessarily indicative that there's an issue with HTTP access, as ICMP could be blocked at one or more hops from certain directions/routes and those blocking hops aren't in the other path.

Perhaps for those domains that have their content hosted on multiple A records (either round-robin, SRV, or geo-based DNS) we need to build a hosts file that can be used by SoCal FiOS subscribers to get around those IPs that don't work.


Looks like only one A record per domain, and similar results from all servers except that the Verizon server didn't return as much info for costco.com.

<snip>



costco.com redirects to www.costco.com, but both A records are 170.167.8.1.  Based on your testing, changing DNS servers shouldn't make a difference with in regards to accessing techmeme.com and costco.com because they seem to use the same A records regardless.

0 Likes
frnkblk
Enthusiast - Level 3

@PJL wrote:

@Scottage wrote:

I left my router unplugged overnight. Plugged it back in this morning and got a new IP.

Now it appears I can get to the sites I had trouble with before.

Of course I "fixed" it once before this way and it eventually started to go bad again.

And if it is tied to the IP address as it appears, this means someone else may now have that bad IP.


It's tied to a Verizon router cache (not the one in your house) for the IP, which in theory would be cleared when someone else was assigned the IP.  Yours if "fixed" because the new IP got a new cache on the Verizon router.  It may again become corrupt.  Did your FiOS TV Remote DVR and Caller ID break when you were assigned the new IP?



Do you mean the routing tables on Verizon(s)' routers?  While rare, it's possible that there could be a bug such that traffic doesn't flow the way that it should, based on the routing tables, or that there is corruption on the routing table.  We had a customer several years back that couldn't get to Walgreens, while our other customers could.  We eventually narrowed it down to any IP address in their /24.  After reaching out the network admins at Walgreens, it turned out that there was a software bug on one or more their routers that led to some kind of corruption that could be manually cleared, but re-occured.  I can't recall if it was a configuration change or a software upgrade, but they did get the issue resolved.

Not saying that anything of what I described in the above paragraph applies to Verizon's FiOS network in SoCal -- they haven't shared much detail with their customers and the issue is clearly not resolved.

0 Likes
PJL
Master - Level 3

@frnkblk wrote:

@PJL wrote:

@Scottage wrote:

I left my router unplugged overnight. Plugged it back in this morning and got a new IP.

Now it appears I can get to the sites I had trouble with before.

Of course I "fixed" it once before this way and it eventually started to go bad again.

And if it is tied to the IP address as it appears, this means someone else may now have that bad IP.


It's tied to a Verizon router cache (not the one in your house) for the IP, which in theory would be cleared when someone else was assigned the IP.  Yours if "fixed" because the new IP got a new cache on the Verizon router.  It may again become corrupt.  Did your FiOS TV Remote DVR and Caller ID break when you were assigned the new IP?



Do you mean the routing tables on Verizon(s)' routers?  While rare, it's possible that there could be a bug such that traffic doesn't flow the way that it should, based on the routing tables, or that there is corruption on the routing table.  We had a customer several years back that couldn't get to Walgreens, while our other customers could.  We eventually narrowed it down to any IP address in their /24.  After reaching out the network admins at Walgreens, it turned out that there was a software bug on one or more their routers that led to some kind of corruption that could be manually cleared, but re-occured.  I can't recall if it was a configuration change or a software upgrade, but they did get the issue resolved.

Not saying that anything of what I described in the above paragraph applies to Verizon's FiOS network in SoCal -- they haven't shared much detail with their customers and the issue is clearly not resolved.


Yes I mean the routing tables on Verizon's routers.  I think what you describe is what we're seeing here is Southern California and what was told to me when a Level 2 network admin spoke with me about my problem.  And he said it wasn't just isolated to Southern California, but the corrupt routing tables were occuring elsewhere on the Verizon network.  Verizon, can you hear us now?

0 Likes
Scottage
Enthusiast - Level 1

Did your FiOS TV Remote DVR and Caller ID break when you were assigned the new IP?

My FiOS TV Remote DVR so far has never worked properly since it was installed,  It only ever tells me "Bad Gateway" when I try to access any DVR functions via the web page interface. I was going to look into that after the internet routing problem was fixed, because it's a feature I really want to use... but I haven't managed to get that far yet. I don't use Verizon for phone, so caller ID isn't applicable.

Does anyone who is having this problem have both a PC running Microsoft Windows and some other kind of device, like an Apple product (iBook, iPoD Touch, etc) and see if the problem exists on both devices or just Windows?

On our two PCs, when we're unable to reach a site because of this issue, my iPhone will also be unable to reach that same site if it's connected to my home WiFi, so it's definitely not device/OS specific. If I switch it to 3G mode (it's AT&T, so not using Verizon at all for that connection) then I'm able to get to those same sites immediatley, without issue.



0 Likes
DiAmore
Enthusiast - Level 2

Okay after grabbing a new IP 71.103.142.*  .... I can now access XBL, and steam store works.  

store.steam.com (UP)

steampowered.com (down)

steamcommunity.com (down)

techmeme.com (down)

alaskaair.com (down)

speedtest.net (UP)

albertsons.com (UP)

costco.com (UP)

runicgames.com (down)

0 Likes
ChristopheJay
Enthusiast - Level 3

Just another Southern California (Long Beach, specifically) experience the same problem as many people on this forum. I have experienced the issues since Friday 2/3/12 evening.

I called tech support once on Saturday morning and again tonight with little help. Tonight I let the representative know about all the commenter's on the forums but is still unaware of the issue. 

I have had problems with facebook (fixed after my second tech support call and router reset.)

I also I had a problem with www.questionpro.com (loading really slow), Alaska airlines (unable to access) xbox live forums and Steam not connections. However after renewing my ip address these issues are fixed. 

However I am still unable to connect to Xbox Live. 

ChristopheJay
Enthusiast - Level 3

It turns out my  video on demand was not working either...I just gave Verizon another call, and got a really helpful representative. 

He did something that was a reset of all services. This fixed my issues with video on my demand. Also I can access websites like alaska airlines which I was not able to do before. 

The representative also let me know that Xbox Live pushed out a update that changed some port configurations and the Verizon techs are working on making sure their network is compitable. I am still not able to connect to Xbox Live, however Xbox Live my test is now passing the internet section, but failing on the Xbox Live section.

If your having problems, ask your Verizon Tech Support for a remote reset on all home services. 

0 Likes
MrShoop
Newbie

I'm in SoCal, having the same problem! Haven't been able to get to store.steampowered.com since Friday. Facebook has been intermittently up and down. Just tried costco.com and club.nintendo.com and neither are reachable.

From my home w/FIOS:

ping store.steampowered.com
PING store.steampowered.com (63.228.223.104): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
^C
--- store.steampowered.com ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss

From a remote pc:

ping store.steampowered.com
PING store.steampowered.com (63.228.223.104) 56(84) bytes of data.
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=1 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=2 ttl=56 time=82.1 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=3 ttl=56 time=82.1 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=4 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=5 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=6 ttl=56 time=82.1 ms
^C
--- store.steampowered.com ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5018ms
rtt min/avg/max/mdev = 82.149/82.217/82.296/0.336 ms

0 Likes
mrballcb
Enthusiast - Level 3

@MrShoop wrote:

I'm in SoCal, having the same problem! Haven't been able to get to store.steampowered.com since Friday. Facebook has been intermittently up and down. Just tried costco.com and club.nintendo.com and neither are reachable.


What's your IP right now?

Regards...    Todd

0 Likes
Ftracy3
Newbie

Relased my IP (again) through the router, shut everything off overnight..Now several sites that I couldn't get yesterday (costco, steam community, alaska air) are working, but speedtest.net (which finally was working yesterday) is not. What you can access seems to be a real crapshoot depending on your IP address or time of day as all of us have common trouble sites but they seem to vary on being either up or down for individuals.

0 Likes
MrShoop
Newbie

@mrballcb wrote:

@MrShoop wrote:

I'm in SoCal, having the same problem! Haven't been able to get to store.steampowered.com since Friday. Facebook has been intermittently up and down. Just tried costco.com and club.nintendo.com and neither are reachable.


What's your IP right now?

Regards...    Todd


108.23.119.* {edited for privacy}

and still having the connection issues. I was hoping once all the superbowl parties were ove it would get fixed. ๐Ÿ™‚

0 Likes