DNS issues in SoCal

[BenM1]

---

@frnkblk wrote:

@BenM1: When you checked each of those sites, which DNS server(s) did you use, and what were the returned A values for each of those domains?  The ability to ping a site from the VPN and not from FiOS is not necessarily indicative that there's an issue with HTTP access, as ICMP could be blocked at one or more hops from certain directions/routes and those blocking hops aren't in the other path.

Perhaps for those domains that have their content hosted on multiple A records (either round-robin, SRV, or geo-based DNS) we need to build a hosts file that can be used by SoCal FiOS subscribers to get around those IPs that don't work.

---

Looks like only one A record per domain, and similar results from all servers except that the Verizon server didn't return as much info for costco.com.

On VPN using private DNS server:

$ host -a techmeme.com
Trying "techmeme.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 57789
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 86400 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 86400 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 86400 IN MX 1 aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 86400 IN MX 10 alt2.aspmx.l.google.com.
techmeme.com. 86400 IN A 173.192.129.24
techmeme.com. 86400 IN SOA ns1.softlayer.com. root.techmeme.com. 2011010901 3600 300 604800 3600
techmeme.com. 86400 IN NS ns1.softlayer.com.
techmeme.com. 86400 IN NS ns2.softlayer.com.

Received 309 bytes from 172.16.255.1#53 in 3723 ms

$ host -a costco.com
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Trying "costco.com"
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 36880
;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 86387 IN A 170.167.8.1
costco.com. 14387 IN TXT "v=spf1 a:mx1.costco.com a:mx1a.costco.com a:mx1b.costco.com a:mx1c.costco.com a:mx1d.costco.com a:mx1f.costco.com ip4:170.167.3.96/28 ip4:170.167.5.0/24 ip4:170.167.21.1 include:_netblocks.google.com -all"
costco.com. 287 IN MX 10 costco.com.s10b1.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10b2.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10a1.psmtp.com.
costco.com. 287 IN MX 10 costco.com.s10a2.psmtp.com.
costco.com. 86387 IN SOA dbru.br.ns.els-gms.att.net. rm-hostmaster.ems.att.com. 123 10800 3600 604800 86400
costco.com. 86387 IN NS dmtu.mt.ns.els-gms.att.net.
costco.com. 86387 IN NS dbru.br.ns.els-gms.att.net.

Received 519 bytes from 172.16.255.1#53 in 31 ms

Using Google public DNS:

$ host -a techmeme.com 8.8.8.8
Trying "techmeme.com"
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8027
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 85848 IN SOA ns1.softlayer.com. root.techmeme.com. 2011010901 3600 300 604800 3600
techmeme.com. 85848 IN NS ns2.softlayer.com.
techmeme.com. 85848 IN NS ns1.softlayer.com.
techmeme.com. 85848 IN A 173.192.129.24
techmeme.com. 85848 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 85848 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 85848 IN MX 1 aspmx.l.google.com.
techmeme.com. 85848 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 85848 IN MX 10 alt2.aspmx.l.google.com.

Received 309 bytes from 8.8.8.8#53 in 102 ms

$ host -a costco.com 8.8.8.8
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21878
;; flags: qr rd ra; QUERY: 1, ANSWER: 9, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 86400 IN SOA dbru.br.ns.els-gms.att.net. rm-hostmaster.ems.att.com. 123 10800 3600 604800 86400
costco.com. 300 IN MX 10 costco.com.s10a1.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10a2.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10b1.psmtp.com.
costco.com. 300 IN MX 10 costco.com.s10b2.psmtp.com.
costco.com. 14400 IN TXT "v=spf1 a:mx1.costco.com a:mx1a.costco.com a:mx1b.costco.com a:mx1c.costco.com a:mx1d.costco.com a:mx1f.costco.com ip4:170.167.3.96/28 ip4:170.167.5.0/24 ip4:170.167.21.1 include:_netblocks.google.com -all"
costco.com. 86400 IN A 170.167.8.1
costco.com. 86400 IN NS dmtu.mt.ns.els-gms.att.net.
costco.com. 86400 IN NS dbru.br.ns.els-gms.att.net.

Received 519 bytes from 8.8.8.8#53 in 102 ms

Using Verizon DNS:

$ host -a techmeme.com 68.238.64.14
Trying "techmeme.com"
Using domain server:
Name: 68.238.64.14
Address: 68.238.64.14#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 23316
;; flags: qr rd ra; QUERY: 1, ANSWER: 11, AUTHORITY: 0, ADDITIONAL: 4

;; QUESTION SECTION:
;techmeme.com. IN ANY

;; ANSWER SECTION:
techmeme.com. 86400 IN NS ns1.softlayer.com.
techmeme.com. 86400 IN NS ns2.softlayer.com.
techmeme.com. 86400 IN SOA ns1.softlayer.com. root.softlayer.com. 2011010901 7200 600 1728000 43200
techmeme.com. 86400 IN MX 1 aspmx.l.google.com.
techmeme.com. 86400 IN MX 5 alt1.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 alt2.aspmx.l.google.com.
techmeme.com. 86400 IN MX 10 aspmx2.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx3.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx4.googlemail.com.
techmeme.com. 86400 IN MX 10 aspmx5.googlemail.com.
techmeme.com. 86400 IN A 173.192.129.24

;; ADDITIONAL SECTION:
ns1.softlayer.com. 84120 IN A 67.228.254.4
aspmx3.googlemail.com. 2471 IN A 74.125.127.27
aspmx4.googlemail.com. 1262 IN A 209.85.229.27
aspmx5.googlemail.com. 1905 IN A 74.125.157.27

Received 373 bytes from 68.238.64.14#53 in 86 ms

$ host -a costco.com 68.238.64.14
Trying "costco.com"
;; Truncated, retrying in TCP mode.
Trying "costco.com"
Using domain server:
Name: 68.238.64.14
Address: 68.238.64.14#53
Aliases:

;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35578
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;costco.com. IN ANY

;; ANSWER SECTION:
costco.com. 84415 IN A 170.167.8.1

Received 44 bytes from 68.238.64.14#53 in 26 ms

[DiAmore]

Okay after grabbing a new IP 71.103.142.*  .... I can now access XBL, and steam store works.  

store.steam.com (UP)

steampowered.com (down)

steamcommunity.com (down)

techmeme.com (down)

alaskaair.com (down)

speedtest.net (UP)

albertsons.com (UP)

costco.com (UP)

runicgames.com (down)

[Scottage]

Did your FiOS TV Remote DVR and Caller ID break when you were assigned the new IP?

My FiOS TV Remote DVR so far has never worked properly since it was installed,  It only ever tells me "Bad Gateway" when I try to access any DVR functions via the web page interface. I was going to look into that after the internet routing problem was fixed, because it's a feature I really want to use... but I haven't managed to get that far yet. I don't use Verizon for phone, so caller ID isn't applicable.

Does anyone who is having this problem have both a PC running Microsoft Windows and some other kind of device, like an Apple product (iBook, iPoD Touch, etc) and see if the problem exists on both devices or just Windows?

On our two PCs, when we're unable to reach a site because of this issue, my iPhone will also be unable to reach that same site if it's connected to my home WiFi, so it's definitely not device/OS specific. If I switch it to 3G mode (it's AT&T, so not using Verizon at all for that connection) then I'm able to get to those same sites immediatley, without issue.

[frnkblk]

---

@BenM1 wrote:

---

@frnkblk wrote:

@BenM1: When you checked each of those sites, which DNS server(s) did you use, and what were the returned A values for each of those domains?  The ability to ping a site from the VPN and not from FiOS is not necessarily indicative that there's an issue with HTTP access, as ICMP could be blocked at one or more hops from certain directions/routes and those blocking hops aren't in the other path.

Perhaps for those domains that have their content hosted on multiple A records (either round-robin, SRV, or geo-based DNS) we need to build a hosts file that can be used by SoCal FiOS subscribers to get around those IPs that don't work.

---

Looks like only one A record per domain, and similar results from all servers except that the Verizon server didn't return as much info for costco.com.

<snip>

---

costco.com redirects to www.costco.com, but both A records are 170.167.8.1.  Based on your testing, changing DNS servers shouldn't make a difference with in regards to accessing techmeme.com and costco.com because they seem to use the same A records regardless.

[ChristopheJay]

Just another Southern California (Long Beach, specifically) experience the same problem as many people on this forum. I have experienced the issues since Friday 2/3/12 evening.

I called tech support once on Saturday morning and again tonight with little help. Tonight I let the representative know about all the commenter's on the forums but is still unaware of the issue. 

I have had problems with facebook (fixed after my second tech support call and router reset.)

I also I had a problem with www.questionpro.com (loading really slow), Alaska airlines (unable to access) xbox live forums and Steam not connections. However after renewing my ip address these issues are fixed. 

However I am still unable to connect to Xbox Live. 

[mrballcb]

Does anyone who is having this problem have both a PC running Microsoft Windows and some other kind of device, like an Apple product (iBook, iPoD Touch, etc) and see if the problem exists on both devices or just Windows?

---

When the problem existed for me, it was with Linux devices (more than 1).

What kind of router are people using that are experiencing this?  I had an ActionTek WR-424MI Rev A, and tech support, as part of the troubleshooting process, sent me a replacement router, a Rev E (i.e. newer firmware).  In addition to a new IP when I swapped them out, I have not experienced the issue again.  *BUT*, I also have had the same IP since I swapped out to the Rev E router.  So, are any of you that are experiencing this still running the older (less than Rev C) ActionTek routers?

[DiAmore]

I am using MI424WR Rev. F... still problems.  Like I said new IP fixed XBL and I can now get to store.steampowered.com but not steampowered itself or the community.  It's not a problem with routers/modem.  It happens on Windows, OSX, and iOS.  So, yeah.. there you have it.  Verizon's network has problems somewhere. @VerizonSupport on twitter said to DM them with problems, I did.. no response.  Lovely, I can't wait to have to call tomorrow, sit on hold.. go through a bunch of reset this and that... ugh... 

[BenM1]

My router is a Westell 9100VM. It's a somewhat unusual setup where a shared MDU ONT in the basement serves multiple apartments in the building via VDSL over preexisting phone wiring.

[ChristopheJay]

It turns out my  video on demand was not working either...I just gave Verizon another call, and got a really helpful representative. 

He did something that was a reset of all services. This fixed my issues with video on my demand. Also I can access websites like alaska airlines which I was not able to do before. 

The representative also let me know that Xbox Live pushed out a update that changed some port configurations and the Verizon techs are working on making sure their network is compitable. I am still not able to connect to Xbox Live, however Xbox Live my test is now passing the internet section, but failing on the Xbox Live section.

If your having problems, ask your Verizon Tech Support for a remote reset on all home services. 

[MrShoop]

I'm in SoCal, having the same problem! Haven't been able to get to store.steampowered.com since Friday. Facebook has been intermittently up and down. Just tried costco.com and club.nintendo.com and neither are reachable.

From my home w/FIOS:

ping store.steampowered.com
PING store.steampowered.com (63.228.223.104): 56 data bytes
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
Request timeout for icmp_seq 4
Request timeout for icmp_seq 5
^C
--- store.steampowered.com ping statistics ---
7 packets transmitted, 0 packets received, 100.0% packet loss

From a remote pc:

ping store.steampowered.com
PING store.steampowered.com (63.228.223.104) 56(84) bytes of data.
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=1 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=2 ttl=56 time=82.1 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=3 ttl=56 time=82.1 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=4 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=5 ttl=56 time=82.2 ms
64 bytes from store.steampowered.com (63.228.223.104): icmp_seq=6 ttl=56 time=82.1 ms
^C
--- store.steampowered.com ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5018ms
rtt min/avg/max/mdev = 82.149/82.217/82.296/0.336 ms