Child Item

Site Totals

61487 Questions Answered 271 Categories

All Apple updates time out. But through a VPN, it's fine!

Something might be wrong with Verizon Fios in the NYC area. Since last night, any download from Apple times out. Also http://www.speakeasy.net/speedtest/ reports a speed of less than 1Mbps. Connected to a VPN and everything worked normally. Verizon support guy said nothing was wrong, but someone please tell me why this is happening?

thanks

Comments

PearTree85

Yup, exactly the same problem here. Trying to download iOS8 from Manhattan and the download crawls - 10Kb/s. If I run it through a Chicago proxy it gets back up to around 2Mb/s (which may well be limited from the Apple side).

Impacting all Apple downloads by the looks of it - I tried to update iTunes and it suffered the same fate.

eljefe2

I seem to be having a similar problem here. Speedtest.net shows my normal 75+ mb/s download (and upload), but trying to install iOS 8 on my wife's iPhone, and some updates on her MacBook Pro, the download speeds are almost zero.

Yet I just updated iTunes on my PC and download speed was normal. Smiley Frustrated

smith6612

You might be seeing a problem with the regional CDN. Apple uses their CDN as a primary means of distribution, with Akamai and Limelight being used as alternate sources.

eljefe2

I found a solution that worked for the iOS update.

With my iPhone 5s connected to an at&t LTE tower, I turned on the iPhone's personal hotspot. I connected my wife's phone to mine via the personal hotspot and presto....the iOS 8 update completed in 9 minutes.

So it would seem there is a routing problem between Verizon's network and Apple's servers. I didn't both to trace the route, but it seems very similar to the problems many have had with NetFlix, etc.

In the mean time, my better half's MacBook Pro is still downloading updates at ~25 kB/s.

getOffMyTV

Thanks for posting everyone. At least I'm not alone. I've been pretty happy with Fios (compared to TWC) and this is the first time that I've had a major issue with the service. I'm just hope that I won't have to recover one of my Macs with Internet Recovery, VPN would not be an option in that case. Sure, there are always ways around a problem, but we shouldnt have to jump through hoops to access the data that we require.

I wonder is Verizon try to shake Apple down like they did Netflix?

Hubrisnxs

This is another good example of a VPN being a heavy handed approach to fixing the problem and giving a false impression of peering issues or throttling.

Simply change your DNS

FIOS horrible Apple download speeds.

Use Google DNS configured directly to your machine or router,

8.8.4.4 and the quad 8888's 8.8.8.8

eljefe2

@Hubrisnxs wrote:
This is another good example of a VPN being a heavy handed approach to fixing the problem and giving a false impression of peering issues or throttling.
Simply change your DNS
FIOS horrible Apple download speeds.
Use Google DNS configured directly to your machine or router,
8.8.4.4 and the quad 8888's 8.8.8.8

I tried that. I set my router to Google's DNS servers and it made no difference. Download from the Apple server(s) is still taking forever.

jimdwestboro

I've experienced the same problem here in Massachusetts. At first I thought it was an Apple problem, but then I tried downloading iOS 8 via cellular and found it to be orders of magnitude faster than my FIOS 75/75 network despite speedtest showing full speed capability!

I then tried downloading from Apple through my company's VPN on my home computer (same computer, same physical network as my FIOS setup) and was able to download iOS in about 8 minutes (about 7mb/sec) versus 11-12 kb/sec without the VPN.

This is outrageous for a supposedly high-speed network. Does anyone from Verizon monitor this forum? I'm reluctant to get involved with a level 1 support call on this, as it's clearly not a problem at my end.

getOffMyTV

When I first discovered a couple days ago, upon iOS 8 being released, I did a tier 1 chat with Verizon support, the guy "reset the main box" in our apartment, and then when that didnt do anything, he said that it MUST be Apple's fault. Typical tech support redirection **bleep**. Not his fault per se, but I work in IT and I know its too often the way things work.

eljefe2

It is frustrating.

I'm trying to update my wife's iPad to iOS 8 now. I can't even get the download to start via my FiOS service and I don't have enough data left this month on my at&t cellular plan to use the same trick as I used on her iPhone.

I'm thinking about walking down the street and trying a neighbor's cable connection to see if that's any better.

Hubrisnxs

Did you put them directly into the machine/device or in the router? some people said they had to put them in the device

Also if those don't work also try level 3's and also open's

Level 3 is 4.2.2.1 and 4.2.2.2 and open dns is 208.67.222.222 208.67.220.220

Hubrisnxs

It's happening because Apple's CDN in your area is getting taxed or it has a peering issue with a particular network. You would have to run an MTR scan to find out if it has a peering issue like that, a regular traceroute will only give a smidgen of actual info.

most likely getting taxed, they sold 6 million new devices in the first 3 or 4 days and that isn't including any of their legacy customers trying to get the update too.

eljefe2

Yep...I understand that. And yes, I tried Google, OpenDNS and Level 3, each entered directly into my Actiontec router.

It's still at least interesting that I run into the problem through FiOS, regardless of what DNS I use, but using at&t cellular data I didn't have the issue.

Hubrisnxs

That would be a fun experiment. If you child get their dns and try it. That could point to a peering issue. Also they did don't put it in the router. Instead put it in your device (laptop ipad etc)

eljefe2

What difference does it make if I put the DNS server in the device, iPad, etc., or in the router, if the device defaults to 192.168.1.1 to get DNS service from its wifi connection?

Hubrisnxs

They said they were testing and it wasn't passing through the new dns to the devices from the router. I thought it was them misinterpreting what they were seeing because they naturally see the default gateway, but they said there was a difference when they put it in the device direct. I think it's noted on that BBR link I posted earlier.

Hubrisnxs

Also There's a lot of reports that this is compatible for Iphone 4 and 4s but when applied it makes the phone nearly unuseable. So beware, for eljefe, and all others reading this, if you are updating a device like that.

eljefe2

@Hubrisnxs wrote:
They said they were testing and it wasn't passing through the new dns to the devices from the router. I thought it was them misinterpreting what they were seeing because they naturally see the default gateway, but they said there was a difference when they put it in the device direct. I think it's noted on that BBR link I posted earlier.

Well, I tried Google's DNS directly in the iPad in question and still no joy.

I may walk down the street and try our neighbor's cable connection, or I may just wait a few days until the iOS 8 / iPhone 6 storm calms down and update my wife's iPad then. Smiley Wink

theblanks

Looking into what those of us with FiOS internet can do to download iOS 8 via iTunes. Several iOS users are experiencing 40+ hour download times that ultimately time out due to network errors and the common theme on Apple Support Community forums seems to be Verizon FiOS.

I'm on the east coast and have tried it at 5:00 am, 1:00 pm, and 12:00 am with a hard-wired connection on a brand new MacMini with 16GB of RAM and it's failed every time after one hour.

confusedbrit

I also have had this problem. I have tried changing my DNS servers to Google's on both my PCs and the router. I also tried my work VPN and had the same results. Brought me laptop into the office (which has a super slow connection) and I'm getting 45 min as a download speed. It seems it is a Verizon Fios problem.

rjw246

Yes, it's true -- this is not an Apple issue (although they're culpable on some level here) but rather a FIOS problem (it certatinly seems they're throttling here).

Regardless of who is to blame, here is a workaround: change the DNS to the Google Public DNS -- instructions for both Mac and Windows can be found here:

https://developers.google.com/speed/public-dns/docs/using

I was trying to upload ios8 over the weekend (wired and wirelessly) with upload times of 72+ hours, and timing out after an hour or so.

I just changed the DNS following the instructions (and be sure to reboot after you change them), and I downloaded the ios8 update in 6 minutes. Literally.

It would've been nice to know this from Verizon (considering how much we get fleeced for sub-standard service), or even Apple, but.... well, now you know how to fix it. Enjoy

Hubrisnxs

Another advanced user found the bad Apple CDN's "blackholes" and posted the fix work around.

If you're relatively savvy you can try it.

he writes

This NANOG poster hit the nail on the head, Apple didn't buy enough or the right capacity into Verizon:

»permalink.gmane.org/gmane.org.op···g/124173

The good news is there are options... if your router supports static routing and blackholes, you can route 17.253.0.0/16 to the bit bucket there. Otherwise, in OSX you can do something like 'sudo route add 17.253.0.0/16 127.0.0.1' from the command prompt with the same results. (Of course this won't change how your iOS device fetches updates, only your mac.) The OS will fail back to other CDNs and the transfers will be fast.

Hopefully Apple will realize the mistake here and go back to using reliable CDNs like Akamai. Their in house one just isn't ready for prime time.

Hubrisnxs

@rjw246 wrote:
Yes, it's true -- this is not an Apple issue (although they're culpable on some level here) but rather a FIOS problem (it certatinly seems they're throttling here).

Regardless of who is to blame, here is a workaround: change the DNS to the Google Public DNS -- instructions for both Mac and Windows can be found here:

https://developers.google.com/speed/public-dns/docs/using

I was trying to upload ios8 over the weekend (wired and wirelessly) with upload times of 72+ hours, and timing out after an hour or so.

I just changed the DNS following the instructions (and be sure to reboot after you change them), and I downloaded the ios8 update in 6 minutes. Literally.

It would've been nice to know this from Verizon (considering how much we get fleeced for sub-standard service), or even Apple, but.... well, now you know how to fix it. Enjoy

RW

Verizon wouldn't have any way of notifying you of this as they aren't the ones choosing how to send you the data.

You make a request for the update, Verizon is responsible for the forward path to Apple.

Apple sends you the update, they Choose what pipe to send it to you on, they control their forward path.

edtorrey1

rjw246 - right on point - Thank You. My download is looking like it will finish before I finish writing this.

On my MacOSX tried 20-30 times to download iOS8 via iTunes from yesterday evening. I read your piece, added Google DNS to my Mac's Airport WiFi connection that is how I am connected, applied the change, and simplty stopped and started the iTunes download. Done.

JimKPhilly1

I don't understand why Verizon would be blocking or severely throttling the traffic related to apple software updates. On all of my computers/devices, trying to install or update applications from Apple take a ridiculously long time (a 1.5GB download over FIOS was listed as going to take 2-3 days to download!). As soon as I activate a VPN connection on any of the devices, the speed immediately improves (the 1.5GB download took less than 10 minutes on the VPN connection). I don't notice this with anyother service (though I suspect Netflix is slower than it should be).

Please stop throttling our traffic. We're paying for bandwidth each month, not the bandwidth that Verizon decides we should get to use.

JimKPhilly1

I did set the GoogleDNS first, and tried openDNS as well. No difference until I used VPN.

CaptainSTX

For those individuals that think that changing DNS servers is a cure all for speed problems I suggest that you download the free DNSBench.exe utility.

It will test 4,849 DNS servers and tell you which is fastest in your location. I just ran a test and the fastest DNS server was 8.8 ms and number fifty tested in at 42.7 ms. Of course resetting your DNS servers to the three fastest on the list is a game of whack a mole in that the three fastest now probably won't be the three fastest next time you test.

Probably the most valuable thing about testing is that when you go to Steve Gibson's site to download the utility and run it there is a lot of informative information about what DNS servers actually do and how to select DNS severs.

My experience has been to pick two from Google and one from OpenDNS. While the servers I have selected might not always test in the top twenty five they have been reliable for me. I personally can't relly tell the difference in how quickly a web site resolves if it only takes 15 ms or 25 ms for my DNS query to be resolved.

If you think your DNS server is holding you back compare how quicky a site reolves if you use a DNS server or you type in the numeric public IP for the site.

Hubrisnxs

This particular issue isn't about the actual name resolution piece built into DNS, but the Geo Locate portion that works with CDN's and direct''s you to one server in the east coast or west coast based on the DNS address.

CaptainSTX

It sounds wonderful that using a different DNS provider would magically take your East coast connection and instead connect using a DNS server on the West coast.

At least for Google, when you use one of their public DNS servers, the information quoted below is from their site is what actually happens according their list of FAQ.

"How does Google Public DNS know which data center to send me to?Google Public DNS uses anycast routing to direct all packets to the closest DNS server. For more information on anycast routing, see theWikipedia entry."

I assume OpenDNS must do something similar.

To get a California DNS server when you are located in New York would require that you find a West Coast Only public DNS server and use it.

If you are using a VPN then that is a diifrent scenario and you probably would want to use the DNS server of the VPN provider.

Hubrisnxs

Just to be clear we are talking about the same thing. We aren't in fact talking about DNS. We are talking about CDN's and how DNS plays and impact with how they deliver content.

Fundamental aspect of how DNS and CDNs work. When you get pointed to a specific DNS name for a server, your PC contacts the DNS server and requests that IP associated with that host. If the server doesn't happen to have that information already cached, it looks up the authoritative DNS server for the domain, and asks that server what the IP is. On something that isn't distributed, that is always going to be the same IP, no matter where you're coming from. No matter where you are, www.dslreports.com is always going to come up as 64.91.255.98. On a distributed network, however, the DNS server knows the IP of your DNS server, and knows that it is coming from Verizon FiOS in the PA/NJ/DE region (because the assigned DNS server is always close to the user to minimize lookup times). It knows that a server at some specific IP has been designated to reach users in that area, based on the ASN or however else they have it divided up. That IP gets returned to the server, and back to you. You connect to it, try to download, and see how that goes. When you switch to Google's 8.8.8.8/8.8.4.4 (or 4.2.2.1/4.2.2.2, or OpenDNS, or whatever else) you're tricking the server into thinking you're somewhere else, so you get pointed to a different server, one that is less loaded because there aren't nearly as many people hitting that server.

The actual distribution of servers and connections from them to various ISPs is a complex, ever-changing game. Akamai and the other big players pay people a lot of money solely to manage that sort of thing, and they've gotten to be pretty **bleep** good at it, and have a lot of big names like Apple, Microsoft, and Neulion (who runs most of the legal streaming sports packages) listed as customers. Apple is just starting to get into the game themselves, and they don't have it all figured out yet, especially when it comes to multi-gigabyte files.

Also that's actually a new standard that Google is introducing and I'm not terribly aware of any cdn that is using it. Good thing proof is in the pudding, was testable, verifiable and able to be duplicated time and again. Fortunately, yesterday apple off loaded the ios8 traffic and brought akamai in board to give relief to there cdn

It is performing better here, too -- It looks like they fixed it by resolving to an akamai address (63.217.211.x) rather than the apple one (17.253.x.x).

Most people shouldn't have this issue anymore, and no need to change DNS since Akamai was brought back for this part of the Apple Downloads.

CaptainSTX

With anycast typing in 8.8.8.8 anywhere in the world results in you being connected to the nearest available Google DNS server be it in Istanbul or Indianapolis.

This protocol is also available and used by some CDNs.

The various network configuration and topologies deployed across the Internet probably explains why in some cases changing DNS servers or a VPN may result in a faster connection and for some and for others no improvement.

Child Item