IPv6 expanding! FINALLY!

Anonymous
Not applicable

After years of testing in a couple of limited areas of Virginia, along with Waltham MA, it appears that Verizon may finally be starting to roll out IPv6 support to more areas. Over the past couple of weeks on another community site, there have been six reports from users in five Maryland areas near Baltimore that IPv6 is now working for them. In Virginia, reports have come from two users in the Yorktown/Newport News area, as well as Sterling in Northern VA. 

Verizon had sent emails to Fios business accounts late last year about IPv6 rolling out this year (an account in MD had received a January date, and an account in NJ had received a February date). Obviously those dates were missed, but there are also reports of new emails being sent (a NY business customer recently received an email with a date in June).

The implementation is done on a per-CO (central office) basis, so it's relatively localized (large cities often have multiple CO's serving them), though they could upgrade multiple nearby CO's at the same time. There does seem to be a pattern of at least one momentary overnight outage in the days leading up to it being enabled. Based on the original emails sent to businesses, it looks like they're starting in Virginia/MD and moving north.

Additionally, areas receiving IPv6 seem to be seeing a fix for the 2-hop ICMP Traceroute issue that's been plaguing the network for a few years. So if you notice your traceroutes returning complete results now, try enabling IPv6 and see if you have it.

I don't believe Verizon routers have IPv6 enabled by default, so if you want to see if you have IPv6 in your area, you'll probably need to enable it in your router's settings first. It's possible that Verizon could enable IPv6 in their routers in the future. A note that some older routers may not perform at full speed with IPv6 enabled because of hardware acceleration that only works for IPv4. 

If you don't use a Verizon router, here are links for info on setting up pfSense and Ubiquiti EdgeRouter. If you use something else, the general requirement is to send a DHCPv6 Prefix Delegation request for a /56 prefix. Then allocate /64 prefixes from the /56 received for the local network(s) you have. Using DHCPv6 to get a WAN address is not necessary, Verizon does not provide a separate global WAN IPv6 address. Link-local on the WAN interface is more than sufficient for IPv6 to function. Also make sure to set your firewall to allow IPv6 connections from your local network(s) to the internet.

I've been fortunate enough to be in a test area, so if there are questions, I'll do my best to answer them.  I don't use a Verizon router though (just pfSense), so I may not be able to answer router-specific questions.

33 Replies
Anonymous
Not applicable

Just an update on the past two weeks... 

Between RIPE Atlas probes and reports from users of the other community site, there's been continued expansion of IPv6 availability around the immediate DC area, including a CO within DC. APNIC's IPv6 statistics tracker shows that over the past two weeks, Verizon's network has gone from 3% IPv6 capable to 9% IPv6 capable.

The issue with relying on user reports is that users need to know how to check for or enable the functionality, or there may not be community members in some areas at all. So it may be available in more areas than have been reported. But with that said, here are the new CO's reported as being enabled for IPv6 over the past two weeks:

DC:

- "Dupont"

Maryland:

- Silver Spring

- Gaithersburg

- Bethesda "Wood Acres"

- Hyattsville "Chillum"

Virginia:

- Arcola

- Herndon (north of Dulles toll road)

- Herndon "Dulles Corner" (south of Dulles toll road)

- Springfield

- Arlington (Ballston and surrounding areas)

Anonymous
Not applicable

Much of Fios-served Northern VA and MD around the DC and Baltimore areas look to be complete. There are still a small number of CO's in MD and VA that may not be there yet, but they'll probably be completed very soon. There have been few reports from elsewhere in Virginia (Richmond and Newport News areas). I've only seen reports from two or three CO's in DC itself (there are 15 CO's within DC limits), so not sure if they're all done or if there's more to come. 

Yesterday I noticed a possible sign of progress in Pennsylvania, as two RIPE Atlas probes in the Harrisburg area began showing full ICMP traceroutes. Today, a DSLR user has reported IPv6 in Pittsburgh, but they didn't get more specific than that (Pittsburgh and suburbs have like 30 CO's providing service to them).

Progress!!!

Cang_Household
Community Leader
Community Leader

Yes. PA is added to the IPv6 expansion list.

Anonymous
Not applicable

And to close out the month of July, more reports from PA and NJ!  Most of the NJ reports have been from Southern NJ, though one of the first NJ reports (part of the previous post) was across the Hudson from Manhattan. 

Also, here's a graph from APNIC (The Asia-Pacific Network Information Center; they give out the IP addresses for the Asia/Pacific part of the world) showing Verizon's growth in IPv6 rollout over the past few months!

imageAPNIC Labs graph of Verizon IPv6 growth

coninram
Newbie

Got IPv6 in the Northern NJ area. I don't know why but I checked after my IP changed after upgrading to gigabit and saw that we got IPv6 now.

0 Likes
Reply
Anonymous
Not applicable

Continued progress in PA... A number of confirmed reports from the Harrisburg, Pittsburgh, and Philadelphia areas. Still plenty to be done in at least Pittsburgh and Philadelphia, but progress is being made!

A couple of additional reports from previously unreported CO's in MD and VA... but I think most of the work in DC, MD and VA is complete at this point.

I think there are some issues that need to be fixed in some of the original testing areas. Leesburg VA and a couple of CO's west of Richmond don't have the full ICMP traceroute yet, and Ashburn seems like it might have exhausted its IPv6 prefix delegation range, as some are reporting DHCPv6 issues (non-Verizon routers, so they can see logs indicating the problem).

What's next???

Anonymous
Not applicable

PA IPv6 expansion is continuing, both in the Pittsburgh and Philadelphia areas... and a couple of reports have been received in NJ, and one in DE so far.

SoNic671
Enthusiast - Level 2

I am a Fios customer in Newport News, with G1100, firmware 02.03.00.13.

I just enabled IPv6 today (7/31/2022) and.. it works! It was disabled by default.

SoNic671
Enthusiast - Level 2

Today I had all kinds of hiccups in my network, WhatsApp not linking big uploaded on my desktop, YT stuttering.

For now I turned off the IPv6... maybe next year will be better?

0 Likes
Reply
bobbytables
Enthusiast - Level 1

IPv6 is not the problem; it is the solution. Please turn IPv6 back on.

0 Likes
Reply
bobbytables
Enthusiast - Level 1

Youtube is available via IPv6, and has been for years. Same with facebook & whatsapp. IPv6 on those sites works fine.

0 Likes
Reply
SoNic671
Enthusiast - Level 2

I found out, reading through topics posted here, that it was an issue with the hardware acceleration on my Intel I210 NIC.

My Dell has also an Intel 82579LM NIC and that one seems unaffected. 

LE: It is still glitching randomly, in YT is very noticeable.

0 Likes
Reply
Cang_Household
Community Leader
Community Leader

@SoNic67 wrote:

For now I turned off the IPv6... maybe next year will be better?


No, unless you replace your faulty Intel PHYs and NICs. Virtually all Intel NICs have TCP checksum offload, so replacing them is strongly recommended.

Anonymous
Not applicable

@Cang_Household wrote:

@SoNic67 wrote:

For now I turned off the IPv6... maybe next year will be better?


No, unless you replace your faulty Intel PHYs and NICs. Virtually all Intel NICs have TCP checksum offload, so replacing them is strongly recommended.


Some would say that it's Alcatel-Lucent's ONT that's faulty, since it shouldn't be sending out packets with bad data. Others will say it's Intel's issue for not handling bad data better. Both companies will say it's the other's fault, and neither will fix their issue. And because the percentage of users experiencing the issue is tiny in the grand scheme of things, we're the ones stuck with the bad experience.

Regardless, you can turn off the hardware offloading for the Intel NIC in Windows Device Manager (or through other means, if using another operating system) and the issue should be resolved. 

Also, there seems to be an issue a number of people in New Jersey are currently seeing... it looks to be a routing issue on a regional level, as all of the IPv6 traceroutes stop receiving responses after 2 or 3 hops, and all seem to stop at a point where the packets would likely be leaving Verizon's network, indicating an issue with routing on Verizon's end, or the next provider outside of Verizon (which would likely be based on data Verizon is providing to them).

Cang_Household
Community Leader
Community Leader

If you connect an Intel NIC directly to the ONT, the problem would disappear. Does this mean the fault is with routers after all?

Anonymous
Not applicable

Possibly? I think routers with hardware-accelerated NAT (which many older routers, including the G1100, not sure about the 3100, do have) basically change some element of the packet based on the NAT table, then send it on.  IPv6 doesn't use NAT, so maybe the router does something to the packet that it shouldn't be doing? I don't know all the detals about the internals of some of the various consumer/small biz routers though, so I could be completely off.

Not sure how the blame got put on Alcatel-Lucent if so though...

Also... the IPv6 routing issue in NJ seems to have been resolved... reports from people that had been having issues yesterday are that things are working for them now.

0 Likes
Reply
ChristopherDeez
Enthusiast - Level 1

You know what would've been nice?

You know what would've REALLY NICE?

It would've been REALLY NICE if Verizon let its end users know that a potentially breaking change was being pushed to their routers. 

I have literally spent 1 month chasing down and even reformatting a brand new Windows 11 box working perfectly since February 2022 with an Intel Ethernet controller (hint hint) had issues with all web browsers. 

This computer wasn't a work computer and is primarily gaming / media center so I could only research a bit as it wasn't affecting my work (on Wifi and / or non Intel ethernet wired controllers).

I find it incredible that Intel's "addressing" of this issue was to release an unreasonable solution https://www.intel.com/content/www/us/en/download/19174/disabling-tcp-ipv6-checksum-offload-capabilit... to it and placing blame on the ONT. 

I find it incredibly frustrating that Verizon would make a change like this and not notify its users of such, especially with so many Intel cards out there. 


I only figured this out because I had a friend with 2 computers (1 brand new, 1 10 years old) exhibiting the same issues on Verizon FiOS.  Incredible... thanks for the waste of time and thanks Intel too.

0 Likes
Reply
Cang_Household
Community Leader
Community Leader

@ChristopherDeez wrote:

It would've been REALLY NICE if Verizon let its end users know that a potentially breaking change was being pushed to their routers. 

...

Incredible... thanks for the waste of time and thanks Intel too.


You can come to this Forums earlier, as we are providing Tier 4 support (unofficial) here.

Pushing IPv6 does not cause the Intel cards to break. Intel cards are broken already and the IPv6 merely allows the defect to surface.

I agree with you that VZ needs to post some public notices; however, don't you think this is going to lead to a flood of calls asking why is/isn't my network broken? Considering the IPv6 has not reached all serving areas yet, somebody would ask when would they get IPv6? Why isn't my Intel NICs broken yet? Why the rollout is so slow? Why are you doing the rollout after all? What is IPv6? etc... etc...

Or let's mail some IPv6 and networking crash course pamphlets to customers and hope they read.

ChristopherDeez
Enthusiast - Level 1

@Cang_Household wrote:

@ChristopherDeez wrote:

It would've been REALLY NICE if Verizon let its end users know that a potentially breaking change was being pushed to their routers. 

...

Incredible... thanks for the waste of time and thanks Intel too.


You can come to this Forums earlier, as we are providing Tier 4 support (unofficial) here.

Pushing IPv6 does not cause the Intel cards to break. Intel cards are broken already and the IPv6 merely allows the defect to surface.

I agree with you that VZ needs to post some public notices; however, don't you think this is going to lead to a flood of calls asking why is/isn't my network broken? Considering the IPv6 has not reached all serving areas yet, somebody would ask when would they get IPv6? Why isn't my Intel NICs broken yet? Why the rollout is so slow? Why are you doing the rollout after all? What is IPv6? etc... etc...

Or let's mail some IPv6 and networking crash course pamphlets to customers and hope they read.


Hey There,

I understand that Intel wired NICs were already broken and that implementing IPV6 manifests the issue.

I appreciate these forums, and the help others provide, certainly. 

Here is a snippet of an email received in August 2021 regarding port forwarding and new router SW being pushed:

 	Account number ending in:

Hi,

Verizon is upgrading the software on your Fios Router to provide Wi-Fi and security enhancements as well as to refresh the user interface for configuring your router. We expect to make this update in your area on August 20, 2021.

Be aware that when the Fios Router software is updated, it may impact some of the settings that you have configured through your router user interface. Specifically, if you defined Port Forwarding rules, you may need to re-enter some of them through the upgraded user interface, including:

    Any rules that were drafted within the Advanced - Network Settings - Port Forwarding Rules section.
    Any rules which contain source and destination ports which do not match.

On the regional IPv6 rollouts, they could have at least sent out emails by region saying IPV6 is being enabled and if you experience any connectivity issues particularly on wired eithernet connections it could be X, Y, or Z.

I have a brand new 2022 Mobo that exhibits this issue.  My friend had a brand new computer, and a 10 year old computer, all with Intel Nics, all wired (wired is superior, is it not?), all started manifesting the Intel issue. 

I've been reading these forums and I've read the stories about Verizion customer support being clueless about this and sending out new Verizion equipment etc to fix it, when a simple head's up would alleviate the significant amount of issues this is going to create for wired Intel Nic customers.  It's a disgrace that Intel didn't even fix the issue on chips being used in 2022 Mobos. 

My point is, June 29th, 2022, brand new computer wired Intel Ethernet working perfectly. July 1st 2022, brand new computer, NO changes that I am aware of, all Gmail services broken and dropping packets.  2 other computers (1 wired, 1 wireless) working fine.  What is the first thing I'm going to think of.  It's not Verizon silently pushing a change to their router at my house, it's that I have a H/W issue with new computer that randomly manifested with a Windows update or something else. Ugh. I am just so frustrated about it...still.  I didn't think to look at the Verizon forums, because after the last email about a SW update, I thought for sure I'd get additional notifications regarding updates/changes. 


Thanks

Anonymous
Not applicable

A report late last week from MA came in (and today the IPv6 test area in MA received the update to get full ICMP traceroutes too)... additional reports today from two areas in RI and our first report from NY in the Buffalo area.

0 Likes
Reply