Problems with FIOS Internet Going Down Every 4 - 6 Days (KIWI Syslog Extract Attached)

[leNJ3567]

Apr  9 12:07:20 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew ip address:

Apr  9 12:07:20 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew netmask:

Apr  9 12:07:20 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew gateway:

Apr  9 12:07:20 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew dns server:

Apr  9 11:17:58 2012 Wireless_Broadband_Router CONF: [71] Configuration change (WBM user Unknown (0.0.0.0) has changed security settings)

Apr  9 11:11:20 2012 Wireless_Broadband_Router ACS ACK: event = 7 TRANSFER COMPLETE

Apr  9 11:11:20 2012 Wireless_Broadband_Router ACS ACK: event = 1 BOOT

Apr  9 11:11:20 2012 Wireless_Broadband_Router Switch to BACK-OFF state: inform_failures = 2, delay_time = 12938s, remain_time=1237s, random_time=5963s

Apr  9 11:07:33 2012 Wireless_Broadband_Router ACS ACK: event = 7 TRANSFER COMPLETE

Apr  9 11:07:33 2012 Wireless_Broadband_Router ACS ACK: event = 4 VALUE CHANGE

Apr  9 11:07:34 2012 Wireless_Broadband_Router mt_cwmp: session error: No error

Apr  9 11:07:34 2012 Wireless_Broadband_Router mt_cwmp: CMS inform failure

Apr  9 11:07:34 2012 Wireless_Broadband_Router Switch to BACK-OFF state: inform_failures = 1, delay_time = 225s, remain_time=6975s, random_time=225s

Dec 14 19:00:09 2007 Wireless_Broadband_Router update_conn_wizard_status:120: Conn Wizard [eth1]: DHCP detected

Dec 14 19:00:09 2007 Wireless_Broadband_Router dhcp client: WAN Ethernet renew ip address:

Dec 14 19:00:09 2007 Wireless_Broadband_Router dhcp client: WAN Ethernet renew netmask:

Dec 14 19:00:09 2007 Wireless_Broadband_Router dhcp client: WAN Ethernet renew gateway:

Dec 14 19:00:09 2007 Wireless_Broadband_Router dhcp client: WAN Ethernet renew dns server:

Dec 14 19:00:09 2007 Wireless_Broadband_Router Got new time: Apr  9 11:07:21 2012

Dec 14 19:00:08 2007 Wireless_Broadband_Router mt_conn_wizard_init_devices:412: Conn Wizard [eth1]: Start. Detect DHCP, Disable PPP

Dec 14 19:00:08 2007 Wireless_Broadband_Router update_conn_wizard_status:120: Conn Wizard [eth1]: Disable PPPOE, Enable DHCP.

Dec 14 19:00:08 2007 Wireless_Broadband_Router mt_conn_wizard_open:555: Conn Wizard [eth1]: Wizard is being opened.

Dec 14 19:00:08 2007 Wireless_Broadband_Router update_conn_wizard_status:120: Conn Wizard [eth1]: Disable PPPOE, Enable DHCP.

Dec 14 19:00:08 2007 Wireless_Broadband_Router eth1: link up, device will be up

Dec 14 19:00:07 2007 Wireless_Broadband_Router clink1: LOF 1000Mhz

Dec 14 19:00:07 2007 Wireless_Broadband_Router clink1: link up, device will be up

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/rg_stp.o 0xbf1b6000 -s .data 0xbf1b8e48 -s .bss 0xbf1b8fc0

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/tcp_mss.o 0xbf1ba000 -s .data 0xbf1baa1c -s .bss 0xbf1baba0

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/rg_dhcp_pktfil.o 0xbf1bc000 -s .data 0xbf1bd430 -s .bss 0xbf1bd5a0

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/rg_ipv4.o 0xbf1bf000 -s .data 0xbf1bf470 -s .bss 0xbf1bf5e0

Dec 14 19:00:01 2007 Wireless_Broadband_Router IPV4 device driver registered

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/pppoe_relay.o 0xbf1c1000 -s .data 0xbf1c4489 -s .bss 0xbf1c45e0

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/rg_pppoe_relay.o 0xbf1d8000 -s .data 0xbf1d8cbc -s .bss 0xbf1d8e40

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/usb_storage.o 0xbf1da000 -s .data 0xbf1e05e8 -s .bss 0xbf1e1ba0

Dec 14 19:00:01 2007 Wireless_Broadband_Router Initializing USB Mass Storage driver...

Dec 14 19:00:01 2007 Wireless_Broadband_Router usbcore: registered new driver usb-storage

Dec 14 19:00:01 2007 Wireless_Broadband_Router USB Mass Storage support registered.

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/btn.o 0xbf1e3000 -s .data 0xbf1e3cf0 -s .bss 0xbf1e3e80

Dec 14 19:00:01 2007 Wireless_Broadband_Router insmod: add-symbol-file build/debug/qos_ingress.o 0xbf1e5000 -s .data 0xbf1e5560 -s .bss 0xbf1e56e0

Dec 14 19:00:01 2007 Wireless_Broadband_Router Requested WAN mode is identical to the current mode - doing nothing...

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth0: mac address changed

Dec 14 19:00:01 2007 Wireless_Broadband_Router pool #1: pkt_size=1536, buf_size=1552 - 9216 of 9216 buffers added

Dec 14 19:00:01 2007 Wireless_Broadband_Router pool #3: pkt_size=256, buf_size=272 - 8176 of 8176 buffers added

Dec 14 19:00:01 2007 Wireless_Broadband_Router port 1: setting MAC to 0x2, status: 0x243b

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth0: link up

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth0: started

Dec 14 19:00:01 2007 Wireless_Broadband_Router device eth0 entered promiscuous mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router Bridge Managed Ports(eth0): 5

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/0 enters blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/1 enters blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/2 enters blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/3 enters blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/4 enters blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/4 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: port eth0/3 in BLOCKING mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: port eth0/2 in BLOCKING mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: port eth0/1 in BLOCKING mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: port eth0/0 in BLOCKING mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/3 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/2 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/1 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/0 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router Bridge Managed Ports(eth0): 5

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/2 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/1 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router STP: eth0/0 exits blocking mode

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth1: mac address changed

Dec 14 19:00:01 2007 Wireless_Broadband_Router pool #0: pkt_size=1536, buf_size=1552 - 9216 of 9216 buffers added

Dec 14 19:00:01 2007 Wireless_Broadband_Router pool #3: pkt_size=256, buf_size=272 - 8176 of 8176 buffers added

Dec 14 19:00:01 2007 Wireless_Broadband_Router port 0: setting MAC to 0x1, status: 0x2c3d

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth1: link up

Dec 14 19:00:01 2007 Wireless_Broadband_Router eth1: started

Dec 14 19:00:01 2007 Wireless_Broadband_Router unhandled IOCTL 8946

Dec 14 19:00:01 2007 Wireless_Broadband_Router MoCA phy 10 reset:1, gpio:21

Dec 14 19:00:01 2007 Wireless_Broadband_Router MoCA phy 10 reset:0, gpio:21

Dec 14 19:00:01 2007 Wireless_Broadband_Router qos_ingress_defaults_set: alloc not atomic in interrupt

Dec 14 19:00:01 2007 Wireless_Broadband_Router unhandled IOCTL 8946

Dec 14 19:00:01 2007 Wireless_Broadband_Router MoCA phy 11 reset:1, gpio:22

Dec 14 19:00:01 2007 Wireless_Broadband_Router MoCA phy 11 reset:0, gpio:22

Apr  9 11:05:55 2012 Wireless_Broadband_Router Cannot find cache entry for mac ret=-1

Apr  9 11:05:56 2012 Wireless_Broadband_Router Cannot find cache entry for mac ret=-1

Apr  9 11:05:56 2012 Wireless_Broadband_Router Cannot find cache entry for mac  ret=-1

Apr  9 11:05:53 2012 Wireless_Broadband_Router RATELIMIT: 2 messages of type [61] NAT Error: Not enough free memory to add a new connection suppressed in 1 second(s)

Apr  9 11:05:52 2012 Wireless_Broadband_Router RATELIMIT: 3 messages of type [61] NAT Error: Not enough free memory to add a new connection suppressed in 1 second(s)

Apr  9 11:05:51 2012 Wireless_Broadband_Router RATELIMIT: 3 messages of type [61] NAT Error: Not enough free memory to add a new connection suppressed in 1 second(s)

Apr  9 11:05:49 2012 Wireless_Broadband_Router OUT: [61] NAT Error: Not enough free memory to add a new connection ([61] NAT Error: Not enough free memory to add a new connection: )

Apr  8 14:07:21 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew ip address:

Apr  8 14:07:21 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew netmask:

Apr  8 14:07:21 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew gateway:

Apr  8 14:07:21 2012 Wireless_Broadband_Router dhcp client: WAN Ethernet renew dns server:

Apr  8 14:02:39 2012 Wireless_Broadband_Router estream_connect_ssl_efd 45: failed ssl connetion

Apr  8 13:51:27 2012 Wireless_Broadband_Router estream_connect_ssl_efd 45: failed ssl connetion

Apr  8 13:51:24 2012 Wireless_Broadband_Router Unknown PTR name format

Apr  8 13:47:45 2012 Wireless_Broadband_Router estream_connect_ssl_efd 45: failed ssl connetion

Apr  8 13:46:22 2012 Wireless_Broadband_Router estream_connect_ssl_efd 45: failed ssl connetion

Apr  8 13:45:51 2012 Wireless_Broadband_Router estream_connect_ssl_efd 45: failed ssl connetion

Apr  8 13:33:06 2012 Wireless_Broadband_Router CONF: [71] Configuration change (WBM user Unknown (0.0.0.0) has changed security settings)

Apr  8 13:30:18 2012 Wireless_Broadband_Router ACS ACK: event = 7 TRANSFER COMPLETE

Apr  8 13:30:18 2012 Wireless_Broadband_Router ACS ACK: event = 2 PERIODIC

Apr  8 13:30:18 2012 Wireless_Broadband_Router Switch to BACK-OFF state: inform_failures = 53, delay_time = 4172s, remain_time=3505s, random_time=3695s

Apr  8 13:08:06 2012 Wireless_Broadband_Router Cannot find cache entry for mac ret=-1

Apr  8 13:08:06 2012 Wireless_Broadband_Router Cannot find cache entry for mac ret=-1

Apr  8 13:08:06 2012 Wireless_Broadband_Router Cannot find cache entry for mac 00:ret=-1

Apr  8 13:08:06 2012 Wireless_Broadband_Router Cannot find cache entry for mac ret=-1

Apr  8 13:08:06 2012 Wireless_Broadband_Router CONF: [71] Configuration change (WBM user Unknown (0.0.0.0) has changed security settings)

[mackworth]

What router rev do you have?  How many and what kind of devices do you have?

Also, did you have to do anything special to get kiwi logging to work?  Any changes settings?

[leNJ3567]

I have Rev I of the M1424wr and firmware version 20.2.  I have a netgear switch with 11 inputs used: 7 PC's, 2 remote wireless routers I'm using as wireless access points, and two home theater devices.  I had to set up syslog to read from my router's log.   I have a wired network, and I'm using the MoCa interface between the ONT and the router and CAT 5e Ethernet cable for my network.

[mackworth]

I am having the same issue(I suspect) with a Rev I, a similar number of machines and access points and I have an HD Home Run Prime cable card tuner.

I tried to get kiwi working but for some reason the machine that is running kiwi isn't receiving the messages.

Apr  9 11:05:53 2012 Wireless_Broadband_Router RATELIMIT: 2 messages of type [61] NAT Error: Not enough free memory to add a new connection suppressed in 1 second(s)

It sounds like its running out of memory because there are too many connections.  This could be because of torrents or other activity (in my case I think its the HD Home Run Prime) and the router not doing a good enough of job cleaning out old connections.

I am almost at the point where I am going to see if I can turn my rev I into a bridge from the ONT and use my old router.  I don't have any STBs, it is not a big deal for me to do that.

[leNJ3567]

To get KIWI to work you need to go into the Actiontec Advanced page and input the IP address of the computer where the KIWI syslog application is running.  Did you do that?  There are instructions on the DLS Reports Broadband website that tell you how to do this.  If you need more information on this let me know.

[viafax999]

---

@mackworth wrote:

I am almost at the point where I am going to see if I can turn my rev I into a bridge from the ONT and use my old router.  I don't have any STBs, it is not a big deal for me to do that.

---

If you don't have any STB's you don't need the VZ router if you have an ethernet connection to the ONT. 

If your ONT connection is coax you can request VZ to provision it for ethernet and when that is done just connect your own router directly to the ethernet port.

[mackworth]

Sorry @leNJ3567, I didn't mean to hijack your thread.

Anyway, @viafax999, I live in an apartment, so getting cat5 from the ont isn't easy.  Anyway, I got a Moca adapter and connected that to the line from the wan, and am now using my own router.  Hopefully I won't have any more issues.

[leNJ3567]

The root of the problem that had plagued my home network for 4-months has been successfully identified and resolved!

Thorough investigation using the KIWI Syslog tool verified that at the time of failure my NAT was overflowing causing the router to lock up.  I substituted a Netgear router in place of the Actiontec M1424WR-Gen3 Rev I and that substitution solved my network problem.  Everything is now working as expected indicating that the problem was not the size of my network, large, nor the type switches used on my network.  I no longer have network dropouts.  The problem was due to the Actiontec router.  Please read what I found below.

For me one key thing I discovered with my Actiontec M1424WR-Gen3 Rev I, was that the firmware that shipped with the router, 40.19.22, got automatically rolled back by the Verizon network when the technician first installed the router to an older version of the firmware which was intended for an older version of the router.  That is, my router didn't have the correct firmware version.

I have attempted to obtain the correct firware version, 40.19.22, with no success to date.  If anyone can determine where on the internet this firmware can be downloaded, please let me know.  Verizon has a firmware download site on the web but they don't include the firmware for the Gen3 Rev I router.

Check the firmware version currently loaded on your router to see whether it is the correct version, 40.19.22, or an older version like, 40.19.20.2.

[leNJ3567]

The root of the problem that had plagued my home network for 4-months has been successfully identified and resolved!

Thorough investigation using the KIWI Syslog tool verified that at the time of failure my NAT was overflowing causing the router to lock up.  I substituted a Netgear router in place of the Actiontec M1424WR-Gen3 Rev I and that substitution solved my network problem.  Everything is now working as expected indicating that the problem was not the size of my network, large, nor the type switches used on my network.  I no longer have network dropouts.  The problem was due to the Actiontec router.  Please read what I found below.

For me one key thing I discovered with my Actiontec M1424WR-Gen3 Rev I, was that the firmware that shipped with the router, 40.19.22, got automatically rolled back by the Verizon network when the technician first installed the router to an older version of the firmware which was intended for an older version of the router.  That is, my router didn't have the correct firmware version.

I have attempted to obtain the correct firware version, 40.19.22, with no success to date.  If anyone can determine where on the internet this firmware can be downloaded, please let me know.  Verizon has a firmware download site on the web but they don't include the firmware for the Gen3 Rev I router.

Check the firmware version currently loaded on your router to see whether it is the correct version, 40.19.22, or an older version like, 40.19.20.2.

[smith6612]

Heh, go figure. It sounds like Verizon installed beta firmware to the router. ActionTec is saying  40.19.22 is supposed to be what the Rev. I has on it considering it is newer, beefier hardware that should have no NAT table issues what so ever. I suppose that explains what has been going on with a handful of Rev. I units. ActionTec does have the source code up for the firmware, but it needs to be developed a bit more before the actual firmware can be loaded to the router via a Serial/TFTP connection. Bummer.

Whatever Verizon did, I do hope they fix it.

[mackworth]

My router had 40.19.22 and I still had this problem.  I have been running since my post without issue using an asus RT-N56U.