xtreme-networks

Network Events

Currently affecting the system

Network statusThere are no events currently affecting the system.

Future / Scheduled events

network-status-warningThere are no future events scheduled.

Sign up to receive email alerts

Historic events

Traffic Disruption
Date: 13 Aug 20 07:05 – 13 Aug 20 07:17
Affected services:
This morning some configuration changes were completed – which should have had no impact on traffic flows through our core network.
On applying the configuration changes we noticed within minutes changes to normal traffic flows and we reverted the changes immediately.

We sincerely apologise for the inconvenience and interruption to business this caused you.

 

Chorus Upgrades
Date: 12 Sep 19 01:00 – 12 Sep 19 12:00
Affected services:
This Thursday morning between 1 AM and 6 AM UFB connections in the lower North Island will experience an outage of up to 10 minutes as Chorus are installing updates to resolve faults that have been reported on some connections.

If you have any connectivity issues after 6 AM on Thursday please do not hesitate to give us a call.

 

UFB Outside Wellington
Date: 20 Aug 19 07:27 – 20 Aug 19 10:04
Affected services:

Update 10:04 All services have now been restored.
Please do not hesitate to contact us if you are still aware of any issues.

We currently have an issue where our UFB connections outside the Wellington region have no connectivity, our backhaul provider believes it is associated with maintenance performed overnight, they are currently investigating.

 

Palmerston North UFB
Date: 08 Jul 19 11:59 – 08 Jul 19 15:17
Affected services:
Update 08/07 15:17 – This was resolved at 12:15, Chorus advised a software configuration issue was causing the loss of services.

At Approximately 11:15 we noticed sever packet loss across our Palmerston North UFB connections.

We have a fault currently logged with Chorus and will update as we can.

 

Chorus UFB Upgrades
Date: 15 Apr 19 12:00 – 16 Apr 19 12:00
Affected services:
UFB connections from Johnsonville through to Porirua following State Highway 1, and Palmerston North and Napier, will experience a 20 minute loss of service on Tuesday morning between 1 AM and 6 AM as Chorus upgrade some core network elements.

Please do not hesitate to contact us if you have any connectivity issues after 6 AM Monday morning.

 

CityLink Issues

Date: 10 Apr 19 09:34 – 11 Apr 19 10:00
Affected services: CityLink

Update 11/04 10:00 – CityLink advised last night that they replaced a faulty SFP module on one leg of a port channel and have seen no further issues.

CityLink are currently investigating technical issues on their Wellington network causing intermittent outages.

 

Global Gateway Upgrades
Date: 22 Jan 19 16:29 – 02 Feb 19 12:00
Affected services:
Between the 23rd of January and the 2nd of February Spark Wholesale will be completing some upgrades on our connectivity to their national and international “Global Gateway” circuits.

Depending on traffic flows you might notice brief (less than 2 minutes) outages while traffic flows fail over to our other upstream providers between the hours of 1 AM and 6 AM on the 23rd and 25th of January, and between midnight and 8 AM on the 26th of January and the 2nd of February.

Other than this you should not experience any impact.

 

Data Centres Brownout
Date: 20 Dec 18 00:00 – 22 Jan 19 00:00
Affected services:
Data Centre

Good morning,

We would like to advise that our Data Centres will have a total work restriction from the 20th of December through to the 21st of January (inclusive).

The only exceptions will be for emergency rebooting of equipment or repair (or removal for offsite repair) of failed equipment.

All other services will remain as standard, but will be dependent on any third party provider’s brownout conditions.

Thank you all for your business over the year and we wish you all a safe and enjoyable summer break.

 

Urgent CityLink Outage
Date: 22 Nov 18 12:49 – 23 Nov 18 12:00
Affected services:
CityLink
CityLink have advised of urgent work required to address an issue on their network.

They have advised some connections will experience an outage of up to 4 minutes at 12:01 Friday morning.

 

CityLink Outages
Date: 01 Nov 18 07:09 – 02 Nov 18 07:05
Affected services: CityLink
Update 02/11 7:05 – CityLink are still working through repairs but all Xtreme’s customers now have service. We have been told a crane took down the overhead cables yesterday morning.

Update 9:05 – CityLink are indicating mid afternoon for full service restoration.
https://www.citylink.co.nz/fibre-cut-wellington-cbd
https://www.stuff.co.nz/dominion-post/news/108267657/downed-power-lines-in-wellington-close-roads

Update 7:30 – CityLink have confirmed they have a cut fibre.

An incident in Featherston Street (we suspect) at approximately 6:05 this morning has caused a number of outages on the CityLink network, we are waiting on an update from CityLink and will provide further information as we receive it.

 

Connectivity Issues This Morning
Date: 14 Sep 18 05:20 – 18 Sep 18 10:00
Affected services:
Update 17/09 16:00 – Tomorrow morning at approximately 6:45 we will be upgrading our handover for UFB customers outside the Wellington region to improve protection against scenarios similar to Friday. The upgrade should be seamless and customers should not notice any break in service.

Please do not hesitate to contact us if you do experience any issues.

At 5:20 this morning our monitoring systems made us aware of unusually high traffic volumes impacting on general internet access.

We tracked the flows down to one customer who we were unable to contact.

As the traffic flows were unexplainably high in relation to the speed of the customers connection, and traffic policing made no difference to our upstream utilisation we investigated further and determined that the customer was running uncontrolled UDP transfers from hundreds of international IP addresses. As we were still unable to contact the customer we made the decision to shut their connection down at 8:55.

This primarily affected UFB customers outside the Wellington region and we apologise for the disruption and inconvenience caused.

 

Chorus Firmware Update
Date: 10 Sep 18 08:00 – 11 Sep 18 12:00
Affected services:
Next week between Monday the 10th at 11 PM to Tuesday the 11th at 6 AM some Wellington region UFB connections will experience an outage of up to 23 minutes as they upgrade firmware on a fibre line terminator.

If you have any issues on the morning of Tuesday the 11th please do not hesitate to contact us.

 

Chorus UFB Optimisations
Date: 04 Sep 18 08:00 – 05 Sep 18 12:00
Affected services:
This Wednesday morning between 1 AM and 6 AM all Wellington region UFB connections will experience an outage of less than one minute as Chorus rolls out an optimisation update.

If you have any issues on Wednesday morning please do not hesitate to contact us.

 

CityLink Issues
Date: 03 Sep 18 16:36 – 04 Sep 18 12:00
Affected services: CityLink
CityLink have advised they are aware of an issue affecting some customers.
They have a plan to reboot a line card shortly after midnight but may bring this action forward if further degradation occurs.

National Disruption
Date: 29 Aug 18 10:03 – 30 Aug 18 15:25
Affected services:
Update 30/08 15:25 – Devoli are satisfied that this is resolved and their breakdown can be found at https://www.dropbox.com/s/fjny8tx1ekgkrgx/-Devoli%20RFO%20290818.pdf?dl=0

Update 17:51 – Devoli have advised the following: “Our vendor TAC has advised that we need to perform a full chassis reload to ensure stability of the device. We have scheduled this for 0100 tomorrow morning (30 August). If there are any more events before this time, we may elect to perform the reload then. We expect this will interrupt services for approx 20-30 minutes.”

Update 16:06 – UFB connections outside Wellington are experiencing continued disruptions again.

Update 11:54 – all services have been restored, we will update once again when we receive the full incident report.

Our backhaul provider for UFB connections outside Wellington is currently experiencing major issues, they are aware of it and are investigating. We have no ETA at this time.

 

Chorus UFB Changes
Date: 21 Aug 18 23:00 – 22 Aug 18 12:00
Affected services:
Chorus have advised they will be making configuration changes in their UFB network on the 21st of August starting from 11:00 PM.

This will result in an outage of up to 20 minutes on all UFB connections. If you find you have no internet connectivity on the 22nd of August please do not hesitate to call us on 0800 469 873.

 

Wellington Power Outage
Date: 03 Aug 18 12:22 – 03 Aug 18 14:25
Affected services: CityLink
Update 14:25 – Power has now been restored.

Update 12:46 – We are aware of a power outage in the Wellington CBD. This can indirectly affect CityLink connections in powered buildings depending on CityLink’s upstream path.

CityLink are currently experiencing issues in the network affecting the northern area of Wellington’s CBD. We are awaiting an update from them.

 

CityLink Maintenance
Date: 15 Apr 18 12:00 – 16 Apr 18 12:00
Affected services: CityLink
CityLink have advised a planned outage in relation to maintenance on one of their core switches.

This work concerns Wellington CityLink Ethernet services only – FibreLINK/dark fibre services are not affected.

Outage Details: Monday, 16th April 2018
Work Start Time: 00:10am
Work End Time: 01:00am

Outage Duration: It is expected that there will be an outage of up to 15 minutes

Impacted Services: All Wellington Ethernet Services.

 

UFB Handover Upgrade
Date: 11 Apr 18 12:00 – 12 Apr 18 12:00
Affected services:
At midnight Wednesday the 11th/Thursday the 12th we will be upgrading our UFB handover with Chorus to allow for future capacity. Wellington based UFB connections will experience a short outage (this should be well under one minute).

If you find your UFB connection is not working on Thursday morning please first try power cycling your firewall and if you still do not have a connection please do not hesitate to call us.

 

Core Switching Infrastructure Upgrades
Date: 15 Mar 18 12:00 – 16 Mar 18 12:00
Affected services: Data Centre

Update 2: This Thursday the 15th of March will be the final upgrade.

Update: This Thursday the 8th of March will be the final upgrade.

Beginning this Thursday the 8th of February and the following 3 Thursdays, at 11:00 PM we will be upgrading our Cisco core network switching infrastructure to stay ahead of customer demand.

Each one of our four core switches (one per Thursday) will need both switching supervisors upgraded. This process will result in a loss of connectivity for all connections (only to the switch being upgraded) for up to 5 minutes.

If you do experience any sustained loss of connectivity to your equipment over this time please do not hesitate to call us direct (Dion Hallam/021 755 873 or Mike Lee/021 755 872)