28 October 2011

Network reconfiguration

UPDATE 19:16 - both sql.sjkwi.com.au and mail.sjkwi.com.au are operational again.

We have now finished a network reconfiguration that has seen services on thor.sjkwi.com.au moved to our Sydney-based hosts (syd1 and syd2). These servers have greatly increased bandwidth and network connectivity so we shall hopefully see an improvement in response times and availability.

Customers do not have to do anything on their end - all services are operational on their new hosts.

As part of this reconfiguration, we have retired the following web-based services, effective immediately:
  • paste.sjkwi.com.au
  • source.sjkwi.com.au
  • users.sjkwi.com.au
The following web-based services are currently being updated to their latest versions and will come online shortly:
  • sql.sjkwi.com.au
  • mail.sjkwi.com.au
We apologise for any inconvenience this may cause.

28 August 2011

All services restored

All services have been restored after the network fault with thor.

We sincerely apologise for the inconvenience caused and we are working toward moving all critical services to a more reliable setup so this type of downtime does not occur again.

If there are any further issues please contact us and we would be only too happy to help.

26 August 2011

Packet loss on thor

As of approximately 07:10 this morning, thor is experiencing heavy packet loss on all inbound traffic.

The following services are likely affected:
  • HTTP
  • IMAP/POP3/SMTP
  • DNS
At the moment, DNS and SMTP have failed over to backup servers, but other services remain unavailable for the most part. We are trying to get to the bottom of the issue, stay tuned.

UPDATE 14:12: Packet loss is now at 100%, though outbound traffic still appears to be flowing normally. Attempts to power cycle modem had no effect. Raised problem with upstream provider and are awaiting more information.

UPDATE 27 Aug 09:27: At some point overnight packets became unroutable to server, in any direction. Upstream provider suggests possible line fault. Decision has been made to speed up planned server move to another site with working connection. DNS changes have started propagating to new subnet, and server will be moved in the next few hours.

UPDATE 29 Aug 16:52: All services restored. We sincerely apologise for the inconvenience caused and we are working toward moving all critical services to a more reliable setup so this type of downtime does not occur again.