Incident Post-Mortem: GER-1 Intermittent Connectivity

Date: 29 January 2026

Duration: ~18 minutes (00:24 - 00:42 CET)

Affected: GER-1 node

Severity: Minor - intermittent packet loss, no complete outage

---

What happened

Between 00:24 and 00:42 CET on 29 January 2026, our GER-1 node experienced intermittent connectivity issues. Servers on this node remained online, but some players may have experienced brief connection drops or increased latency during this window.

Root cause

Our upstream provider experienced a fault on one of their core routers. This caused intermittent packet loss affecting traffic routed through that device - including our GER-1 node.

Timeline (CET)

Time	Event
00:21	First packet loss detected by our monitoring
00:24	Incident triggered after sustained failures
00:25	Alert received and acknowledged by our team
00:24-00:42	Intermittent connectivity - server responding, then dropping, then recovering
00:42	Incident auto-resolved after stable connectivity restored
00:47	Upstream provider confirmed router drained and issue mitigated

Impact

• Not a full outage: The node never went completely offline. Connectivity was flapping - some requests succeeded, others failed.
• Duration of actual disruption: Players would have experienced brief drops lasting seconds to a minute at most, not sustained downtime.
• No data loss: All server data remained intact.

What we're doing about it

Short-term:
- We've reviewed logs and confirmed no action was needed on our side - this was purely an upstream issue.

Long-term:
- We're evaluating multi-path routing options that could provide failover during upstream issues like this. No promises yet, but it's on our radar.

SLA status

This incident did not meet the threshold for SLA credits (15+ consecutive minutes of complete unavailability). The node remained reachable throughout, albeit with intermittent packet loss.

If you experienced significant disruption during this window, reach out to us - we're happy to discuss your specific situation.

Our providers have identified a fault within their routers.

Engineers are hard at work resolving the failing routers, connection issues should begin to recover shortly.

We will continue monitoring

We are experiencing intermittent connection issues to our GER nodes.

We are working with our upstream provider to diagnose a cause.