Get updates
Expand group
Locations
Previous page
Next page
Services
Collapse group
FiveM
Keymaster - Operational
Keymaster
Platform Server (FXServer) - Operational
Platform Server (FXServer)
Client authentication (CnL) - Operational
Client authentication (CnL)
CFX has said it's resolved and it does appear that is the case. Marking as resolved.
A fix has been implemented (as reported by cfx.re) and it appears it's online. We will consider this resolved when Cfx.re concludes the incident.
We will monitor their status page and update ours once it's back up. You can also check here for the latest results: https://status.cfx.re/
During this time, servers will not be able to authenticate your server key and your server won't start.
The node appears stable so we're marking this as resolved for now.
We've been able to get the system up due to a hardware reboot but we are still investigating the logs to see what caused the machine to go down. RN27 will resume operation unless specified otherwise. We will monitor the system for the time being.
We've been informed techs are being sent to inspect the server and bring it back online. We will keep you informed.
We have been made aware via our monitoring stack that RN27 Texas is down. We are investigating the cause and escalating this issue upstream to get the server checked to see why it's down and resolve the issue as soon as possible! We'll keep you updated.
Seems like everything is working as intended and the overall load of RN27 is within acceptable levels.
The machine is back online. We identified there was a faulty fan in the system which caused the machine to overheat. This fan has now been replaced and the system is back online. We will continue to monitor system performance and move some servers to our new RN30 Texas Node to lighten the load on RN27 for improved performance.
We have been informed the tech is on-site. We will update you once we have a newer update on the situation.
We are still waiting for an on-site tech to attend to our server.
We are aware that RN27 is down and have identified the cause. We are working with the datacenter to get it back online, but due to delays, we are still waiting for it to be brought back online. We will keep you all up to date.
This incident has been resolved.
We are continuing to monitor the attacks against our VA network.
Our engineering teams have recently pushed out a new feature set to automatically enable our enhanced protection on a per-server basis. We will be backfilling servers tomorrow morning (restarting your server tonight can also adopt the protection if you don't want to wait until tomorrow). The more servers we adopt into our enhanced protection, the less room there is for an attack to hurt. We will ensure everything is locked down ironclad tomorrow.
To also provide some additional clarity, enhanced protection is not a new subscription plan or something like that, but rather our game specific filtering which will allow our network to thwart attacks in milliseconds rather than seconds (like the 60 second downtime sprints we've been seeing).
We will keep everyone updated with any further developments. Thank you!
For the past few days or so, our network engineering team has been actively handling DDoS attacks against the VA network.
We believe the attacks have concluded and have fully identified the perpetrator. We will continue to monitor the network and be available to provide any manual intervention necessary. In the meantime, our team is still working around the clock to improve our automated DDoS mitigation strategies and response time.
We will keep everyone updated with any further developments.
For the past day or so, our network engineering team has been actively handling DDoS attacks against the VA network. We are aware of these attacks and have a strong understanding of their scope.
If you are seeing your server or our panel go down for around 60 seconds and then come back, this would be the cause. Please bare with us during this period - all our teams are working around the clock to improve the latency response of the network.
Maintenance has been completed successfully. We will be writing a post-mortem detailing this incident later.
We will be monitoring our systems for the next 24 hours to ensure reliability - for now, we are looking good. Thank you so much for dealing with us throughout the entire incident.
A fix has been implemented and we are now recovering the services.
The root cause of the downtime has been identified - we are performing the fixes necessary now
Maintenance is now in progress
We are planning to upgrade our Kubernetes VA cluster in the next few minutes. We expect zero downtime during this upgrade - we always like to inform our customers before we make any adjustments.
Everything appears to be working correctly. This incident is now considered resolved.
A fix has been implemented. We will monitor to make sure it will continue to keep the nodes up.
We have identified the issue and are implementing a fix.
We are currently investigating this incident.
Network traffic is back to routing through our primary upstream. This incident has been resolved.
Our primary upstream has returned back online a few hours ago and seems to be stable. Tomorrow morning, we will transparently re-route all traffic back through our primary upstream, meaning there will be no downtime at all.
Today at 11:51am there was an outage of our BGP sessions with our upstream network provider. This outage seems to be caused by our upstream provider's network going down.
We are currently mitigating their downtime by routing our traffic via fallback upstream providers and are still monitoring the situation. So far, BGP sessions are stable again, but we will keep the current routing setup active for the rest of the day to avoid problems during the weekend.
We will keep this entry up to date and announce further developments here.
Previous
Aug 2023 to Oct 2023
Next
