April 26, 2022 DNS Outage

Preliminary Cause and Analysis Report

This morning, EnCirca's DNS Service went down for about 89 minutes. Service was restored at 7:21am EST.

The problem appears to have originated from our Anycast network vendor. EnCirca uses the internet’s largest Anycast Network provider, with 242 Points of Presence (PoPs) Internet Exchange Points.

Here is the preliminary response from our vendor about the outage: "Today we had an incident with our config manager due the logical flow error, which triggered a process that emptied out the zone configs for our auto-provisioned zones. We have fixed the config generation error and added multiple checks to avoid any similar future incidents."

We are awaiting more detailed assessment and mitigation details from our vendor regarding this outage. EnCirca will be sending another report once we have more information along with our own mitigation plan and recommendation to avoid a recurrence in the future.