Public DENIC mailinglist

[DENICpresse <presse AT denic.de>]

Dear Sir or Madam,

As promised, following you will find detailled information about the 
disturbance of the .de domain nameservice which occured last Wednesday. 
Answers to the most frequently asked questions have been collected for you 
in the attached FAQ-document.

Course of Events and Effects
Starting about 13:30 (CEST) on Wednesday, 12 May 2010, DENIC faced the 
situation that, depending on the service location and the domain queried, 
users sometimes received the incorrect reply “domain does not exist”. In 
such cases, the respective user could not reach the .de domain concerned 
via the domain’s address, and e-mails from or to this address were 
rejected or not sent. 
The reason why this situation occurred was an incomplete copying process 
during the regular name service data update, which is performed at 2-hour 
intervals. Due to this, an incomplete update of the name service data (a 
so-called zone file) was triggered at 12 of the 16 service locations.

The immediately activated incident response team analyzed the error and 
subsequently, starting at 14:20, successively switched off those locations 
that were giving faulty responses. 
Since it was not immediately clear whether the reason for the zone defect 
was a faulty database or a fault in the zone generation process, the 
registration systems were also temporarily put on hold. The latter was 
also decided due to the fact that the registration systems had to cope 
with a high load stemming from an unusually high number of registration 
attempts of reputedly available domains.

Starting at 14:30, the switched-off locations were successively provided 
with a complete zone file and re-integrated into the name server network. 
Due to the data volume and the worldwide distribution of the locations, 
the entire distribution process and the subsequent re-start of all the 
service locations concerned took until about 15:45. At this point in time, 
with respect to DENIC, the service had been fully restored with its full 
performance.

However, due to the caching of ISPs, Internet users might sometimes have 
had to put up with disturbances up to 2 hours after the resolution by 
DENIC.

After completion of the ongoing detailed analyses, additional actions 
might have to be considered.

Technical Details
The zone file generated from the registration database is checked for 
completeness and plausibility several times before it is released for use 
by the globally distributed locations. These checks were also successfully 
executed for the named zone file. This is the reason that four locations 
were not provided with a faulty zone file and that the Frankfurt IPv6 DNS 
location and the DNSSEC testbed were not affected. 

However, within the scope of a project for creating a new name service 
infrastructure, the concept for the zone file distribution was also 
redefined. All the quality checks mentioned above were positively applied 
to the named zone file as well, but, according to the new concept the zone 
was copied once again, before it was distributed to the locations. This 
copying process was interrupted resulting in a file that held just one 
third of the domain data.
Provisions were even in place to insure the copying was performed 
correctly. Unfortunately, since the securing mechanism itself wasn’t 
working properly the copying error was not detected and processing was not 
stopped.

As such, the incident is not related to the switch of the data centre 
operation from Amsterdam to Frankfurt, which took place on the preceding 
Tuesday. Neither is there any correlation to the DNSSEC testbed, nor were 
services provided for cooperating partners or secondaries operated by 
DENIC for other TLDs affected. The incident would not have had any impact 
on the anycast services we are planning to provide for our TLD customers 
either. 

Kind regards

Beate Schulz
Business Services - Public Relations



-- 

DENIC eG
Kaiserstraße 75-77
60329 Frankfurt am Main
GERMANY

E-Mail: presse AT denic.de
Fon: +49 69 27235-274 
Fax: +49 69 27235-235
http://www.denic.de

Angaben nach § 25a Absatz 1 GenG:
DENIC Domain Verwaltungs- und Betriebsgesellschaft eG (Sitz: Frankfurt am 
Main)
Vorstand: Sabine Dolderer, Helga Krüger, Carsten Schiefner, Dr. Jörg 
Schweiger 
Vorsitzender des Aufsichtsrats: Elmar Knipp
Eingetragen unter Nr. 770 im Genossenschaftsregister, Amtsgericht 
Frankfurt am Main

Attachment: FAQ_Nameserver_disturbance_P.pdf
Description: Adobe PDF document