Duck DNS

faster UDP under load, and cleaner addresses

We’ve been working on the bit that matters most day to day: answering DNS over UDP quickly when traffic spikes. The DNS fleet is doing about 120 million requests an hour — roughly 33,000 per second, or on the order of 86 billion a month.

The DNS server used to spin a classic thread per packet. Under bursty miss traffic that burned OS threads and piled pressure on DynamoDB. We’ve moved query handling to Java 21 virtual threads, with a dedicated platform thread on receive and light VT workers for replies. A concurrency cap (duckdns.udp.maxConcurrent) limits how many lookups are in flight so a miss storm doesn’t starve the Dynamo pool — overload drops excess queries and lets the kernel’s UDP buffer apply backpressure, instead of melting the JVM.

We also tuned the Dynamo client (timeouts, connection pool sized to match concurrency) and stopped negative-caching on lookup timeouts, so a slow request doesn’t poison the cache and slow the next good answer.

On the network side we simplified the DNS security groups so UDP is untracked in AWS. Connection tracking used to put a hard ceiling on how many DNS flows those ENIs could handle. By allowing UDP end-to-end (matching ingress and egress), AWS skips state for those packets — the box still only listens on 53; everything else is dropped by the OS — so replies don’t depend on conntrack and the table can’t fill under load. Shorter timeouts for anything that still is tracked cover the leftover cases.

On the write path, we’ve tightened IP validation so addresses with leading-zero octets (185.169.107.000 and friends) never get stored. Those used to blow up in dnsjava as UnknownHostException: Invalid address on resolve. There’s a cluster-wide journal scan now so we can count and track those errors across all DNS hosts while rolling the filter.

Same free DuckDNS — UDP path ready for more traffic, no AWS state ceiling in the way, and fewer bad records in the data.