19 Pro
2025
19 Pro
'25
18:01
Hello,
it seems vanilla kresd (5.7.6-cznic.1~bookworm), when receives NODATA response (NOERROR,
all RRs: 0) from remote authoritative server, stores record in cache with TTL=32768.
Where is this value coming from/how can we alter it?
(Seems hardcoded, not derived from SOA expiry or TTL.)
Impact:
may get negative ~9h cache for domains delegated to a server randomly returning NODATA
Workaround:
sacrifice cache hit ratio with cache.max_ttl(low_value)
Reproduction:
// wait 5-30 minutes
while true; do echo 'cache.clear("mr-z01.tm-azurefd.net")' | socat -
UNIX-CONNECT:/run/knot-resolver/control/1 >/dev/null; sleep 1; host -vvvvt a
mr-z01.tm-azurefd.net 127.0.0.1; curl -s
http://localhost:8451/trace/mr-z01.tm-azurefd.net; echo DONE; done 2>&1 | tee -a
issue.log
Watch log:
// captured with cache.min_ttl(77), irrelevant here
[cache ][65604.03] => satisfied by exact CNAME: rank 030, new TTL 16
[cache ][65604.05] => satisfied by exact RRset: rank 030, new TTL 16
[cache ][65605.01] => satisfied by exact packet: rank 030, new TTL 32768 <------
auth server return NODATA response (reason unknown)
[cache ][65606.01] => satisfied by exact packet: rank 030, new TTL 32768
[cache ][65607.01] => satisfied by exact RRset: rank 030, new TTL 77
[cache ][65608.01] => satisfied by exact RRset: rank 030, new TTL 77
[cache ][65609.01] => satisfied by exact CNAME: rank 030, new TTL 77
[cache ][65609.03] => satisfied by exact CNAME: rank 030, new TTL 11
[cache ][65609.05] => satisfied by exact RRset: rank 030, new TTL 11
[cache ][65610.01] => satisfied by exact CNAME: rank 030, new TTL 77
[cache ][65610.03] => satisfied by exact CNAME: rank 030, new TTL 10
[cache ][65610.05] => satisfied by exact RRset: rank 030, new TTL 10
[cache ][65611.01] => satisfied by exact CNAME: rank 030, new TTL 77
[cache ][65611.03] => satisfied by exact CNAME: rank 030, new TTL 9
[cache ][65611.05] => satisfied by exact RRset: rank 030, new TTL 9
[cache ][65612.01] => satisfied by exact packet: rank 030, new TTL 32768
[cache ][65613.01] => satisfied by exact RRset: rank 030, new TTL 77
[cache ][65614.01] => satisfied by exact RRset: rank 030, new TTL 77
// host
Trying "mr-z01.tm-azurefd.net"
Using domain server:
Name: 127.0.0.1
Address: 127.0.0.1#53
Aliases:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62752
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;mr-z01.tm-azurefd.net. IN A
Received 39 bytes from 127.0.0.1#53 in 4 ms
// kresd trace
[iterat][65612.00] 'mr-z01.tm-azurefd.net.' type 'A' new uid was
assigned .01, parent uid .00
[cache ][65612.01] => satisfied by exact packet: rank 030, new TTL 32768
[iterat][65612.01] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 43459
;; Flags: qr aa QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: do; UDP size: 1232 B; ext-rcode: Unused
;; QUESTION SECTION
mr-z01.tm-azurefd.net. A
;; ADDITIONAL SECTION
[iterat][65612.01] <= rcode: NOERROR
[resolv][65612.01] AD: request NOT classified as SECURE
[resolv][65612.01] finished in state: 4, queries: 1, mempool: 81952 B
// cache miss trace
[iterat][65637.03] <= rcode: NOERROR
[cache ][65637.03] => stashed tm2.dns-tm.com. A, rank 030, 20 B total, incl. 0
RRSIGs
[iterat][65637.01] 'mr-z01.tm-azurefd.net.' type 'A' new uid was
assigned .04, parent uid .00
[select][65637.04] => id: '43668' choosing from addresses: 1 v4 + 0 v6; names
to resolve: 2 v4 + 3 v6; force_resolve: 0; NO6: IPv6 is KO
[select][65637.04] => id: '43668' choosing:
'tm2.dns-tm.com.'(a)'150.171.16.240#00053' with timeout 22 ms zone cut:
'tm-azurefd.net.'
[resolv][65637.04] => id: '43668' querying:
'tm2.dns-tm.com.'(a)'150.171.16.240#00053' zone cut:
'tm-azurefd.net.' qname: 'mr-z01.tm-azurefd.net.' qtype: 'A'
proto: 'udp'
[select][65637.04] => id: '43668' updating:
'tm2.dns-tm.com.'(a)'150.171.16.240#00053' zone cut:
'tm-azurefd.net.' with rtt 4 to srtt: 2 and variance: 1
[iterat][65637.04] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 43668
;; Flags: qr aa QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: do; UDP size: 1232 B; ext-rcode: Unused
;; QUESTION SECTION
mr-z01.tm-azurefd.net. A
;; ADDITIONAL SECTION
[iterat][65637.04] <= rcode: NOERROR
[cache ][65637.04] => stashed packet: rank 030, TTL 32768, A mr-z01.tm-azurefd.net.
(62 B)
[resolv][65637.04] AD: request NOT classified as SECURE
[resolv][65637.04] finished in state: 4, queries: 2, mempool: 81952 B
;; selected from ANSWER sections:
; ranked rrset to_wire false, rank 030 (auth insecure), cached true, qry_uid 3,
revalidations 0
tm2.dns-tm.com. 300 A 150.171.16.240
19 Pro
19 Pro
18:10
On 19/12/2025 18.01, pawmal-knot--- via knot-resolver-users wrote:
it seems vanilla kresd (5.7.6-cznic.1~bookworm), when
receives NODATA response (NOERROR, all RRs: 0) from remote authoritative server, stores
record in cache with TTL=32768.
Where is this value coming from/how can we alter it?
(Seems hardcoded, not derived from SOA expiry or TTL.)
In that instance the authoritative server sends no SOA nor TTL. However,
the particular number seems to be a bug. I'd probably go for the
minimal TTL setting for that case. I'll look at that later.
--Vladimir
20:11
On 19/12/2025 20.09, Paweł Małachowski via knot-resolver-users wrote:
may be EDNS OPT DNSSEC casting as TTL (claude
suggestion)
I don't get the intention. The auth server is obliged to send a SOA
with TTL controlling this in that case. The thing is... what should
Knot Resolver do exactly when the server breaks this obligation.
20:29
There is no data in response.
packet_ttl() checks for !is_negative?
Domain Name System (response)
Transaction ID: 0x9eaf
Flags: 0x8500 Standard query response, No error
1... .... .... .... = Response: Message is a response
.000 0... .... .... = Opcode: Standard query (0)
.... .1.. .... .... = Authoritative: Server is an authority for domain
.... ..0. .... .... = Truncated: Message is not truncated
.... ...1 .... .... = Recursion desired: Do query recursively
.... .... 0... .... = Recursion available: Server can't do recursive queries
.... .... .0.. .... = Z: reserved (0)
.... .... ..0. .... = Answer authenticated: Answer/authority portion was not
authenticated by the server
.... .... ...0 .... = Non-authenticated data: Unacceptable
.... .... .... 0000 = Reply code: No error (0)
Questions: 1
Answer RRs: 0
Authority RRs: 0
Additional RRs: 0
Queries
mr-z01.tm-azurefd.net: type A, class IN
Name: mr-z01.tm-azurefd.net
[Name Length: 21]
[Label Count: 3]
Type: A (1) (Host Address)
Class: IN (0x0001)
[Request In: 661]
[Time: 1.739000 milliseconds]
21 Pro
21 Pro
12:00
On 19/12/2025 18.01, pawmal-knot--- via knot-resolver-users wrote:
Where is this value coming from/how can we alter it?
So far I've been unable to reproduce this. I always get the minimum TTL
in this case, which is 5s by default. I even wrote a simple test for
it. The most relevant code also looks good to me (and unchanged between
5.7.6 and 6.x/master).
I wonder if those Azure servers would rarely send something else at that
malfunctioning period in time. But the cause could be something
completely else; I just don't know so far.
--Vladimir
Log from the test:
[reqdbg][policy][26518.00] request packet:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 26518
;; Flags: rd QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: Unused
;; QUESTION SECTION
example. A
;; ADDITIONAL SECTION
[reqdbg][iterat][26518.00] 'example.' type 'A' new uid was assigned
.01, parent uid .00
[reqdbg][resolv][26518.01] => going insecure because there's no
covering TA
[reqdbg][resolv][26518.01] => using root hints
[reqdbg][iterat][26518.01] 'example.' type 'A' new uid was assigned
.02, parent uid .00
[reqdbg][select][26518.02] => id: '40992' choosing from addresses: 1
v4 + 0 v6; names to resolve: 0 v4 + 0 v6; force_resolve: 0; NO6: IPv6 is OK
[reqdbg][select][26518.02] => id: '40992' choosing:
'k.root-servers.net.'(a)'193.0.14.129#00053' with timeout 400 ms zone cut:
'.'
[reqdbg][resolv][26518.02] => id: '40992' querying:
'k.root-servers.net.'(a)'193.0.14.129#00053' zone cut: '.' qname:
'example.' qtype: 'A' proto: 'udp'
[reqdbg][select][26518.02] => id: '40992' updating:
'k.root-servers.net.'(a)'193.0.14.129#00053' zone cut: '.' with rtt
102 to
srtt: 102 and variance: 51
[reqdbg][iterat][26518.02] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 40992
;; Flags: qr aa QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: do; UDP size: 1232 B; ext-rcode: Unused
;; QUESTION SECTION
example. A
;; ADDITIONAL SECTION
[reqdbg][iterat][26518.02] <= rcode: NOERROR
[reqdbg][cache ][26518.02] => stashed packet: rank 020, TTL 5, A
example. (48 B)
[reqdbg][resolv][26518.02] AD: request NOT classified as SECURE
[reqdbg][resolv][26518.02] finished in state: 4, queries: 1, mempool:
16400 B
[reqdbg][policy][26518.00] following rrsets were marked as interesting:
[reqdbg][policy][26518.00] answer packet:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 26518
;; Flags: qr rd ra QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: Unused
;; QUESTION SECTION
example. A
;; ADDITIONAL SECTION
[reqdbg][policy][03130.00] request packet:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 3130
;; Flags: rd QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: Unused
;; QUESTION SECTION
example. A
;; ADDITIONAL SECTION
[reqdbg][iterat][03130.00] 'example.' type 'A' new uid was assigned
.01, parent uid .00
[reqdbg][cache ][03130.01] => satisfied by exact packet: rank 020, new
TTL 5
[reqdbg][iterat][03130.01] <= answer received:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 34288
;; Flags: qr aa QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 0
;; QUESTION SECTION
example. A
[reqdbg][iterat][03130.01] <= rcode: NOERROR
[reqdbg][resolv][03130.01] AD: request NOT classified as SECURE
[reqdbg][resolv][03130.01] finished in state: 4, queries: 1, mempool:
16400 B
[reqdbg][policy][03130.00] following rrsets were marked as interesting:
[reqdbg][policy][03130.00] answer packet:
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 3130
;; Flags: qr rd ra QUERY: 1; ANSWER: 0; AUTHORITY: 0; ADDITIONAL: 1
;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 4096 B; ext-rcode: Unused
;; QUESTION SECTION
example. A
;; ADDITIONAL SECTION
14:05
Thanks for pointing me towards this issue. Fix:
https://gitlab.nic.cz/knot/knot-resolver/-/merge_requests/1774
All versions are affected (checked recent 5.x and 6.x). We'll fix this
in all future releases (5.x + 6.x).
--Vladimir
22 Pro
22 Pro
0:37
Hello,
Thank you for providing the fix. After further analysis, it appears that this issue
affects a significant number of important and widely used domains, which may have a real
impact on many users.
Would it be possible to prepare an official release and distribution packages that include
this fix? This would greatly simplify deployment in production environments.
Thank you in advance for your time and consideration.
Best regards,
Łukasz Trąbiński
Wiadomość napisana przez Vladimír Čunát via
knot-resolver-users <knot-resolver-users(a)lists.nic.cz> w dniu 21 gru 2025, o godz.
14:05:
Thanks for pointing me towards this issue. Fix:
https://gitlab.nic.cz/knot/knot-resolver/-/merge_requests/1774
All versions are affected (checked recent 5.x and 6.x). We'll fix this in all future
releases (5.x + 6.x).
--Vladimir
--
8:32
On 22/12/2025 00.37, Łukasz Trąbiński wrote:
After further analysis, it appears that this issue
affects a significant number of important and widely used domains, which may have a real
impact on many users.
Perhaps you'd like to share more about this analysis? From what I know
so far, practical impact needs an error on the side of the authoritative
server. (as servers are obliged to send a SOA and lower TTL wins)
Would it be possible to prepare an official release
and distribution packages that include this fix? This would greatly simplify deployment in
production environments.
Thank you in advance for your time and consideration.
Speaking of in-distribution packages, those are almost never in *our*
hands, unfortunately. Releasing and upstream packages are doable,
though honestly they seem unlikely to happen this year.
If you're impatient, I suggest the option of grabbing binary packages
from our CI. We aim to keep the master and master-5 branches in good
shape all the time. Right now, for example:
* 6.x: https://gitlab.nic.cz/knot/knot-resolver/-/pipelines/148723
* 5.x: https://gitlab.nic.cz/knot/knot-resolver/-/pipelines/148736
You just click your distro on the right, and in the artifacts you can
download binary packages tested by the CI (same recipe as for our
official upstream packages).
--Vladimir
28
days inactive
31
days old
knot-resolver-users@lists.nic.cz
9 comments
4 participants
participants (4)
-
Paweł Małachowski -
pawmal-knot＠freebsd.lublin.pl
-
Vladimír Čunát -
Łukasz Trąbiński