5 Dub
2024
5 Dub
'24
14:31
Hi, Ulrich,
I am not very familiar with setups where the whole root is nfs-mounted,
but I have a suspicion it could be the cause if the system also treats
/var/cache as nfs-mounted (this can be checked using the 'mount' command
without arguments). This could cause problems, as the cache is mapped to
Resolver's virtual memory and the Resolver writes into it frequently,
which I suspect could fill up some kind of an nfs journal or something
like that, causing the "No space left on device" error.
What I would recommend is to get the cache (and ideally the workdir as
well) into a tmpfs-mounted space (this is generally /tmp, /dev/shm, or
anything you specify using /etc/fstab) so that it is locally in the
RAM/swap. To move the cache, use the cache.storage property in the
config [1].
[1]:
https://knot-resolver.readthedocs.io/en/stable/daemon-bindings-cache.html#e…
Best regards,
Oto Šťáva | Knot Resolver team leader | CZ.NIC z.s.p.o.
On 4/5/24 1:22 PM, Ulrich Wisser via knot-resolver-users wrote:
Hi,
for some time now I have a problem running kresd on my raspberry pi.
I am running pihole and use kresd as resolver behind pihole.
Everything works fine until some day where kresd "decides" to crash.
It is always the same error message (please see below).
I then have to manually stop the garbage collector, remove all cache
files and restart kresd (which automatically starts the garbage
collector).
My pi is pxe boot and the root partition is on an nfs mounted volume.
The volume has several terra byte of space left.
After a restart it will run flawless for tow or three weeks just to
crash with the same error again.
Any idea how this happens?
/Ulrich
root@pi-hole1:~# systemctl status kresd
● kresd.service - Knot Resolver daemon
Loaded: loaded (/lib/systemd/system/kresd.service; enabled;
vendor preset: enabled)
Active: failed (Result: exit-code) since Fri 2024-04-05 02:31:47
CEST; 6h ago
Docs: man:kresd.systemd(7)
man:kresd(8)
Process: 5106 ExecStart=/usr/sbin/kresd -c
/usr/lib/arm-linux-gnueabihf/knot-resolver/distro-preconfig.lua -c
/etc/knot-resolver/kresd.conf -n (code=exited, status=1/FAILURE)
Process: 5110 ExecStopPost=/usr/bin/env rm -f
/run/knot-resolver/control/ (code=exited, status=1/FAILURE)
Main PID: 5106 (code=exited, status=1/FAILURE)
CPU: 648ms
Apr 05 02:31:45 pi-hole1 kresd[5106]: [C]: at 0x0001b2d8
Apr 05 02:31:45 pi-hole1 kresd[5106]: [C]: in function 'pcall'
Apr 05 02:31:45 pi-hole1 kresd[5106]:
...b/arm-linux-gnueabihf/knot-resolver/distro-preconfig.lua:9: in main
chunk
Apr 05 02:31:45 pi-hole1 kresd[5106]: ERROR: net.listen() failed to bind
Apr 05 02:31:46 pi-hole1 kresd[5106]: [system] error while loading
config: /usr/lib/arm-linux-gnueabihf/knot-resolver/sandbox.lua:402:
can't open cache path '/var/cache/knot-resolver'; working directory
'/var/lib/knot-resolver'; No space left on device (workdir
'/var/lib/knot-resolver')
Apr 05 02:31:47 pi-hole1 systemd[1]: kresd.service: Main process
exited, code=exited, status=1/FAILURE
Apr 05 02:31:47 pi-hole1 env[5110]: rm: cannot remove
'/run/knot-resolver/control/': Is a directory
Apr 05 02:31:47 pi-hole1 systemd[1]: kresd.service: Control process
exited, code=exited, status=1/FAILURE
Apr 05 02:31:47 pi-hole1 systemd[1]: kresd.service: Failed with result
'exit-code'.
Apr 05 02:31:47 pi-hole1 systemd[1]: Failed to start Knot Resolver
daemon.
root@pi-hole1:~# ps aux Op | grep kres
knot-re+ 569 0.1 0.1 114480 4140 ? Ss Apr02 5:15
/usr/sbin/kres-cache-gc -c /var/cache/knot-resolver -d 1000
root 23835 0.0 0.0 10292 504 pts/0 S+ 09:00 0:00 grep
kres
root@pi-hole1:~# systemctl status | grep kres
│ └─23861 grep kres
├─system-kresd.slice
│ └─kres-cache-gc.service
│ └─569 /usr/sbin/kres-cache-gc -c
/var/cache/knot-resolver -d 1000
root@pi-hole1:~# systemctl stop kres-cache-gc
--