8 Říj
2020
8 Říj
'20
20:04
Hi Daniel
I've made the suggested changes, a slight improvement I think, but load is
still fairly high:
[image: image.png]
Any further suggestions greatly appreciated.
Thanks
Jonathan
On Thu, 8 Oct 2020 at 08:29, Daniel Salzman <daniel.salzman(a)nic.cz> wrote:
Hi Jonathan
Try increasing the number or transmit channels:
ethtool -L eno1 rx 4 tx 4
And reduce the number of udp-workers
Daniel
On 10/8/20 2:12 AM, Jonathan Hewlett wrote:
--
--\--
Jonathan Hewlett
^Senior\ Systems\ Engineer\n$
CentralNic Group plc (LSE:CNIC)
https://www.centralnic.com
Tel: +44.7548243020
CentralNic Group plc is a company registered in England and Wales with
company number 8576358. Registered Offices: Saddlers House, Gutter Lane,
London EC2V 6AE.
Hi Danel,
I'm sorry for the late reply; I've been on holiday/vacation and playing
catch-up! ;)
ethtool:
Channel parameters for eno1:
Pre-set maximums:
RX: 4
TX: 4
Other: 0
Combined: 0
Current hardware settings:
RX: 4
TX: 1
Other: 0
Combined: 0
knotc output:
Knot DNS 2.9.5
Target: linux-gnu x86_64
Compiler: gcc
CFLAGS: -O2 -g -pipe -Wall -Werror=format-security
-Wp,-D_FORTIFY_SOURCE=2
-Wp,-D_GLIBCXX_ASSERTIONS -fexceptions
-fstack-protector-strong -grecord-gcc-switches
-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
-specs=/usr/lib/rpm/redhat/redhat-annobin-cc1 -m64 -mtune=generic
-fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection
-DNDEBUG
-Wno-unused -Wall -Wshadow
-Werror=format-security -Werror=implicit
-Wstrict-prototypes
LIBS: -Wl,-z,relro -Wl,-z,now
-specs=/usr/lib/rpm/redhat/redhat-hardened-ld
LibURCU: -lurcu
GnuTLS: -lgnutls -I/usr/include/p11-kit-1
Libedit: -ledit -I/usr/include/editline
LMDB: shared -llmdb
Config: 500 MiB default mapsize
Prefix: /usr
Run dir: /run/knot
Storage dir: /var/lib/knot
Config dir: /etc/knot
Module dir:
Static modules: cookies dnsproxy dnstap geoip noudp onlinesign
queryacl rrl
stats synthrecordwhoami
Shared modules:
Knot DNS libraries: yes
Knot DNS daemon: yes
Knot DNS utilities: yes
Knot DNS documentation: yes
Use recvmmsg: yes
Use SO_REUSEPORT(_LB): yes
Memory allocator: auto
Fast zone parser: yes
Utilities with IDN: libidn2
Utilities with Dnstap: yes
MaxMind DB support: yes
Systemd integration: yes
POSIX capabilities: yes
PKCS #11 support: yes
Ed25519 support: yes
Ed448 support: no
Code coverage: no
Sanitizer: no
LibFuzzer: no
OSS-Fuzz: no
Continued thanks! ;)
Jonathan
On Thu, 24 Sep 2020 at 11:14, Daniel Salzman <daniel.salzman(a)nic.cz
<mailto:daniel.salzman@nic.cz>> wrote:
Hi Jonathan,
If you have just one 1GbE port, I would recommend reducing the
number of
udp-workers. What about 12 or less?
How do you install Knot DNS (repository or source code)?
What do say these commands?
# sudo ethtool -l eno1
# knotc status configure
Daniel
On 9/23/20 5:45 PM, Jonathan Hewlett wrote:
> Hi Daniel
>
> * Server listens: 967 (multiple v4 and v6 networks)
> * Physical server
> * NIC details
> *-network:0
> description: Ethernet interface
> product: NetXtreme BCM5720 2-port Gigabit Ethernet PCIe
> vendor: Broadcom Inc. and subsidiaries
> physical id: 0
> bus info: pci@0000:02:00.0
> logical name: eno1
> version: 00
> serial: b0:83:fe:c8:f3:ec
> size: 1Gbit/s
> capacity: 1Gbit/s
> width: 64 bits
> clock: 33MHz
> capabilities: pm vpd msi msix pciexpress bus_master
cap_list rom
ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt
1000bt-fd autonegotiation
> configuration: autonegotiation=on
broadcast=yes driver=tg3
driverversion=3.137 duplex=full firmware=FFV21.60.2 bc
5720-v1.39
ip=38.88.50.50 latency=0 link=yes multicast=yes port=twisted pair
> speed=1Gbit/s
> resources: irq:16 memory:d80a0000-d80affff
memory:d80b0000-d80bffff memory:d80c0000-d80cffff memory:dd800000-dd83ffff
>
> Thanks for the continuing support
> Jonathan
>
>
> On Wed, 23 Sep 2020 at 12:47, Daniel Salzman <
daniel.salzman(a)nic.cz
<mailto:daniel.salzman@nic.cz> <mailto:
daniel.salzman(a)nic.cz <mailto:daniel.salzman@nic.cz>>> wrote:
>
> Jonathan,
>
> there must be something wrong with the networking. How many
`server.listen` configuration values
> do yo have? Is it a physical server
or a virtual one? What is
the network card used?
>
> Daneil
>
> On 9/23/20 1:18 PM, Jonathan Hewlett wrote:
> > Daniel
> > Thank you for your time
> >
> > # knotc status workers
> > UDP workers: 24, TCP workers 24, background workers: 10
(running: 0, pending: 0)
> >
> > image.png
> >
> > Kind regards
> > Jonathan
> >
> >
> > On Wed, 23 Sep 2020 at 07:59, <daniel.salzman(a)nic.cz
<mailto:daniel.salzman@nic.cz> <mailto:daniel.salzman@nic.cz <mailto:
daniel.salzman(a)nic.cz>> <mailto:daniel.salzman@nic.cz
<mailto:daniel.salzman@nic.cz>
<mailto:daniel.salzman@nic.cz
<mailto:daniel.salzman@nic.cz>>>>
wrote:
> >
> > Hi Jonathan,
> >
> > On 2020-09-22 23:11, Jonathan Hewlett wrote:
> > > Hi,
> > >
> > > We run around 50 knot servers around the world, with
great success -
> > > thanks for the
software!
> >
> > Thanks for this feedback :-)
> >
> > >
> > > We are just a little curious about seemingly high CPU
usage, and how
> > > best to interpret the
results, if someone could
advise? Specifically,
> > > we are seeing the
majority of the cpu time in 'system'
and not 'user',
> > > what would that
suggest is happening perhaps?
> >
> > Could you provide me the output from `knotc status
workers`?
> >
> > I like htop. Are you able to send me a screenshot when
running htop? :-)
> > [Don't forget to
configure:
> > F2
> > Display options->Detailed CPU time
> > (System/IO-Wait/Hard-IRQ/Soft-IRQ/Steal/Guest)
> > F10
> > ]
> >
> > Best,
> > Daniel
> >
> > >
> > > Any tuning help gratefully received!
> > >
> > > Our platform (sample)
> > >
> > > 24 Core CPU E5-2440 0 @ 2.40
> > > 64Gb RAM
> > > SSD
> > >
> > > knot-2.9.5, running on Centos 8.2
> > >
> > > Currently receiving around 4k pps
> > >
> > > Server is authoritative for a 278 zones, the largest
zone file being
> > > around 600Mb
> > >
> > > Output of top
> > >
> > > top - 20:50:38 up 6 days, 4:49, 2 users, load
average: 13.47,
> > > 12.34, 11.83
> > > Tasks: 321 total, 2 running, 318 sleeping, 0
stopped, 1 zombie
> > > %Cpu0 : 4.0 us, 34.1
sy, 0.0 ni, 60.3 id, 0.3 wa,
0.7 hi, 0.7
> > > si, 0.0 st
> > > %Cpu1 : 4.3 us, 19.4 sy, 0.0 ni, 75.3 id, 0.0 wa,
0.3 hi, 0.7
> > > si, 0.0 st
> > > %Cpu2 : 4.3 us, 33.8 sy, 0.0 ni, 60.9 id, 0.0 wa,
0.7 hi, 0.3
> > > si, 0.0 st
> > > %Cpu3 : 5.0 us, 21.2 sy, 0.0 ni, 73.2 id, 0.0 wa,
0.3 hi, 0.3
> > > si, 0.0 st
> > > %Cpu4 : 3.3 us, 34.9 sy, 0.0 ni, 61.5 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu5 : 3.0 us, 20.2 sy, 0.0 ni, 76.2 id, 0.0 wa,
0.3 hi, 0.3
> > > si, 0.0 st
> > > %Cpu6 : 4.0 us, 35.2 sy, 0.0 ni, 60.4 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu7 : 5.0 us, 18.7 sy, 0.0 ni, 76.0 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu8 : 2.0 us, 35.0 sy, 0.0 ni, 62.3 id, 0.0 wa,
0.7 hi, 0.0
> > > si, 0.0 st
> > > %Cpu9 : 2.0 us, 20.0 sy, 0.0 ni, 77.7 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu10 : 3.7 us, 34.4 sy, 0.0 ni, 61.2 id, 0.0 wa,
0.7 hi, 0.0
> > > si, 0.0 st
> > > %Cpu11 : 3.3 us, 20.7 sy, 0.0 ni, 75.7 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu12 : 2.3 us, 34.6 sy, 0.0 ni, 62.1 id, 0.0 wa,
0.7 hi, 0.3
> > > si, 0.0 st
> > > %Cpu13 : 3.3 us, 20.7 sy, 0.0 ni, 75.7 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu14 : 2.3 us, 33.6 sy, 0.0 ni, 59.7 id, 0.0 wa,
0.7 hi, 3.7
> > > si, 0.0 st
> > > %Cpu15 : 3.3 us, 19.7 sy, 0.0 ni, 76.3 id, 0.0 wa,
0.3 hi, 0.3
> > > si, 0.0 st
> > > %Cpu16 : 2.3 us, 34.0 sy, 0.0 ni, 58.7 id, 0.0 wa,
1.0 hi, 4.0
> > > si, 0.0 st
> > > %Cpu17 : 2.3 us, 21.5 sy, 0.0 ni, 75.8 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > %Cpu18 : 3.3 us, 34.8 sy, 0.0 ni, 57.2 id, 0.0 wa,
0.7 hi, 4.0
> > > si, 0.0 st
> > > %Cpu19 : 5.0 us, 19.3 sy, 0.0 ni, 75.1 id, 0.0 wa,
0.3 hi, 0.3
> > > si, 0.0 st
> > > %Cpu20 : 2.7 us, 35.3 sy, 0.0 ni, 60.0 id, 0.0 wa,
0.7 hi, 1.3
> > > si, 0.0 st
> > > %Cpu21 : 1.7 us, 20.5 sy, 0.0 ni, 77.2 id, 0.0 wa,
0.3 hi, 0.3
> > > si, 0.0 st
> > > %Cpu22 : 4.7 us, 32.4 sy, 0.0 ni, 58.5 id, 0.0 wa,
0.7 hi, 3.7
> > > si, 0.0 st
> > > %Cpu23 : 2.3 us, 19.0 sy, 0.0 ni, 78.3 id, 0.0 wa,
0.3 hi, 0.0
> > > si, 0.0 st
> > > MiB Mem : 64107.4 total, 46764.9 free, 11733.6
used,
5608.8
> > > buff/cache
> > > MiB Swap: 32248.0 total, 32248.0 free, 0.0
used.
51670.0 avail
> > > Mem
> > >
> > > PID USER PR NI VIRT RES SHR S %CPU
%MEM TIME+
> > > COMMAND
> > > 213370 knot 20 0 34.0g 9.6g 41784 S 644.9
15.3 1319:12
> > > /usr/sbin/knotd
> > >
> > > Thanks again for this great product.
> > >
> > > Jonathan
> >
> >
> >
> > --
> > --\--
> > Jonathan Hewlett
> > ^Senior\ Systems\ Engineer\n$
> > CentralNic Group plc (LSE:CNIC)
> > https://www.centralnic.com <https://www.centralnic.com/>
> > Tel: +44.7548243020
> > CentralNic Group plc is a company registered in England and
Wales with company number 8576358. Registered Offices: Saddlers House,
Gutter Lane, London EC2V 6AE.
>
>
>
> --
> --\--
> Jonathan Hewlett
> ^Senior\ Systems\ Engineer\n$
> CentralNic Group plc (LSE:CNIC)
> https://www.centralnic.com <https://www.centralnic.com/>
> Tel: +44.7548243020
> CentralNic Group plc is a company registered in England and Wales
with
company number 8576358. Registered Offices: Saddlers House, Gutter
Lane, London EC2V 6AE.
--
--\--
Jonathan Hewlett
^Senior\ Systems\ Engineer\n$
CentralNic Group plc (LSE:CNIC)
https://www.centralnic.com <https://www.centralnic.com/>
Tel: +44.7548243020
CentralNic Group plc is a company registered in England and Wales with
company
number 8576358. Registered Offices: Saddlers House, Gutter Lane,
London EC2V 6AE.
