18 Čen
2014
18 Čen
'14
12:22
I see, thanks, we'll try 1.5.0.
BTW, does it have the same bug with NAPTR ddns updates? I can't figure out as the code
changed since 1.4.6.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Wednesday, June 18, 2014 1:05 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
1.5.0-rc1 does not have this limitation, updates are stored in a queue there.
As for this code, we could change it to pthread_mutex_lock(), but that way updates could
block all the answering threads. This is somewhat okay for hidden master scenario, but not
okay in general. We could use queue in the 1.4.x as well, but since it's already in
1.5.x, would you be wiling to give that a try?
Oh, and just to clarify one thing, even with 1.5, DDNS will be *really,
really* slow with big zones, nowhere near BIND's performance (think 1 qps of actual
processed updates with 100M zone :( - queuing itself will be much faster, but I suppose
it's the actual processing (i.e. when the changes are visible) that's important to
you). 5k qps were only possible when we turned RCU and some other stuff off, but sadly, a
proper solution will come no sooner than in 1.6. If your BIND setup is no longer suitable,
we could try to hack some code that would speed up the updates using our current code, but
I'd rather focus on the proper solution that does not have similar design problems to
our current solution.
Regards,
Jan
On Wed, 2014-06-18 at 09:33 +0000, Alex Massover wrote:
Hi Jan,
What does this code in update.c do? We're experiencing this error.
if (pthread_mutex_trylock(&zone->ddns_lock) != 0) {
qdata->rcode = KNOT_RCODE_SERVFAIL;
log_zone_error("Failed to process UPDATE for "
"zone %s: Another UPDATE in progress.\n",
zone->conf->name);
return NS_PROC_FAIL;
}
Is it a global lock on the whole zone, meaning that only one concurrent update can run
and the rest are rejected?
So should we force KNOT to a single thread?
What about using lock instead of trylock and increase a number of threads?
Thanks, Alex.
-----Original Message-----
From: knot-dns-users-bounces(a)lists.nic.cz
[mailto:knot-dns-users-bounces@lists.nic.cz] On Behalf Of Alex
Massover
Sent: 05 June 2014 16:38
To: Jan Kadlec
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Thanks Jan,
5k of dynamic updates does the trick for us, at least for one year.
We'll do stress tests and will share the results to the forum as well.
Looking forward for the 1.6.0 release.
BR, Alex.
-----Original Message-----
From: Jan Kadlec [mailto:jan.kadlec@nic.cz]
Sent: Thursday, June 05, 2014 4:28 PM
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hello Alex,
unfortunately, in its current state, Knot DNS is not very fast when it comes to DDNS,
and it's partly because of RCU, since for every update, we have to do a partial copy
of the zone. We'll be doing a lot of improvements to Knot's internal structures
during summer, and we'll keep DDNS performance in mind, so hopefully things will get
better soon.
As of now, we do not plan to parallelize DDNS processing. We've done some
measurements, and we're confident we can easily achieve performance of ~5k QPS in
single thread only. We'll let you know once we have some working code and we can
discuss it further then. I'm sorry I don't have anything better for you, but
hopefully that will change with the 1.6.0 release.
Have a nice day,
Jan
On Wed, 2014-05-28 at 09:29 +0000, Alex Massover wrote:
Hi Ondřej,
We have a master server which receives only updates and a farm of slave servers (ifxr),
which handle reads.
The issue is scaling master server, 1 zone, which is 80M (text), currently we cannot get
more than 200 QPS updates from BIND on 4-way SMP Linux machine.
Each update request is usually 1-2 records. And usually we don't have parallel
updates which are working on the same record.
Our target is 10x from current - i.e. ~1G single zone, which does ~2-4k QPS of updates.
Actually as much as possible in terms of QPS, CPUs & memory is not an issue - we can
get stronger machine.
Thanks, Alex.
-----Original Message-----
From: Ondřej Surý [mailto:ondrej.sury@nic.cz]
Sent: 28 May 2014 11:23
To: Alex Massover
Cc: knot-dns-users(a)lists.nic.cz
Subject: Re: [knot-dns-users] Parallel dynamic DNS updates
Hi Alex,
could you describe your scenario in more detail?
How many zones are we talking about?
How big the zones are?
How many updates per second per zone?
O.
--
Ondřej Surý -- Chief Science Officer
-------------------------------------------
CZ.NIC, z.s.p.o. -- Laboratoře CZ.NIC
Americka 23, 120 00 Praha 2, Czech Republic
mailto:ondrej.sury@nic.cz http://nic.cz/
-------------------------------------------
----- Original Message -----
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
From: "Alex Massover"
<alex(a)jajah.com>
To: knot-dns-users(a)lists.nic.cz
Sent: Wednesday, May 28, 2014 10:01:26 AM
Subject: [knot-dns-users] Parallel dynamic DNS updates
Hello,
I'm trying to understand if Knot can solve
our performance problem
with big scale dynamic DNS deployment. Currently we use BIND and
the performance of Dynamic Updates is insufficient for our requirements.
Doing some research I understood that BIND processes updates
sequentially and can't really benefit from SMP.
I understand how RCU allows parallel reads with
updates. But reads
is not an issue in our deployment. My question is, does Knot able
to process multiple updates in parallel (and not sequentially) and
provide a better scale for _ updates _?
As far as I understand it depends on data
structure hierarchy
implemented in Knot.
--
Alex Massover | Telefónica Digital Architect M +972-54-2279512
alex(a)jajah.com
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users
_______________________________________________
knot-dns-users mailing list
knot-dns-users(a)lists.nic.cz
https://lists.nic.cz/cgi-bin/mailman/listinfo/knot-dns-users