Language Selection

English French German Italian Portuguese Spanish

Announcing etcd 3.4

Filed under
Server
OSS

etcd v3.4 includes a number of performance improvements for large scale Kubernetes workloads.

In particular, etcd experienced performance issues with a large number of concurrent read transactions even when there is no write (e.g. “read-only range request ... took too long to execute”). Previously, the storage backend commit operation on pending writes blocks incoming read transactions, even when there was no pending write. Now, the commit does not block reads which improve long-running read transaction performance.

We further made backend read transactions fully concurrent. Previously, ongoing long-running read transactions block writes and upcoming reads. With this change, write throughput is increased by 70% and P99 write latency is reduced by 90% in the presence of long-running reads. We also ran Kubernetes 5000-node scalability test on GCE with this change and observed similar improvements. For example, in the very beginning of the test where there are a lot of long-running “LIST pods”, the P99 latency of “POST clusterrolebindings” is reduced by 97.4%. This non-blocking read transaction is now used for compaction, which, combined with the reduced compaction batch size, reduces the P99 server request latency during compaction.

More improvements have been made to lease storage. We enhanced lease expire/revoke performance by storing lease objects more efficiently, and made lease look-up operation non-blocking with current lease grant/revoke operation. And etcd v3.4 introduces lease checkpoint as an experimental feature to persist remaining time-to-live values through consensus. This ensures short-lived lease objects are not auto-renewed after leadership election. This also prevents lease object pile-up when the time-to-live value is relatively large (e.g. 1-hour TTL never expired in Kubernetes use case).

Read more

More in Tux Machines

Type Title Author Replies Last Postsort icon
Story Events: openSUSE Asia Summit, EmacsConf and LaKademy Roy Schestowitz 18/10/2019 - 1:42am
Story Ubuntu 19.10 “Eoan Ermine” Released. Here's What's New arindam1989 8 18/10/2019 - 1:37am
Story Snaptastic – elementary OS snap manager Roy Schestowitz 18/10/2019 - 1:19am
Story Ubuntu and GNU/Linux Audiocasts/Shows Roy Schestowitz 18/10/2019 - 1:06am
Story today's howtos Roy Schestowitz 18/10/2019 - 1:02am
Story Games: Tannenberg, Project Zomboid and Jackbox Party Pack 6 Roy Schestowitz 18/10/2019 - 12:51am
Story GhostBSD Reaffirms To Being TrueOS+BSD Desktop OS With Official MATE Desktop Roy Schestowitz 18/10/2019 - 12:42am
Story OpenBSD 6.6 Released Roy Schestowitz 18/10/2019 - 12:37am
Story antiX-19 isos available. Roy Schestowitz 18/10/2019 - 12:34am
Story Android Leftovers Rianne Schestowitz 17/10/2019 - 11:02pm