libmdbx  0.12.4.1 (2023-03-17T10:54:40+03:00)
One of the fastest compact embeddable key-value ACID database without WAL.
Overall

Brief

libmdbx is an extremely fast, compact, powerful, embedded, transactional key-value database, with permissive license. libmdbx has a specific set of properties and capabilities, focused on creating unique lightweight solutions.

  1. Allows a swarm of multi-threaded processes to ACIDly read and update several key-value maps and multimaps in a locally-shared database.
  2. Provides extraordinary performance, minimal overhead through Memory-Mapping and Olog(N) operations costs by virtue of B+ tree.
  3. Requires no maintenance and no crash recovery since it doesn't use WAL, but that might be a caveat for write-intensive workloads with durability requirements.
  4. Compact and friendly for fully embedding. Only ≈25KLOC of C11, ≈64K x86 binary code of core, no internal threads neither server process(es), but implements a simplified variant of the Berkeley DB and dbm API.
  5. Enforces serializability for writers just by single mutex and affords wait-free for parallel readers without atomic/interlocked operations, while writing and reading transactions do not block each other.
  6. Guarantee data integrity after crash unless this was explicitly neglected in favour of write performance.
  7. Supports Linux, Windows, MacOS, Android, iOS, FreeBSD, DragonFly, Solaris, OpenSolaris, OpenIndiana, NetBSD, OpenBSD and other systems compliant with POSIX.1-2008.

We have migrated to a reliable trusted infrastructure

The origin for now is at GitFlic since on 2022-04-15 the Github administration, without any warning nor explanation, deleted libmdbx along with a lot of other projects, simultaneously blocking access for many developers. For the same reason Github is blacklisted forever.

Questions, feedback and suggestions are welcome to the Telegram' group.

The Future will (be) Positive. Всё будет хорошо.

Table of Contents

This manual is divided into parts, each of which is divided into several sections.

  1. The Introduction
  2. Usage
  3. The C/C++ API manual:

Please do not hesitate to point out errors in the documentation, including creating merge-request with corrections and improvements.

MithrilDB and Future

The next version is under non-public development from scratch and will be released as MithrilDB and libmithrildb for libraries & packages. Admittedly mythical Mithril is resembling silver but being stronger and lighter than steel. Therefore MithrilDB is a rightly relevant name.

MithrilDB is radically different from libmdbx by the new database format and API based on C++20. The goal of this revolution is to provide a clearer and robust API, add more features and new valuable properties of the database. All fundamental architectural problems of libmdbx/LMDB have been solved there, but now the active development has been suspended for top-three reasons:

  1. For now libmdbx «mostly» enough for all our products, and I’m busy in development of replication for scalability.
  2. Waiting for fresh Elbrus CPU of e2k architecture, especially with hardware acceleration of Streebog and Kuznyechik, which are required for Merkle tree, etc.
  3. The expectation of needs and opportunities due to the wide use of NVDIMM (aka persistent memory), modern NVMe and Ангара.

However, MithrilDB will not be available for countries unfriendly to Russia (i.e. acceded the sanctions, devil adepts and/or NATO). But it is not yet known whether such restriction will be implemented only through a license and support, either the source code will not be open at all. Basically we are not inclined to allow our work to contribute to the profit that goes to weapons that kill our relatives and friends. NO OPTIONS.

Nonetheless, I try not to make any promises regarding MithrilDB until release.

Contrary to MithrilDB, libmdbx will forever free and open source. Moreover with high-quality support whenever possible. Tu deviens responsable pour toujours de ce que tu as apprivois. So we will continue to comply with the original open license and the principles of constructive cooperation, in spite of outright Github sabotage and sanctions. I will also try to keep (not drop) Windows support, despite it is an unused obsolete technology for us.

History

Historically, libmdbx is a deeply revised and extended descendant of the Lightning Memory-Mapped Database. At first the development was carried out within the ReOpenLDAP project. About a year later libmdbx was separated into a standalone project, which was presented at Highload++ 2015 conference.

Since 2017 libmdbx is used in Fast Positive Tables, and development is funded by Positive Technologies.

On 2022-04-15 the Github administration, without any warning nor explanation, deleted libmdbx along with a lot of other projects, simultaneously blocking access for many developers. Therefore on 2022-04-21 we have migrated to a reliable trusted infrastructure. The origin for now is at GitFlic with backup at ABF by ROSA Лаб. For the same reason Github is blacklisted forever.

Acknowledgments

Howard Chu hyc@o.nosp@m.penl.nosp@m.dap.o.nosp@m.rg is the author of LMDB, from which originated the libmdbx in 2015.

Martin Hedenfalk marti.nosp@m.n@bz.nosp@m.ero.s.nosp@m.e is the author of btree.c code, which was used to begin development of LMDB.

Contributors

Author loc coms fils distribution
leo@y.nosp@m.urie.nosp@m.v.ru 74390 3617 94 98.8/94.9/58.4
vroma.nosp@m.nov@.nosp@m.gmail.nosp@m..com 503 5 7 0.7/ 0.1/ 4.3
jsore.nosp@m.f@us.nosp@m.ers.n.nosp@m.orep.nosp@m.ly.gi.nosp@m.thub.nosp@m..com 122 2 23 0.2/ 0.1/14.3
crypt.nosp@m.o.so.nosp@m.wle@g.nosp@m.mail.nosp@m..com 69 1 1 0.1/ 0.0/ 0.6
andre.nosp@m.a.la.nosp@m.nfran.nosp@m.chi@.nosp@m.gmail.nosp@m..com 48 3 3 0.1/ 0.1/ 1.9
dream.nosp@m.sxin.nosp@m.@126..nosp@m.com 47 1 1 0.1/ 0.0/ 0.6
hyc@s.nosp@m.ymas.nosp@m..com 30 72 2 0.0/ 1.9/ 1.2
rouzi.nosp@m.er@g.nosp@m.mail..nosp@m.com 26 5 4 0.0/ 0.1/ 2.5
hallv.nosp@m.ard@.nosp@m.openl.nosp@m.dap..nosp@m.org 22 71 3 0.0/ 1.9/ 1.9
demfl.nosp@m.oro@.nosp@m.demfl.nosp@m.oro..nosp@m.ru 12 1 1 0.0/ 0.0/ 0.6
jens@.nosp@m.moos.nosp@m.eyard.nosp@m..com 10 3 4 0.0/ 0.1/ 2.5
notsp.nosp@m.am@b.nosp@m.uraka.nosp@m.rsla.nosp@m.n.com 8 1 2 0.0/ 0.0/ 1.2
sasga.nosp@m.s@ca.nosp@m.stis..nosp@m.com 5 2 3 0.0/ 0.1/ 1.9
i@rmw.nosp@m..lin.nosp@m.k 5 2 2 0.0/ 0.1/ 1.2
AskAl.nosp@m.exSh.nosp@m.arov@.nosp@m.gmai.nosp@m.l.com 4 1 1 0.0/ 0.0/ 0.6
andre.nosp@m.y.as.nosp@m.hikhm.nosp@m.in@g.nosp@m.mail..nosp@m.com 4 4 2 0.0/ 0.1/ 1.2
janbi.nosp@m.eder.nosp@m.mann@.nosp@m.gitf.nosp@m.lic.r.nosp@m.u 2 1 1 0.0/ 0.0/ 0.6
heire.nosp@m.cka@.nosp@m.exher.nosp@m.bo.o.nosp@m.rg 1 1 1 0.0/ 0.0/ 0.6
vcer@.nosp@m.qq.c.nosp@m.om 1 1 1 0.0/ 0.0/ 0.6
wowem.nosp@m.uh@g.nosp@m.mail..nosp@m.com 1 1 1 0.0/ 0.0/ 0.6
13943.nosp@m.32+b.nosp@m.yronh.nosp@m.e@us.nosp@m.ers.n.nosp@m.orep.nosp@m.ly.gi.nosp@m.thub.nosp@m..com 1 1 1 0.0/ 0.0/ 0.6
crypt.nosp@m.o.zo.nosp@m.idber.nosp@m.g@gm.nosp@m.ail.c.nosp@m.om 1 1 1 0.0/ 0.0/ 0.6
ghaza.nosp@m.n@mi.nosp@m.randa.nosp@m..im 1 1 1 0.0/ 0.0/ 0.6
Erk-@.nosp@m.user.nosp@m.s.nor.nosp@m.eply.nosp@m..gith.nosp@m.ub.c.nosp@m.om 1 1 1 0.0/ 0.0/ 0.6
lmb@c.nosp@m.loud.nosp@m.flare.nosp@m..com 0 2 0 0.0/ 0.1/ 0.0
lyeag.nosp@m.er@n.nosp@m.vidia.nosp@m..com 0 1 0 0.0/ 0.0/ 0.0
orive.nosp@m.j@gm.nosp@m.x.fr 0 2 0 0.0/ 0.1/ 0.0
phili.nosp@m.pp.s.nosp@m.torz@.nosp@m.bare.nosp@m.os.co.nosp@m.m 0 1 0 0.0/ 0.0/ 0.0
quana.nosp@m.h@op.nosp@m.enlda.nosp@m.p.or.nosp@m.g 0 1 0 0.0/ 0.0/ 0.0
you@e.nosp@m.xamp.nosp@m.le.co.nosp@m.m 0 1 0 0.0/ 0.0/ 0.0
moner.nosp@m.omoo.nosp@m.o-mon.nosp@m.ero@.nosp@m.users.nosp@m..nor.nosp@m.eply..nosp@m.gith.nosp@m.ub.co.nosp@m.m 0 3 0 0.0/ 0.1/ 0.0

License

The OpenLDAP Public License Version 2.8, 17 August 2003

Redistribution and use of this software and associated documentation ("Software"), with or without modification, are permitted provided that the following conditions are met:

  1. Redistributions in source form must retain copyright statements and notices,
  2. Redistributions in binary form must reproduce applicable copyright statements and notices, this list of conditions, and the following disclaimer in the documentation and/or other materials provided with the distribution, and
  3. Redistributions must contain a verbatim copy of this document.

The OpenLDAP Foundation may revise this license from time to time. Each revision is distinguished by a version number. You may use this Software under terms of this license revision or under the terms of any subsequent revision of the license.

THIS SOFTWARE IS PROVIDED BY THE OPENLDAP FOUNDATION AND ITS CONTRIBUTORS `‘AS IS’' AND ANY EXPRESSED OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE OPENLDAP FOUNDATION, ITS CONTRIBUTORS, OR THE AUTHOR(S) OR OWNER(S) OF THE SOFTWARE BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

The names of the authors and copyright holders must not be used in advertising or otherwise to promote the sale, use or other dealing in this Software without specific, written prior permission. Title to copyright in this Software shall at all times remain with copyright holders.

OpenLDAP is a registered trademark of the OpenLDAP Foundation.

Copyright 1999-2003 The OpenLDAP Foundation, Redwood City, California, USA. All Rights Reserved. Permission to copy and distribute verbatim copies of this document is granted.