libmdbx  0.11.9.0 (2022-08-02T12:00:30+03:00)
One of the fastest compact embeddable key-value ACID database without WAL.
Settings

Typedefs

typedef enum MDBX_option_t MDBX_option_t
 

Functions

LIBMDBX_API int mdbx_env_set_option (MDBX_env *env, const MDBX_option_t option, uint64_t value)
 Sets the value of a runtime options for an environment. More...
 
LIBMDBX_API int mdbx_env_get_option (const MDBX_env *env, const MDBX_option_t option, uint64_t *pvalue)
 Gets the value of runtime options from an environment. More...
 
int mdbx_env_set_syncbytes (MDBX_env *env, size_t threshold)
 Sets threshold to force flush the data buffers to disk, even any of MDBX_SAFE_NOSYNC flag in the environment. More...
 
int mdbx_env_set_syncperiod (MDBX_env *env, unsigned seconds_16dot16)
 Sets relative period since the last unsteady commit to force flush the data buffers to disk, even of MDBX_SAFE_NOSYNC flag in the environment. More...
 
LIBMDBX_API int mdbx_env_set_flags (MDBX_env *env, MDBX_env_flags_t flags, bool onoff)
 Set environment flags. More...
 
LIBMDBX_API int mdbx_env_set_geometry (MDBX_env *env, intptr_t size_lower, intptr_t size_now, intptr_t size_upper, intptr_t growth_step, intptr_t shrink_threshold, intptr_t pagesize)
 Set all size-related parameters of environment, including page size and the min/max size of the memory map. More...
 
MDBX_DEPRECATED int mdbx_env_set_mapsize (MDBX_env *env, size_t size)
 
int mdbx_env_set_maxreaders (MDBX_env *env, unsigned readers)
 Set the maximum number of threads/reader slots for for all processes interacts with the database. More...
 
int mdbx_env_set_maxdbs (MDBX_env *env, MDBX_dbi dbs)
 Set the maximum number of named databases for the environment. More...
 
LIBMDBX_API int mdbx_env_set_userctx (MDBX_env *env, void *ctx)
 Sets application information (a context pointer) associated with the environment. More...
 

Detailed Description

Typedef Documentation

◆ MDBX_option_t

Function Documentation

◆ mdbx_env_get_option()

LIBMDBX_API int mdbx_env_get_option ( const MDBX_env env,
const MDBX_option_t  option,
uint64_t *  pvalue 
)

Gets the value of runtime options from an environment.

Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]optionThe option from MDBX_option_t to get value of it.
[out]pvalueThe address where the option's value will be stored.
See also
MDBX_option_t
mdbx_env_get_option()
Returns
A non-zero error value on failure and 0 on success.

◆ mdbx_env_set_flags()

LIBMDBX_API int mdbx_env_set_flags ( MDBX_env env,
MDBX_env_flags_t  flags,
bool  onoff 
)

Set environment flags.

This may be used to set some flags in addition to those from mdbx_env_open(), or to unset these flags.

See also
mdbx_env_get_flags()
Note
In contrast to LMDB, the MDBX serialize threads via mutex while changing the flags. Therefore this function will be blocked while a write transaction running by other thread, or MDBX_BUSY will be returned if function called within a write transaction.
Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]flagsThe env_flags to change, bitwise OR'ed together.
[in]onoffA non-zero value sets the flags, zero clears them.
Returns
A non-zero error value on failure and 0 on success, some possible errors are:
Return values
MDBX_EINVALAn invalid parameter was specified.

◆ mdbx_env_set_geometry()

LIBMDBX_API int mdbx_env_set_geometry ( MDBX_env env,
intptr_t  size_lower,
intptr_t  size_now,
intptr_t  size_upper,
intptr_t  growth_step,
intptr_t  shrink_threshold,
intptr_t  pagesize 
)

Set all size-related parameters of environment, including page size and the min/max size of the memory map.

In contrast to LMDB, the MDBX provide automatic size management of an database according the given parameters, including shrinking and resizing on the fly. From user point of view all of these just working. Nevertheless, it is reasonable to know some details in order to make optimal decisions when choosing parameters.

Both mdbx_env_set_geometry() and legacy mdbx_env_set_mapsize() are inapplicable to read-only opened environment.

Both mdbx_env_set_geometry() and legacy mdbx_env_set_mapsize() could be called either before or after mdbx_env_open(), either within the write transaction running by current thread or not:

  • In case mdbx_env_set_geometry() or legacy mdbx_env_set_mapsize() was called BEFORE mdbx_env_open(), i.e. for closed environment, then the specified parameters will be used for new database creation, or will be applied during opening if database exists and no other process using it.

    If the database is already exist, opened with MDBX_EXCLUSIVE or not used by any other process, and parameters specified by mdbx_env_set_geometry() are incompatible (i.e. for instance, different page size) then mdbx_env_open() will return MDBX_INCOMPATIBLE error.

    In another way, if database will opened read-only or will used by other process during calling mdbx_env_open() that specified parameters will silently discarded (open the database with MDBX_EXCLUSIVE flag to avoid this).

  • In case mdbx_env_set_geometry() or legacy mdbx_env_set_mapsize() was called after mdbx_env_open() WITHIN the write transaction running by current thread, then specified parameters will be applied as a part of write transaction, i.e. will not be completely visible to any others processes until the current write transaction has been committed by the current process. However, if transaction will be aborted, then the database file will be reverted to the previous size not immediately, but when a next transaction will be committed or when the database will be opened next time.
  • In case mdbx_env_set_geometry() or legacy mdbx_env_set_mapsize() was called after mdbx_env_open() but OUTSIDE a write transaction, then MDBX will execute internal pseudo-transaction to apply new parameters (but only if anything has been changed), and changes be visible to any others processes immediately after succesful completion of function.

Essentially a concept of "automatic size management" is simple and useful:

  • There are the lower and upper bounds of the database file size;
  • There is the growth step by which the database file will be increased, in case of lack of space;
  • There is the threshold for unused space, beyond which the database file will be shrunk;
  • The size of the memory map is also the maximum size of the database;
  • MDBX will automatically manage both the size of the database and the size of memory map, according to the given parameters.

So, there some considerations about choosing these parameters:

  • The lower bound allows you to prevent database shrinking below certain reasonable size to avoid unnecessary resizing costs.
  • The upper bound allows you to prevent database growth above certain reasonable size. Besides, the upper bound defines the linear address space reservation in each process that opens the database. Therefore changing the upper bound is costly and may be required reopening environment in case of MDBX_UNABLE_EXTEND_MAPSIZE errors, and so on. Therefore, this value should be chosen reasonable large, to accommodate future growth of the database.
  • The growth step must be greater than zero to allow the database to grow, but also reasonable not too small, since increasing the size by little steps will result a large overhead.
  • The shrink threshold must be greater than zero to allow the database to shrink but also reasonable not too small (to avoid extra overhead) and not less than growth step to avoid up-and-down flouncing.
  • The current size (i.e. size_now argument) is an auxiliary parameter for simulation legacy mdbx_env_set_mapsize() and as workaround Windows issues (see below).

Unfortunately, Windows has is a several issue with resizing of memory-mapped file:

  • Windows unable shrinking a memory-mapped file (i.e memory-mapped section) in any way except unmapping file entirely and then map again. Moreover, it is impossible in any way when a memory-mapped file is used more than one process.
  • Windows does not provide the usual API to augment a memory-mapped file (i.e. a memory-mapped partition), but only by using "Native API" in an undocumented way.

MDBX bypasses all Windows issues, but at a cost:

  • Ability to resize database on the fly requires an additional lock and release SlimReadWriteLock during each read-only transaction.
  • During resize all in-process threads should be paused and then resumed.
  • Shrinking of database file is performed only when it used by single process, i.e. when a database closes by the last process or opened by the first. = Therefore, the size_now argument may be useful to set database size by the first process which open a database, and thus avoid expensive remapping further.

For create a new database with particular parameters, including the page size, mdbx_env_set_geometry() should be called after mdbx_env_create() and before mdbx_env_open(). Once the database is created, the page size cannot be changed. If you do not specify all or some of the parameters, the corresponding default values will be used. For instance, the default for database size is 10485760 bytes.

If the mapsize is increased by another process, MDBX silently and transparently adopt these changes at next transaction start. However, mdbx_txn_begin() will return MDBX_UNABLE_EXTEND_MAPSIZE if new mapping size could not be applied for current process (for instance if address space is busy). Therefore, in the case of MDBX_UNABLE_EXTEND_MAPSIZE error you need close and reopen the environment to resolve error.

Note
Actual values may be different than your have specified because of rounding to specified database page size, the system page size and/or the size of the system virtual memory management unit. You can get actual values by mdbx_env_sync_ex() or see by using the tool mdbx_chk with the -v option.

Legacy mdbx_env_set_mapsize() correspond to calling mdbx_env_set_geometry() with the arguments size_lower, size_now, size_upper equal to the size and -1 (i.e. default) for all other parameters.

Parameters
[in]envAn environment handle returned by mdbx_env_create()
[in]size_lowerThe lower bound of database size in bytes. Zero value means "minimal acceptable", and negative means "keep current or use default".
[in]size_nowThe size in bytes to setup the database size for now. Zero value means "minimal acceptable", and negative means "keep current or use default". So, it is recommended always pass -1 in this argument except some special cases.
[in]size_upperThe upper bound of database size in bytes. Zero value means "minimal acceptable", and negative means "keep current or use default". It is recommended to avoid change upper bound while database is used by other processes or threaded (i.e. just pass -1 in this argument except absolutely necessary). Otherwise you must be ready for MDBX_UNABLE_EXTEND_MAPSIZE error(s), unexpected pauses during remapping and/or system errors like "address busy", and so on. In other words, there is no way to handle a growth of the upper bound robustly because there may be a lack of appropriate system resources (which are extremely volatile in a multi-process multi-threaded environment).
[in]growth_stepThe growth step in bytes, must be greater than zero to allow the database to grow. Negative value means "keep current or use default".
[in]shrink_thresholdThe shrink threshold in bytes, must be greater than zero to allow the database to shrink and greater than growth_step to avoid shrinking right after grow. Negative value means "keep current or use default". Default is 2*growth_step.
[in]pagesizeThe database page size for new database creation or -1 otherwise. Once the database is created, the page size cannot be changed. Must be power of 2 in the range between MDBX_MIN_PAGESIZE and MDBX_MAX_PAGESIZE. Zero value means "minimal acceptable", and negative means "keep current or use default".
Returns
A non-zero error value on failure and 0 on success, some possible errors are:
Return values
MDBX_EINVALAn invalid parameter was specified, or the environment has an active write transaction.
MDBX_EPERMTwo specific cases for Windows: 1) Shrinking was disabled before via geometry settings and now it enabled, but there are reading threads that don't use the additional SRWL (which is required to avoid Windows issues). 2) Temporary close memory mapped is required to change geometry, but there read transaction(s) is running and no corresponding thread(s) could be suspended since the MDBX_NOTLS mode is used.
MDBX_EACCESSThe environment opened in read-only.
MDBX_MAP_FULLSpecified size smaller than the space already consumed by the environment.
MDBX_TOO_LARGESpecified size is too large, i.e. too many pages for given size, or a 32-bit process requests too much bytes for the 32-bit address space.

◆ mdbx_env_set_mapsize()

MDBX_DEPRECATED int mdbx_env_set_mapsize ( MDBX_env env,
size_t  size 
)
inline

◆ mdbx_env_set_maxdbs()

int mdbx_env_set_maxdbs ( MDBX_env env,
MDBX_dbi  dbs 
)
inline

Set the maximum number of named databases for the environment.

This function is only needed if multiple databases will be used in the environment. Simpler applications that use the environment as a single unnamed database can ignore this option. This function may only be called after mdbx_env_create() and before mdbx_env_open().

Currently a moderate number of slots are cheap but a huge number gets expensive: 7-120 words per transaction, and every mdbx_dbi_open() does a linear search of the opened slots.

See also
mdbx_env_get_maxdbs()
Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]dbsThe maximum number of databases.
Returns
A non-zero error value on failure and 0 on success, some possible errors are:
Return values
MDBX_EINVALAn invalid parameter was specified.
MDBX_EPERMThe environment is already open.

◆ mdbx_env_set_maxreaders()

int mdbx_env_set_maxreaders ( MDBX_env env,
unsigned  readers 
)
inline

Set the maximum number of threads/reader slots for for all processes interacts with the database.

This defines the number of slots in the lock table that is used to track readers in the the environment. The default is about 100 for 4K system page size. Starting a read-only transaction normally ties a lock table slot to the current thread until the environment closes or the thread exits. If MDBX_NOTLS is in use, mdbx_txn_begin() instead ties the slot to the MDBX_txn object until it or the MDBX_env object is destroyed. This function may only be called after mdbx_env_create() and before mdbx_env_open(), and has an effect only when the database is opened by the first process interacts with the database.

See also
mdbx_env_get_maxreaders()
Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]readersThe maximum number of reader lock table slots.
Returns
A non-zero error value on failure and 0 on success, some possible errors are:
Return values
MDBX_EINVALAn invalid parameter was specified.
MDBX_EPERMThe environment is already open.

◆ mdbx_env_set_option()

LIBMDBX_API int mdbx_env_set_option ( MDBX_env env,
const MDBX_option_t  option,
uint64_t  value 
)

Sets the value of a runtime options for an environment.

Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]optionThe option from MDBX_option_t to set value of it.
[in]valueThe value of option to be set.
See also
MDBX_option_t
mdbx_env_get_option()
Returns
A non-zero error value on failure and 0 on success.

◆ mdbx_env_set_syncbytes()

int mdbx_env_set_syncbytes ( MDBX_env env,
size_t  threshold 
)
inline

Sets threshold to force flush the data buffers to disk, even any of MDBX_SAFE_NOSYNC flag in the environment.

See also
mdbx_env_get_syncbytes
MDBX_opt_sync_bytes

The threshold value affects all processes which operates with given environment until the last process close environment or a new value will be settled.

Data is always written to disk when mdbx_txn_commit() is called, but the operating system may keep it buffered. MDBX always flushes the OS buffers upon commit as well, unless the environment was opened with MDBX_SAFE_NOSYNC, MDBX_UTTERLY_NOSYNC or in part MDBX_NOMETASYNC.

The default is 0, than mean no any threshold checked, and no additional flush will be made.

Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]thresholdThe size in bytes of summary changes when a synchronous flush would be made.
Returns
A non-zero error value on failure and 0 on success.

◆ mdbx_env_set_syncperiod()

int mdbx_env_set_syncperiod ( MDBX_env env,
unsigned  seconds_16dot16 
)
inline

Sets relative period since the last unsteady commit to force flush the data buffers to disk, even of MDBX_SAFE_NOSYNC flag in the environment.

See also
mdbx_env_get_syncperiod
MDBX_opt_sync_period

The relative period value affects all processes which operates with given environment until the last process close environment or a new value will be settled.

Data is always written to disk when mdbx_txn_commit() is called, but the operating system may keep it buffered. MDBX always flushes the OS buffers upon commit as well, unless the environment was opened with MDBX_SAFE_NOSYNC or in part MDBX_NOMETASYNC.

Settled period don't checked asynchronously, but only by the mdbx_txn_commit() and mdbx_env_sync() functions. Therefore, in cases where transactions are committed infrequently and/or irregularly, polling by mdbx_env_sync() may be a reasonable solution to timeout enforcement.

The default is 0, than mean no any timeout checked, and no additional flush will be made.

Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]seconds_16dot16The period in 1/65536 of second when a synchronous flush would be made since the last unsteady commit.
Returns
A non-zero error value on failure and 0 on success.

◆ mdbx_env_set_userctx()

LIBMDBX_API int mdbx_env_set_userctx ( MDBX_env env,
void *  ctx 
)

Sets application information (a context pointer) associated with the environment.

See also
mdbx_env_get_userctx()
Parameters
[in]envAn environment handle returned by mdbx_env_create().
[in]ctxAn arbitrary pointer for whatever the application needs.
Returns
A non-zero error value on failure and 0 on success.