|
kvs is a key-value datastore based on bplus trees, on-disk, thread-safe datastore. kvs.jar a single Java library with no third party dependencies, add it to the class path in order to use it. Disclaimer: create regular backups and use it with caution, any unusual use may result in errors or data loss. Last updated: 09.04.2026 kvs-2.0.2.jar sha256: c39ac965bcecbfdb29e85aa45ff9b085d86aee394b0f6827bb98268c96dff90e kvs datastore - ex_kvs.zip. kvs datastore - ex_KeyValue.zip. kvs datastore with ep_server - ep_server_with_kvs.zip. kvs - previous_releases. First released: 19.06.2021
Quick start notes:
Set datastore path.
Create datastore instance.
InstanceConfig cfg = new InstanceConfig(PATH, DB_INSTANCE);
Datastore datastore = Datastore.instance(cfg);
for example:
InstanceConfig cfg = new InstanceConfig("f:/data/", 501);
Datastore datastore = Datastore.instance(cfg);
will create this path for this instance:
f:/data/datastore_501/
n is between 0 and 999, you can have up to 1000 instances, independent datastores, organized under the datastore path:
DATASTORE_PATH\datastore_000
DATASTORE_PATH\datastore_001
DATASTORE_PATH\datastore_002
...
DATASTORE_PATH\datastore_999
See examples how to get, insert, delete and create queries.
Entities
Key parent = KeyFactory.create_key(key_kind, name);
Entity entity = new Entity(e_kind, name_id, parent);
datastore.insert(entity);
Key k = KeyFactory.create_key(kind, name_id);
Entity e = datastore.get(k);
datastore.delete(entity);
name_id is one of the two type Long or String.
There is an implicit index "NAME_ID" which is always build for all kinds, ancestors, and kindless ancestors,
and is used to build other indexes.
Do not use "NAME_ID", "ROOT" or any of the above symbols when naming entities, kinds, keys etc.
"ROOT" is the default kind of an entity created without kind.
Multi-value properties are not supported for indexing.
Note: Properties values of the entities are red on demand, when you call e.get_property(),
and there is a StoreContext attached to each entity. Entities cannot inserted from one datastore instance
into another directly. You need to create a copy of the entity and then insert it.
Properties
Properties types must be one of the Java types:
TYPE_BYTE;
TYPE_SHORT;
TYPE_INT;
TYPE_LONG;
TYPE_FLOAT;
TYPE_DOUBLE;
TYPE_STRING;
TYPE_BYTE_ARRAY;
Queries
Queries read the indexes forward or reverse. Queries do not fetch data. To find index:
set_sort(...)
Queries execute on existing indexes only. There is no query planner or implicit full iteration for filtering.
It fails if the index in not found.
Set ranges:
set_id_range(Object from, Object to)
set_property_range(Object from[], Object to[])
The query executes on a single index and the result is from entity to entity within single index.
strat_key and end_key are build from id_range and property_range. Ranges are set in declaration order.
The property_names are compared in declaration order of the index and the id is a tie-breaker.
The range values must be the same type as the property_value. For ex. 5 for Integer and 5L or new Long(5) for Long.
set_policy(RangePolicy.INCLUSIVE, RangePolicy.EXCLUSIVE);
The policy applies to composite strat_key and end_key compiled from the ranges set.
reverse()
Possibly read the result in reverse. For ex. a composite index of
(NAME_ID, a, b) ASCENDING, DESCENDING, ASCENDING will read as DESCENDING, ASCENDING, DESCENDING.
Example:
try
{
Query q = new Query(kind);
QueryIterator it = datastore.iterator(q);
while(it.hasNext())
{
Entity entity = it.next();
}
} catch (DataStoreException e) {
e.printStackTrace();
}
Query q = new Query(kind).set_sort(property_name, SortDirection.ASCENDING).set_property_range(from, to);
Declared indexes.
Indexes are declared programmatically. You can change and rebuild indexes later. See Offline tools for rebuilding indexes.
Each index can be red in ASCENDING and DESCENDING order.
Declare composite index:
private void add_index()
{
DeclaredIndex dindex = new DeclaredIndex();
dindex.kind = kind;
dindex.auto = false;
dindex.properties = new DeclaredIndexProperty[2];
String p0[] = new String[]{P1, P2};
String p1[] = new String[]{P1, P2};
int p0_sort[] = {DeclaredIndexProperty.ASCENDING, DeclaredIndexProperty.ASCENDING};
int p1_sort[] = {DeclaredIndexProperty.ASCENDING, DeclaredIndexProperty.DESCENDING};
dindex.properties[0] = new DeclaredIndexProperty(p0, p0_sort);
dindex.properties[1] = new DeclaredIndexProperty(p1, p1_sort);
datastore.kind_configs().add_declared_index(dindex);
}
Create a Copy of the datastore;
Create copy instance of the datastore, recycles free space, removes entities changes history,
sorts btree blocks breadth-first in favor of queries.
Log file rvloga.
The changes after every operation (insert, delete) are committed in rvloga file.
Every commit represents a different snapshot of the datastore.
You can choose between auto flush, or manual flush, default is auto_flush=true with max_size=128MB.
The log will auto flush when exceeds the max_size;
If you choose manual flush you will have to lookup for the log file and flush it manually. Note: the log grows very fast.
In case of manual flush, you could backup the log files, before every flush, which will provide you with consistent restore points.
Note: auto flush may cause currently running queries to expire and transactions to fail. This is because ones the log is flushed,
the current snapshot within which the query executes is lost.
Offline Tools
Offline is a set of unsynchronized methods that can be run only offline.
Running these methods while the datastore is in use may result in a data loss.
"D:\Program Files\Java\jdk1.8.0_40"\bin\java -classpath E:/g_server/kvstore/kvs-2.0.1.jar com.aza.kvs.store.tools.Offline build-indexes f:/data/ 0
"D:\Program Files\Java\jdk1.8.0_40"\bin\java -classpath E:/g_server/kvstore/kvs-2.0.1.jar com.aza.kvs.store.tools.Offline check-indexes f:/data/ 0
"D:\Program Files\Java\jdk1.8.0_40"\bin\java -classpath E:/g_server/kvstore/kvs-2.0.1.jar com.aza.kvs.store.tools.Offline flush f:/data/ 0
"D:\Program Files\Java\jdk1.8.0_40"\bin\java -classpath E:/g_server/kvstore/kvs-2.0.1.jar com.aza.kvs.store.tools.Offline datastore-copy f:/data/ 0 f:/data/ 501
KeyValue
Since 1.0.5 introduces new, fast KeyValue interface for mapping primitive types Long/Int to Long/Double/Int, etc,
as well as limited size Strings and ByteArrays. Indexes have ordered keys only. Nodes are limited node size to 4096,
see examples how to configure KeyValue index. For ordered values and keys, Entities should be used.
IdxConfig ib = new IdxConfig("idx_name", KvType.KEY_STRING, KvType.VALUE_STRING);
ib.set_factor(30).set_key_len(8).set_value_len(8);
boolean created = datastore.get_kv_controller().create_if_not_exist(ib);
KeyValue kv = datastore.get_kv_controller().get_index(index_name);
AKey k = kv.create_SKey("key1"); // max len 8 bytes
kv.insert_string(k, "value1");
kv.commit();
kvs stores Strings in UTF-8. UTF-8 uses 1-4 bytes per character. For ex.: for Cyrillic set length to (num_chars*2bytes).
All values types take the whole length given in the IdxConfig. Thus, byte array values must be the same size as configured.
Strings are null-terminated within the declared length.
KeyValue methods are synchronized, but indexes are cached and any thread may call get_index(index_name) and any
of its methods. In which case you may have to synchronize the access to concrete indexes.
KeyValue.clear() will clear uncommitted inserts/deletes. It may be good to call it before inserts, or in try finally
block.
New in 2.0.2
- bug fix: export missing files
New in 2.0.1
- changed query interface
- optimized index set and name set
New in 2.0
- improved interface
- more relaxed exceptions model
- configure indexes programmatically
- optional In_MEMORY mode, cache parts or the whole datastore in memory
- optimized IO
- flattened entity keys, flattened entity values up to 8 bytes
New in 1.0.5.r3
- KeyValue - max value length is 1200 bytes.
New in 1.0.5.r2
- added byte array key - KeyValue.create_BAKey() for UUID, see examples.
- added more controls over the types and lengths of keys and values.
New in 1.0.5.r1
- revision of KeyValue interface, changed the behaviour of reads get/query
see KeyValue.read_uncommited(), examples 19, 20.
- bug fix optimizations
New in 1.0.5
- new KeyValue interface
- bug fix optimizations
New in 1.0.4
- new copy method
- bug fix composite index interface
New in 1.0.3
- bug fix in query range.policy.inclusive/exclusive
New in 1.0.2
- optimized log file
- optimized data formats
- 1.0.2 will flush the log before start
|
Contacts: aza384 at http://www.eptools4.com:9000/send.html