AgaveDB

A multiuser-aware key/value store built using the Agave metadata API

The library interface is modeled on pickledb, which is inspired by Redis. A key difference between AgaveDB and some other key stores is that it supports access controls on a per-key basis.

Usage:

from agavedb import AgaveKeyValStore, Agave
ag = Agave.restore()
db = AgaveKeyValStore(ag)
db.set('keyname', 'value')

Quick Tour

An implicit assumption is that you have an existing TACC.cloud client set up using AgavePy or the TACC Cloud CLI. Install AgaveDB from PyPi:

pip install agavedb

Start coding. AgaveDB doesn’t have a CLI at present.

>>> from agavedb import AgaveKeyValStore, Agave
>>> ag = Agave.restore()
>>> db = AgaveKeyValStore(ag, prefix="testing")
>>> db.set('key1', 'value1')
'key1'
>>> db.set('key2', 'value2')
'key2'
>>> db.set('key3', 'value4')
True
>>> db.setacl('key3', {'username': 'taco', 'permission': {'read': True, 'write': True}}')
True
>>> db.setacl('key3', {'username': 'world', 'permission': {'read': True, 'write': False}}')
True
>>> db.getacls('key3')
[{'username': 'matt', 'permission': {'read': True, 'write': True}}, {'username': 'taco', 'permission': {'read': True, 'write': True}}, {'username': 'world', 'permission': {'read': True, 'write': False}}]
>>> db.getacls('key3', 'taco')
[{'username': 'taco', 'permission': {'read': True, 'write': True}}]
>>> db.setacl('key3', {'username': 'taco', 'permission': {'read': True, 'write': False}}')
True
>>> db.getacls('key3', 'taco')
[{'username': 'taco', 'permission': {'read': True, 'write': False}}]
>>> db.remacl('key3', 'taco')
True
>>> db.getacls('key3', 'taco')
# Inherits +read from the world user
[{'username': 'taco', 'permission': {'read': True, 'write': False}}]
>>> db.get('key1')
u'value1'
>>> db.set('key1', 'value3')
'key1'
>>> db.get('key1')
u'value3'
>>> db.getall()
[u'key1', u'key2', u'key3']
>>> db.rem('key1')
True
>>> db.getall()
[u'key2', u'key3']
>>> db.set(db.genid(), 'hello world')
'MZgY69k1ZMd8'
>>> db.get('MZgY69k1ZMd8')
u'hello world'
>>> db.deldb()
True
>>> db.getall()
[]

Keys and Values

Keys

AgaveDB keys are Python str objects.

You have much flexibility in key naming them but there are rules:

  • Minimum key length is 4 characters. Really short keys aren’t necessary. You will probably thank yourself for writing user:1001:bugreports instead of u1001bg when it comes time to debug or maintain our code. The added space and memory requirement is trivial.
  • Maximum key length is 512 characters, but such long keys aren’t a great idea because they will consume memory and will be slow to retrieve. Consider using a hash (such as SHA1) to represent a larger value.
  • Schemas are good. Examples include object-type:id and user:1001. Dots, dashes, and underscores are often used as separators.
  • Whitespace is not allowed in keynames. Invalid characters will be removed via a safening process.

Values

There are two constraints on values:

  • The maximum size for an AgaveDB value is currently 4096 bytes.
  • AgaveDB values can only be str objects. Currently, numeric values are coerced to a string representation. Support for lists, dicts, and tuples will be added in future releases.

Indices and tables