SQL-based storage of GC3pie objects.

class gc3libs.persistence.sql.IntId
class gc3libs.persistence.sql.SqlStore(url, table_name=None, idfactory=None, extra_fields=None, create=True, **extra_args)

Save and load objects in a SQL db, using python’s pickle module to serialize objects into a specific field.

Access to the DB is done via SQLAlchemy module, therefore any driver supported by SQLAlchemy will be supported by this class.

The url argument is used to access the store. It is supposed to be a gc3libs.url.Url class, and therefore may contain username, password, host and port if they are needed by the db used.

The table_name argument is the name of the table to create. By default it’s store. Alternatively, the table name can be given in the “fragment” part of the database URL, as #table=... (replace ... with the actual table name). The constructor argument takes precedence over the table name specified in the DB URL.

The constructor will create the table_name table if it does not exist, but if there already is such a table it will assume that its schema is compatible with our needs. A minimal table schema is as follows:

| Field     | Type         | Null | Key | Default |
| id        | int(11)      | NO   | PRI | NULL    |
| data      | blob         | YES  |     | NULL    |
| state     | varchar(128) | YES  |     | NULL    |

The meaning of the fields is:

  • id: this is the id returned by the save() method and uniquely identifies a stored object.
  • data: serialized Python object.
  • state: if the object is a Task instance, this will be its current execution state.

The extra_fields constructor argument is used to extend the database. It must contain a mapping *column*: *function* where:

  • column is a sqlalchemy.Column object.
  • function is a function which takes the object to be saved as argument and returns the value to be stored into the database. Any exception raised by this function will be ignored. Classes GetAttribute and GetItem in module get provide convenient helpers to save object attributes into table columns.

For each extra column the save() method will call the corresponding function in order to get the correct value to store into the DB.

Any extra keyword arguments are ignored for compatibility with FilesystemStore.


Clear the loaded objects cache (if any).

Subsequent load() calls are guaranteed to re-load the data directly from the backing store.


Return list of IDs of saved Job objects.

This is an optional method; classes that do not implement it should raise a NotImplementedError exception.


Load a saved object given its ID, and return it.


Dispose current SQLAlchemy engine (if any). A new SQLAlchemy engine will be initialized upon the next interaction with a DB.

This method only exists to allow SessionBasedDaemon and similar applications that can do DB operations after fork()ing to continue to operate, without incurring into a SQLAlchemy “OperationalError: (…) could not receive data from server: Transport endpoint is not connected”


Delete a given object from persistent storage, given its ID.

replace(id_, obj)

Replace the object already saved with the given ID with a copy of obj.


Save an object, and return an ID.


Deprecated compatibility alias for SqlStore._tables

gc3libs.persistence.sql.make_sqlstore(url, *args, **extra_args)

Return a SqlStore instance, given a SQLAlchemy URL and optional initialization arguments.

This function is a bridge between the generic factory functions provided by gc3libs.persistence.make_store() and gc3libs.persistence.register() and the class constructor SqlStore:class.


| >>> ss1 = make_sqlstore(gc3libs.url.Url('sqlite:////tmp/foo.db'))
| >>> ss1.__class__.__name__
| 'SqlStore'