Facade to store and retrieve Job information from permanent storage.
A usage warning¶
This module saves Python objects using the pickle framework: thus,
the Application subclass corresponding to a job must be already
loaded (or at least
import-able) in the Python interpreter for
pickle to be able to ‘undump’ the object from its on-disk
In other words, if you create a custom Application subclass in some client code, GC3Utils won’t be able to read job files created by this code, because the class definition is not available in GC3Utils.
The recommended simple workaround is for a stand-alone script to ‘import self’ and then use the fully qualified name to run the script. In other words, start your script with this boilerplate code:
if __name__ == '__main__': import myscriptname myscriptname.MyScript().run()
The rest of the script now runs as the
myscript module, which does
Of course, the
myscript.py file must be in the search path of
the Python interpreter, or GC3Utils will still complain!
make_store(uri, *args, **extra_args)¶
Factory producing concrete
Given a URL and (optionally) initialization arguments, return a fully-constructed
The only required argument is uri; if any other arguments are present in the function invocation, they are passed verbatim to the constructor associated with the scheme of the given uri.
>>> fs1 = make_store('file:///tmp') >>> fs1.__class__.__name__ 'FilesystemStore'
Argument uri can also consist of a path name, in which case a URL scheme ‘file:///‘ is assumed:
>>> fs2 = make_store('/tmp') >>> fs2.__class__.__name__ 'FilesystemStore'
A mix-in class to mark that an object should be persisted by its ID.
Any instance of this class is saved as an ‘external reference’ when a container holding a reference to it is saved.
IdFactory(prefix=None, next_id_fn=None, id_class=<class 'gc3libs.persistence.idfactory.Id'>)¶
Automatically generate a “unique identifier” (of class Id). Object identifiers are temporally unique: no identifier will (ever) be re-used, even in different invocations of the program.
Return a new “unique identifier” instance (a string).
Pre-allocate n IDs. Successive invocations of the Id constructor will return one of the pre-allocated, with a potential speed gain if many Id objects are constructed in a loop.
IdFactorybehavior and generate IDs starting with a lowercase
FilesystemStore(directory='/home/docs/.gc3/jobs', idfactory=<gc3libs.persistence.idfactory.IdFactory object>, protocol=2, **extra_args)¶
Save and load objects in a given directory. Uses Python’s standard pickle module to serialize objects onto files.
All objects are saved as files in the given directory (default: gc3libs.Default.JOBS_DIR). The file name is the object ID.
If an object contains references to other Persistable objects, these are saved in the file they would have been saved if the save method was called on them in the first place, and only an ‘external reference’ is saved in the pickled container. This ensures that: (1) only one copy of a shared object is ever saved, and (2) any shared reference to Persistable objects is correctly restored when restoring the container.
The default idfactory assigns object IDs by appending a sequential number to the class name; see class Id for details.
The protocol argument specifies the serialization protocol to use, if different from gc3libs.persistence.serialization.DEFAULT_PROTOCOL.
Any extra keyword arguments are ignored for compatibility with SqlStore.
Return list of IDs of saved Job objects.
This is an optional method; classes that do not implement it should raise a NotImplementedError exception.
Load a saved object given its ID, and return it.
Delete a given object from persistent storage, given its ID.
Replace the object already saved with the given ID with a copy of obj.
Save an object, and return an ID.