session - persistent collections of tasks
Session(path, create=True, store_or_url=None, load=True, task_ids=None, **extra_args)¶
A ‘session’ is a persistent collection of tasks.
Tasks added to the session are persistently recorded using an instance of gc3libs.persistence.Store. Stores can be shared among different sessions: each session knows wich jobs it ‘owns’.
A session is associated to a directory, which holds all the data releated to that session. Specifically, two files are always created in the session directory andused internally by this class:
- index.txt: contains a list of all job IDs associated with this session;
- store.url: its contents are the URL of the store to create (as would be passed to the gc3libs.persistence.make_store factory).
To only argument needed to instantiate a session is the path of the directory; the directory name will be used as the identifier of the session itself. For example, the following code creates a temporary directory and the two files mentioned above inside it:
>>> import tempfile; tmpdir = tempfile.mktemp(dir='.') >>> session = Session(tmpdir) >>> for name in sorted(os.listdir(tmpdir)): ... print(name) created session_ids.txt store.url
When a Session object is created with a path argument pointing to an existing valid session, the index of jobs is automatically loaded into memory, and the store pointed to by the
store.urlfile in the session directory will be used, disregarding the contents of the `store_url` argument.
In other words, the store_url argument is only used when creating a new session. If no store_url argument is passed (i.e., it has its default value), a Session object will instantiate and use a
FileSystemStorestore, keeping data in the
jobssubdirectory of the session directory.
Methods add and remove are provided to manage the collection; the len() operator returns the number of tasks in the session; iteration over a session returns the tasks one by one:
>>> task1 = gc3libs.Task() >>> id1 = session.add(task1) >>> task2 = gc3libs.Task() >>> id2 = session.add(task2) >>> len(session) 2 >>> for t in session: ... print(type(t)) <class 'gc3libs.Task'> <class 'gc3libs.Task'> >>> session.remove(id1) >>> len(session) 1
When passed the flush=False optional argument, methods add and remove do not update the session metadata: i.e., the tasks are added or removed from the store and the in-memory task list, but the updated task list is not saved back to disk. This is useful when making many changes in a row; call Session.flush to persist the full set of changes.
Storeobject is anyway accessible in the
storeattribute of each Session instance:
>>> type(session.store) <class 'gc3libs.persistence.filesystem.FilesystemStore'>
However, Session defines methods save and load as a convenient proxy to the corresponding Store methods:
>>> obj = gc3libs.persistence.Persistable() >>> oid = session.save(obj) >>> obj2 = session.load(oid) >>> obj.persistent_id == obj2.persistent_id True
The whole session data can be removed by using method destroy:
>>> session.destroy() >>> os.path.exists(session.path) False
Add a Task to the current session, save it to the associated persistent storage, and return the assigned persistent_id:
>>> # create new, empty session >>> import tempfile; tmpdir = tempfile.mktemp(dir='.') >>> session = Session(tmpdir) >>> len(session) 0 >>> # add a task to it >>> task = gc3libs.Task() >>> tid1 = session.add(task) >>> len(session) 1
Duplicates are silently ignored: the same object can be added many times to the session, but gets the same ID each time:
>>> # add a different task >>> tid2 = session.add(task) >>> len(session) 1 >>> tid1 == tid2 True >>> # do cleanup >>> session.destroy() >>> os.path.exists(session.path) False
Remove the session directory and all the tasks it contains from the store which are associated to this session.
This will remove the associated task storage if and only if the storage is contained in the session directory!
Update session metadata.
Should be used after a save/remove operations, to ensure that the session state and metadata is correctly persisted.
Remove task identified by task_id from the current session but not from the associated storage.
Iterate over all tasks in this session.
TaskCollectionin a breadth-first manner.
Return set of all task IDs belonging to this session.
Return set of names of tasks belonging to this session.
load(obj_id, add=True, flush=True)¶
Load an object from persistent storage and return it.
This is just a convenience proxy for calling method load on the Store instance associated with this session.
load_many(obj_ids, add=True, flush=True)¶
Load objects given their IDs from persistent storage.
Return a dictionary mapping task ID to the actual retrieved
Remove task identified by task_id from the current session and from the associated storage.
Save an object to the persistent storage and return persistent_id of the saved object.
This is just a convenience proxy for calling method save on the Store instance associated with this session.
The object is not added to the session, nor is session meta-data updated:
# create an empty session >>> import tempfile; tmpdir = tempfile.mktemp(dir='.') >>> session = Session(tmpdir) >>> 0 == len(session) True # use `save` on an object >>> obj = gc3libs.persistence.Persistable() >>> oid = session.save(obj) # session is still empty >>> 0 == len(session) True # do cleanup >>> session.destroy() >>> os.path.exists(session.path) False
Save all modified tasks to persistent storage.
Create a file named finished in the session directory. It’s creation/modification time will be used to know when the session has finished.
Please note that Session does not know when a session is finished, so this method should be called by a
Create a file named created in the session directory. It’s creation/modification time will be used to know when the session has sarted.
TemporarySession(store_or_url, task_ids=None, delete=True, **extra_args)¶
Create a session from a store URL.
- The session index (list of task IDs belonging to the session) is
initialized from the entire list of jobs present in the given
Store(unless a list is explicitly passed in the task_ids argument to the constructor). This means that, unlike plain
TemporarySessionobjects cannot share the same store.
- The session directory (
Sessionconstructor) is created on a temporary location on the filesystem and deleted when the
- Timestamps will be set to the time the
TemporarySessionPython object is created; two
TemporarySessioninstances with the same backing store can have different creation timestamps, depending on when exactly they were instanciated.
- The session index (list of task IDs belonging to the session) is initialized from the entire list of jobs present in the given