gc3libs.backends.batch

This module provides a generic BatchSystem class from which all batch-like backends should inherit.

class gc3libs.backends.batch.BatchSystem(name, architecture, max_cores, max_cores_per_job, max_memory_per_core, max_walltime, auth, frontend, transport, keyfile=None, accounting_delay=15, **extra_args)

Base class for backends dealing with a batch-queue system (e.g., PBS/TORQUE, Grid Engine, etc.)

This is an abstract class, that you should subclass in order to interface with a given batch queuing system. (Remember to call this class’ constructor in the derived class __init__ method.)

cancel_job(*args, **kwargs)

Cancel a running job. If app is associated to a queued or running remote job, tell the execution middleware to cancel it.

close(*args, **kwargs)

Return True if the list of files is expressed in one of the file transfer protocols the LRMS supports.

Return False otherwise.

free(*args, **kwargs)

Free up any remote resources used for the execution of app. In particular, this should delete any remote directories and files.

Call this method when app.execution.state is anything other than TERMINATED results in undefined behavior and will likely be the cause of errors later on. Be cautious.

get_epilogue_script(app)

This method will get the epilogue script(s) for the app application and will return a string which contains the contents of the script(s) merged together.

get_jobid_from_submit_output(output, regexp)

Parse the output of the submission command. Regexp is provided by the caller.

get_prologue_script(app)

This method will get the prologue script(s) for the app application and will return a string which contains the contents of the script(s) merged together.

get_results(*args, **kwargs)

Retrieve job output files into local directory download_dir (which must already exists). Will not overwrite existing files, unless the optional argument overwrite is True.

peek(*args, **kwargs)

Download size bytes (at offset offset from the start) from remote file remote_filename and write them into local_file. If size is None (default), then snarf contents of remote file from offset unto the end.

Argument local_file is either a local path name (string), or a file-like object supporting a .write() method. If local_file is a path name, it is created if not existent, otherwise overwritten.

Argument remote_filename is the name of a file in the remote job “sandbox”.

Any exception raised by operations will be passed through.

submit_job(*args, **kwargs)

This method will create a remote directory to store job’s sandbox, and will copy the sandbox in there.

update_job_state(*args, **kwargs)

Query the state of the remote job associated with app and update app.execution.state accordingly. Return the corresponding Run.State; see Run.State for more details.

validate_data(data_file_list)

Return True if the list of files is expressed in one of the file transfer protocols the LRMS supports.

Return False otherwise.

gc3libs.backends.batch.generic_filename_mapping(jobname, jobid, file_name)

Map STDOUT/STDERR filenames (as recorded in Application.outputs) to commonly used default STDOUT/STDERR file names (e.g., <jobname>.o<jobid>).