gc3libs.config

Deal with GC3Pie configuration files.

class gc3libs.config.Configuration(*locations, **extra_args)

In-memory representation of the GC3Pie configuration.

This class provides facilities for:

  • parsing configuration files (methods load() and merge_file());
  • parsing a configuration from a python dictionary (method construct_from_cfg_dict());
  • validating the loaded values;
  • instanciating the internal GC3Pie objects resulting from the configuration (methods make_auth() and make_resource()).

The constructor takes a list of files to load (locations), a python dictionary of sections with key value pairs (cfg_dict), and a list of key=value pairs to provide defaults for the configuration. All three arguments are optional and can be omitted, resulting in a configuration containing only GC3Pie default values. If locations is not empty but there are no config files at those locations, the constructor will raise a NoAccessibleConfigurationFile exception if cfg_dict is None.

Example 1: initialization from config file:

>>> import os
>>> example_cfgfile = os.path.join(
...    os.path.dirname(__file__), 'etc/gc3pie.conf.example')
>>> cfg = Configuration(example_cfgfile)
>>> cfg.debug
'0'

Example 2: initialization from a Python dictionary:

>>> d = dict()
>>> d["DEFAULT"] = {"debug": 0}
>>> d["auth/ssh_bob"] = {
... "type": "ssh", "username": "your_ssh_user_name_on_computer_bob"}
>>> cfg = Configuration(cfg_dict=d)
>>> cfg.debug
0
>>> cfg.auths["ssh_bob"]["type"] == 'ssh'
True

Example 3: initialization from key=value list:

>>> cfg = Configuration(auto_enable_auth=False, foo=1, bar='baz')
>>> cfg.auto_enable_auth
False
>>> cfg.foo == 1
True
>>> cfg.bar == 'baz'
True

When all three arguments are supplied, configuration options are taken in the following order of precedence: * config file [highest priority], * Python dictionary [middle priority], * key=value list [lowest priority]

>>> # config file > Python dictionary
... d = {"DEFAULTS": {"debug": 1}}
>>> cfg = Configuration(example_cfgfile, config_dict=d)
>>> cfg.debug == '0'
True
>>>
>>> # config file > key=value list
... cfg = Configuration(example_cfgfile, debug=1)
>>> cfg.debug == '0'
True
>>>
>>> # Python dictionary > key=value list
... cfg = Configuration(config_dict=d, debug=0)
>>> cfg.debug == '0'
False

Example 4: default initialization:

>>> cfg = Configuration()
>>> cfg.auto_enable_auth
True
auth_factory

The instance of gc3libs.authentication.Auth used to manage auth access for the resources.

This is a read-only attribute, created upon first access with the values set in self.auths and self.auto_enabled.

construct_from_cfg_dict(cfg_dict, filename=None)

Create a Configuration object from the settings defined in cfg_dict.

Parameter cfg_dict may either be a Python dictionary, having the same general format as a configuration file, or a ConfigParser instance into which an INI-format configuration file has been read. See below for an example of a configuration file converted to a dictionary.

Parameters:
  • cfg_dict (dict) – The Python dictionary to load settings from.
  • filename (string) – Optional. If this dictionary was constructed from

a config file, filename is the name of the config file.

Example: A Configuration File:

[auth/ssh]
type = ssh
username = gc3pie

[resource/test]
type = shellcmd
auth = ssh
transport = local
max_memory_per_core = 2
max_walltime = 8
max_cores = 2
architecture = x86_64
override = False

[DEFAULT]
max_cores_per_job = 2

Example: Equivalent Dictionary:

>>> cfg_dict = {
...         'auth/ssh': {
...             'type': 'ssh',
...             'username': 'gc3pie'
...         },
...         'resource/test': {
...             'type': 'shellcmd',
...             'auth': 'ssh',
...             'transport': 'local',
...             'max_memory_per_core': '2',
...             'max_walltime': '8',
...             'max_cores': '2',
...             'architecture': 'x86_64',
...             'override': 'False'
...         },
...         'DEFAULT': {
...             'max_cores_per_job': '2'
...         }
...     }
>>>
load(*locations)

Merge settings from configuration files into this Configuration instance.

Environment variables and ~ references are expanded in the location file names.

If any of the specified files does not exist or cannot be read (for whatever reason), a message is logged but the error is ignored. However, a NoConfigurationFile exception is raised if none of the specified locations could be read.

Raises:gc3libs.exceptions.NoConfigurationFile – if none of the specified files could be read.
make_auth(name)

Return factory for auth credentials configured in section [auth/name].

make_resources(ignore_errors=True)

Make backend objects corresponding to the configured resources.

Return a dictionary, mapping the resource name (string) into the corresponding backend object.

By default, errors in constructing backends (e.g., due to a bad configuration) are silently ignored: the offending configuration is just dropped. This can be changed by setting the optional argument ignore_errors to False: in this case, an exception is raised whenever we fail to construct a backend.

merge_file(filename)

Read configuration files and merge the settings into this Configuration object.

Contrary to load() (which see), the file name is taken literally and an error is raised if the file cannot be read for whatever reason.

Any parameter which is set in the configuration files’ [DEFAULT] section, and whose name does not start with underscore (_) defines an attribute in the current Configuration.

Warning

No type conversion is performed on values set this way - so they all end up being strings!

Raises:gc3libs.exceptions.ConfigurationError – if the configuration file does not exist, cannot be read, is corrupt or has wrong format.