gc3libs.poller¶
This module implements “pollers”. A “Poller” is an object that monitors a given URL and returns events whenever a new object is created inside that URL.
-
class
gc3libs.poller.
FilePoller
(url, recurse=False, **kw)¶ Track events on the filesystem using Python’s standard os module.
Params recurse: When True
, automatically track events in any (already-existing or newly-created) subdirectory.Warning
In order to issue ‘modified’ events, this class relies on checking an inode’s st_mtime field, which only provides 1-second resolution. Modification events that happen too close will not be told apart as distinct; in particular, modifying a file less than 1s after creating it will not be detected.
This implementation is only used to track Url with file schema whenever the
inotify_simple
module is not available.-
get_new_events
()¶ Iterate over events that happened since last call to this method.
Returns a list of tuples (subject, event).
A subject is a unique identifier for a watched “thing”: the exact form and type depends on the actual concrete class; pollers that watch the filesystem or HTTP-accessible resources will use a URL (
gc3libs.url.Url
) as a subject instance, but e.g. pollers that watch a database table might use a row ID instead.The associated event is one or more of the following strings:
created
: the subject has been created since the last call toget_new_events()
;modified
: the subject has changed since the last call toget_new_events()
;deleted
: the subject has been deleted since the last call toget_new_events()
;
Depending on the concrete poller class, some events might never occur, or cannot be detected. Most notably, only filesystem-watching pollers might be able to generate meaningful
modified
events.
-
recurse
¶ Whether the poller is watching the entire directory tree pointed to by
self.url
, or only the directory at its top level.
-
-
class
gc3libs.poller.
INotifyPoller
(url, recurse=False, **kw)¶ Use Linux’ inofity to track new events on the specified filesystem location.
Params recurse: When True
, automatically track events in any (already-existing or newly-created) subdirectory.This poller is used by default when the inotify_simple Python package is available and the URL has a file schema.
Warning
On Linux, the maximum number of inotify descriptors that a user can open is limited by the kernel parameters:
fs.inotify.max_user_instances
fs.inotify.max_user_watches
fs.inotify.max_queued_events
See also the inotify(7) manpage
-
get_new_events
()¶ Iterate over events that happened since last call to this method.
Returns a list of tuples (subject, event).
A subject is a unique identifier for a watched “thing”: the exact form and type depends on the actual concrete class; pollers that watch the filesystem or HTTP-accessible resources will use a URL (
gc3libs.url.Url
) as a subject instance, but e.g. pollers that watch a database table might use a row ID instead.The associated event is one or more of the following strings:
created
: the subject has been created since the last call toget_new_events()
;modified
: the subject has changed since the last call toget_new_events()
;deleted
: the subject has been deleted since the last call toget_new_events()
;
Depending on the concrete poller class, some events might never occur, or cannot be detected. Most notably, only filesystem-watching pollers might be able to generate meaningful
modified
events.
-
recurse
¶ Whether the poller is watching the entire directory tree pointed to by
self.url
, or only the directory at its top level.
-
class
gc3libs.poller.
Poller
(url, **kw)¶ Abstract class for an URL Poller.
A
Poller
is a class that tracks new events on a specificUrl
. When calling theget_events()
it will return a list of tuples (Url, mask) containing the events occurred for each one of the underlying URLs.-
get_new_events
()¶ Iterate over events that happened since last call to this method.
Returns a list of tuples (subject, event).
A subject is a unique identifier for a watched “thing”: the exact form and type depends on the actual concrete class; pollers that watch the filesystem or HTTP-accessible resources will use a URL (
gc3libs.url.Url
) as a subject instance, but e.g. pollers that watch a database table might use a row ID instead.The associated event is one or more of the following strings:
created
: the subject has been created since the last call toget_new_events()
;modified
: the subject has changed since the last call toget_new_events()
;deleted
: the subject has been deleted since the last call toget_new_events()
;
Depending on the concrete poller class, some events might never occur, or cannot be detected. Most notably, only filesystem-watching pollers might be able to generate meaningful
modified
events.
-
-
class
gc3libs.poller.
SwiftPoller
(url, **kw)¶ Periodically check a SWIFT bucket and trigger events when new objects are created.
Right now, a valid URL can be one of the following form:
- If the keystone endpoint is reachable via HTTP, either one of:
- swift://<user>+<tenant>:<password>@<keystone-url>?container
- swt://<user>+<tenant>:<password>@<keystone-url>?container
- If the keystone endpoint is reachable via HTTPS, either one of:
- swifts://<user>+<tenant>:<password>@<keystone-url>?container
- swts://<user>+<tenant>:<password>@<keystone-url>?container
We assume that keystone auth version 2 is used.
-
get_new_events
()¶ Iterate over events that happened since last call to this method.
Returns a list of tuples (subject, event).
A subject is a unique identifier for a watched “thing”: the exact form and type depends on the actual concrete class; pollers that watch the filesystem or HTTP-accessible resources will use a URL (
gc3libs.url.Url
) as a subject instance, but e.g. pollers that watch a database table might use a row ID instead.The associated event is one or more of the following strings:
created
: the subject has been created since the last call toget_new_events()
;modified
: the subject has changed since the last call toget_new_events()
;deleted
: the subject has been deleted since the last call toget_new_events()
;
Depending on the concrete poller class, some events might never occur, or cannot be detected. Most notably, only filesystem-watching pollers might be able to generate meaningful
modified
events.
- If the keystone endpoint is reachable via HTTP, either one of:
-
gc3libs.poller.
make_poller
(url, **extra)¶ Factory method that returns the registered poller for the specified
gc3libs.url.Url
.