GC3Pie programming tutorials¶
Slides on specific topics¶
This is a list of tutorials made for the GC3Pie 2012 Training event that has been held in Zurich on 1st and 2nd of October 2012. The slides and tutorials are an introduction to the GC3Pie python package.
Introduction to the software: what is GC3Pie, what is it for, and an overview of its features for writing high-throughput computing scripts.
The Application class, the smallest building block of GC3Pie. Introduction to the concept of Job, states of an application and to the Core class.
How to define extra requirements for an application, such as the minimum amount of memory it will use, the number of cores needed or the architecture of the CPUs.
Introduction to the highest-level interface to build applications with GC3Pie, the SessionBasedScript. Information on how to create simple scripts that take care of the execution of your applications, from submission to getting back the final results.
Low-level tools to aid debugging the scripts.
Using a practical example (the Warholize Tutorial) we show how workflows are implemented with GC3Pie. The following slides will cover in more details the single steps needed to produce a complex workflow.
Description of the ParallelTaskCollection class, used to run tasks in parallel.
Description of the StagedTaskCollection class, used to run a sequence of a fixed number of jobs.
Description of the SequentialTaskCollection class, used to run a sequence of jobs that can be altered during runtime.
Simplest script you can create. It only uses Application and Engine classes to create an application, submit it, check its status and retrieve its output.
a SessionBasedScript that executes its argument as command. It can also run it multiple times by wrapping it in a ParallelTaskCollection or a SequentialTaskCollection, depending on a command line option. Useful for testing a configured resource.
a simple SessionBasedScript that sums two values by customizing a SequentialTaskCollection.
an enhanched version of the warholize script proposed in the Warholize Tutorial
In this tutorial we will show you how to use GC3Pie libraries in order to build a command line script which will run a complex workflow with both parallel and sequential tasks.
The tutorial itself contains the complete source code of the
application (cfr. Literate Programming on Wikipedia), so that you
will be able to test/modify it and produce a working
script by downloading the
pylit.py:file: script from the PyLit
Homepage and running the following command on the
from within the SVN tree of GC3Pie:
$ ./pylit warholize.txt warholize.py