GC3Pie programming tutorials

Slides on specific topics

This is a list of tutorials made for the GC3Pie 2012 Training event that has been held in Zurich on 1st and 2nd of October 2012. The slides and tutorials are an introduction to the GC3Pie python package.

Introduction ot GC3Pie

Introduction to the software.

Basic GC3Pie programming

The Application class, the smallest building block of GC3Pie. Introduction to the concept of Job, states of an application and to the Core class.

Application requirements

How to define extra requirements for an application, such as the minimum amount of memory it will use, the number of cores needed or the architecture of the CPUs.

Managint applications: the SessionBasedScript class

Introduction to the higest level interface to build applications with GC3Pie, the SessionBasedScript. Info on how to create simple scripts which will take care of the execution of your applications, from submission to getting back the results.

GC3Pie utils

Low level tools to debug your scripts.

Introduction to Workflows with GC3Pie

Using a practical example (the Warholize Tutorial) we show how workflows are implemented with GC3Pie. The following slides will cover in more details the single steps needed to produce a complex workflow.

ParallelTaskCollection

Description of the ParallelTaskCollection class, used to run tasks in parallel.

StagedTaskCollection

Description of the StagedTaskCollection class, used to run a sequence of a fixed number of jobs.

SequentialTaskCollection

Description of the SequentialTaskCollection class, used to run a sequence of jobs that can be altered during runtime.

Example scripts

gdemo_simple.py

Simple script that only uses Application and Core classes to create an application, submit it, check its status and retrieve its output.

gdemo_simple_engine.py

Similar to the previous script, this one uses, instead of Core, a more high-level class to manage jobs workflow: the Engine class.

warholize.py

an enhanched version of the warholize script proposed in the Warholize Tutorial

Warholize Tutorial

In this tutorial we will show you how to use GC3Pie libraries in order to build a command line script which will run a complex workflow with both parallel and sequential tasks.

The tutorial itself contains the complete source code of the application (cfr. Literate Programming on Wikipedia), so that you will be able to test/modify it and produce a working warholize.py script by downloading pylit from the PyLit Homepage and running the following command on the gc3pie/docs/gc3libs/tutorial/warholize.txt file, from whitin the svn tree of GC3Pie:

$    ./pylit warholize.txt warholize.py