Installation of GC3Pie¶

Quick start¶

We provide an installation script which automatically try to install GC3pie in your home directory. The quick installation procedure has only been tested on variants of the GNU/Linux operating system. (However, the script should work on MacOSX as well, provided you follow the preparation steps outlined in the “MacOSX installation” section below.)

To install GC3Pie just type this at your terminal prompt:

sh -c "$(wget -O- http://gc3pie.googlecode.com/svn/install.sh)"

If wget is not installed in your computer, you should use curl instead:

sh -c "$(curl -s http://gc3pie.googlecode.com/svn/install.sh)"

The above command creates a directory $HOME/gc3pie and installs the latest release of GC3Pie and all its dependencies into it.

In case you have trouble running the installation script, please send an email to gc3pie@googlegroups.com asking for help. Include the full output of the script in your email, in order to help us to identify the problem.

Non-standard installation options¶

The installation script accept a few options that select alternatives to the standard behavior. In order to use these options, you have to:

download the installation script into a file named install.sh:
```
wget http://gc3pie.googlecode.com/svn/install.sh
```
run the command:
```
sh ./install.sh [options]
```
replacing the string [options] with the actual options you want to pass to the script.

The accepted options are as follows:

-d DIRECTORY

Install GC3Pie in location DIRECTORY instead of $HOME/gc3pie

--overwrite

Overwrite the destination directory if it already exists. Default behavior is to abort installation.

--develop

Instead of installing the latest release of GC3Pie, it will install the development branch from the SVN repository.

--yes

Run non-interactively, and assume a “yes” reply to every question.

-p PYTHON

Uses the given PYTHON program as python interpreter. By default the installation script looks for a python binary in the standard $PATH.

--no-gc3apps

Do not install any of the GC3Apps, e.g., gcodeml, grosetta and ggamess.

Manual installation¶

In case you can’t or don’t want to use the automatic installation script, the following instructions will guide you through all the steps needed to manually install GC3Pie on your computer.

These instructions show how to install GC3Pie from the GC3 source repository into a separate python environment (called virtualenv). Installation into a virtualenv has two distinct advantages:

All code is confined in a single directory, and can thus be easily replaced/removed.

Better dependency handling: additional Python packages that GC3Pie depends upon can be installed even if they conflict with system-level packages.

Install software prerequisites:
- On Debian/Ubuntu, install packages: subversion, python-dev, python-profiler and the C/C++ compiler:
```
apt-get install subversion python-dev python-profiler gcc g++
```
- On CentOS5, install packages subversion and python-devel and the C/C++ compiler:
```
yum install subversion python-devel gcc gcc-c++
```
- On other Linux distributions, you will need to install:
  - the svn command (from the SubVersion VCS)
  - Python development headers and libraries (for installing extension libraries written in C/C++)
  - the Python package pstats (it’s part of the Python standard library, but sometimes it needs separate installation)
  - a C/C++ compiler (this is usually installed by default).
If you intend to use also the ARC backend (required for SMSCG), you need the NorduGrid/ARC binaries and a working slcs-init command installed on the same machine where GC3Pie is. You can find instructions for installing it at:

http://www.smscg.ch/WP/middleware/ARC1/smscg_ui_Installation.html

Additional OS-specific installation details can be found at:

http://code.google.com/p/gc3pie/wiki/OSSpecificInstallDetails
If virtualenv is not already installed on your system, get the Python package and install it:
```
wget http://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.7.tar.gz
tar -xzf virtualenv-1.7.tar.gz && rm virtualenv-1.7.tar.gz
cd virtualenv-1.7/
```
If you are installing as root, the following command is all you need:
```
python setup.py install
```
If instead you are installing as a normal, unprivileged user, things get more complicated:
```
export PYTHONPATH=$HOME/lib64/python:$HOME/lib/python:$PYTHONPATH
export PATH=$PATH:$HOME/bin
mkdir -p $HOME/lib/python
python setup.py install --home $HOME
```
You will also have to add the two export lines above to the:
- $HOME/.bashrc file, if using the bash shell or to the
- $HOME/.cshrc file, if using the tcsh shell.
In any case, once virtualenv has been installed, you can exit its directory and remove it:
```
cd ..
rm -rf virtualenv-1.7
```
Create a virtualenv to host the GC3Pie installation, and cd into it:
```
virtualenv --system-site-packages $HOME/gc3pie
cd $HOME/gc3pie/
source bin/activate
```
In this step and in the following ones, the directory $HOME/gc3pie is going to be the installation folder of GC3Pie. You can change this to another directory path; any directory that’s writable by your Linux account will be OK.

If you are installing system-wide as root, we suggest you install GC3Pie into /opt/gc3pie instead.

Check-out the gc3pie files in a src/ directory:

svn co http://gc3pie.googlecode.com/svn/branches/2.0/gc3pie src

Install the gc3pie in “develop” mode, so any modification pulled from subversion is immediately reflected in the running environment:
```
cd src/
env CC=gcc ./setup.py develop
cd .. # back into the `gc3pie` directory
```
This will place all the GC3Pie command into the gc3pie/bin/ directory.

GC3Pie comes with driver scripts to run and manage large families of jobs from a few selected applications. These scripts are not installed by default because not everyone needs them.

Run the following commands to install the driver scripts for the applications you need:

# if you are insterested in GAMESS, do the following
ln -s '../src/gc3apps/gamess/ggamess.py' bin/ggamess

# if you are insterested in Rosetta, do the following
ln -s '../src/gc3apps/rosetta/gdocking.py' bin/gdocking
ln -s '../src/gc3apps/rosetta/grosetta.py' bin/grosetta

# if you are insterested in Codeml, do the following
ln -s '../src/gc3apps/codeml/gcodeml.py' bin/gcodeml

Before you can actually run the GC3Pie, you need to have a working configuration file; the ConfigurationFile Wiki page provides an explanation of the syntax.

The example configuration file can be used as a startup point if you want to enable access to the SMSCG infrastructure. Before you can actually use this file, you will need to:
- insert into it three values, for which we can NOT provide defaults:
1. aai_username:
  
  This is the “username” you are asked for when accessing any SWITCHaai/Shibboleth web page, e.g., https://gc3-aai01.uzh.ch/secure/
2. idp:
  
  Find this out with the command “slcs-info”: it prints a list of IdP (Identity Provider IDs) followed by the human-readable name of the associated institution. Pick the one that corresponds to you University. It is always the last two components of the University’s Internet domain name (e.g., “uzh.ch” or “ethz.ch”).
3. vo:
  
  In order to use SMSCG, you must sign up to a VO (Virtual Organisation). One the words “life”, “earth”, “atlas” or “crypto” should be here. Find out more at: http://www.smscg.ch/www/user/
Please, remember to uncomment the three lines, too.
- Enable the SMSCG resource in the Examples section of the file. To do this simply uncomment the following lines:
```
# [resource/smscg]
# enabled = false
# name = smscg
# type = arc0
# auth = smscg
# arc_ldap = ldap://giis.smscg.ch:2135/o=grid/mds-vo-name=Switzerland
# max_cores_per_job = 256
# max_memory_per_core = 2
# max_walltime = 24
# ncores = 8000
# architecture = i686, x86_64
```

Now you can check your GC3Pie installation; just type the command:

gc3utils --help

and you should see the following output appear on your screen:

Usage: gc3utils COMMAND [options]

Command `gc3utils` is a unified front-end to computing resources.
You can get more help on a specific sub-command by typing::
  gc3utils COMMAND --help
where command is one of these:
  clean
  get
  info
  kill
  list
  resub
  stat
  tail

If you get some errors, do not despair! The GC3Pie users mailing-list <gc3pie@googlegroups.com> is there to help you :-)

Upgrade¶

If you used the installation script, the fastest way to upgrade is just to reinstall:

De-activate the current GC3Pie virtual environment:
```
deactivate
```
(If you get an error “command not found”, do not worry and proceed on to the next step; in case of other errors please stop here and report to the GC3Pie users mailing-list <mailto:gc3pie.googlegroups.com>.)
Move the $HOME/gc3pie directory to another location, e.g.:
```
mv $HOME/gc3pie $HOME/gc3pie.OLD
```
Reinstall GC3Pie using the quick-install script (top of this page).
Once you have verified that your new installation is working, you can remove the $HOME/gc3pie.OLD directory.

If instead you installed GC3Pie using the “manual installation” instructions, then the following steps will update GC3Pie to the latest version in the code repository:

cd to the directory containing the GC3Pie virtualenv; assuming it is named gc3pie as in the above installation instructions, you can issue the commands:
```
cd $HOME/gc3pie # use '/opt/gc3pie' if root
```
Activate the virtualenv:
```
source bin/activate
```
Upgrade the gc3pie source and run the setup.py script again:
```
cd src
svn up
env CC=gcc ./setup.py develop
```

Note: A major restructuring of the SVN repository took place in r1124 to r1126 (Feb. 15, 2011); if your sources are older than SVN r1124, these upgrade instructions will not work, and you must reinstall completely. You can check what version the SVN sources are, by running the svn info command in the src directory: watch out for the Revision: line.

MacOSX Installation¶

Installation on MacOSX machines is possible, however there are still a few issues. Please let us know on the GC3Pie users mailing-list <mailto:gc3pie@googlegroups.com> if you need MacOSX support.

Standard usage of the installation script (i.e., with no options) works, but you have to use curl since wget is not installed by default.
In order to install GC3Pie you will need to install XCode and, in some of the MacOSX versions, also the Command Line Tools for XCode
Options can only be given in the abbreviated one-letter form (e.g., -d); the long form (e.g., --directory) will not work.
The shellcmd backend of GC3Pie depends on the GNU time command, which is not installed on MacOSX by default. This means that with a standard MacOSX installation the shellcmd resource will not work. However:
- other resources, like pbs via ssh transport, will work.
- you can install the GNU time command either via MacPorts, Fink, Homebrew or from the this url. After installing it you don’t need to update your PATH environment variable, it’s enough to set the time_cmd option in your GC3Pie configuration file.
At the time of this writing, the ARC client libraries are not available on MacOSX, so ARC backends will not work.

HTML Documentation¶

HTML documentation for the GC3Libs programming interface can be read online at:

http://gc3pie.googlecode.com/svn/branches/2.0/gc3pie/docs/html/index.html

If you installed GC3Pie manually, or if you installed it using the install.sh script with the --develop option, you can also access a local copy of the documentation from the sources:

cd $HOME/gc3pie # or wherever the gc3pie virtualenv is installed
cd src/docs
make html

Note that you need the Python package Sphinx in order to build the documentation locally.