Troubleshooting GC3Pie

This page lists a number of errors and issues that you might run into, together with their solution. Please use the GC3Pie mailing list for further help and for any problem not reported here!

Each section covers a different Python error; the section is named after the error name appearing in the last line of the Python traceback. (See section What is a Python traceback? below)

What is a Python traceback?

A traceback is a long Python error message, detailing the call stack in the code that lead to a specific error condition.

Tracebacks always look like this one (the number of lines printed, the files involved and the actual error message will, of course, vary):

Traceback (most recent call last):
 File "/home/mpackard/gc3pie/bin/gsub", line 9, in <module>
   load_entry_point('gc3pie==1.0rc7', 'console_scripts', 'gsub')()
 File "/home/mpackard/gc3pie/lib/python2.5/site-packages/gc3pie-1.0rc7-py2.5.egg/gc3utils/frontend.py", line 137, in main
   import gc3utils.commands
 File "/home/mpackard/gc3pie/lib/python2.5/site-packages/gc3pie-1.0rc7-py2.5.egg/gc3utils/commands.py", line 31, in <module>
   import cli.app
 File "/home/mpackard/gc3pie/lib/python2.5/site-packages/pyCLI-2.0.2-py2.5.egg/cli/app.py", line 37, in <module>
   from cli.util import ifelse, ismethodof
 File "/home/mpackard/gc3pie/lib/python2.5/site-packages/pyCLI-2.0.2-py2.5.egg/cli/util.py", line 28, in <module>
   BaseStringIO = StringIO.StringIO
AttributeError: 'module' object has no attribute 'StringIO'

Let’s analyize how a traceback is formed, top to bottom.

A traceback is always started by the line:

Traceback (most recent call last):

Then follow a number of line pairs like this one:

File "/home/mpackard/gc3pie/lib/python2.5/site-packages/gc3pie-1.0rc7-py2.5.egg/gc3utils/frontend.py", line 137, in main
  import gc3utils.commands

The first line shows the file name and the line number where the program stopped; the second line displays the instruction that Python was executing when the error occurred. We shall always omit this part of the traceback in the listings below.

Finally, the traceback ends with the error message on the last line:

AttributeError: 'module' object has no attribute 'StringIO'

Just look up this error message in the section headers below; if you cannot find any relevant section, please write to the GC3Pie mailing list for help.

Common errors using GC3Pie

This section section lists Python errors that may happen when using GC3Pie; each section is named after the error name appearing in the last line of the Python traceback. (See section What is a Python traceback? above.)

If you get an error that is not listed here, please get in touch via the GC3Pie mailing list.

AttributeError: args

This error is signaled by a traceback like the following:

   Traceback (most recent call last):
     ...
     File "/opt/nordugrid/lib/python2.4/site-packages/arclib.py",
line 1146, in <lambda>
       __getattr__ = lambda self, name: _swig_getattr(self,
CertificateError, name)
     File "/opt/nordugrid/lib/python2.4/site-packages/arclib.py",
line 54, in _swig_getattr
       raise AttributeError(name)
   AttributeError: args

This is due to a version mismatch between the Python interpreter and the NorduGrid ARClib; it’s really the same issue as in ImportError: .../site-packages/_arclib.so: undefined symbol: Py_InitModule4, which see for a discussion and remedies.

AttributeError: module object has no attribute StringIO

This error:

Traceback (most recent call last):
 ...
 File "/home/mpackard/gc3pie/lib/python2.5/site-packages/pyCLI-2.0.2-py2.5.egg/cli/util.py",
line 28, in <module>
   BaseStringIO = StringIO.StringIO
AttributeError: 'module' object has no attribute 'StringIO'

is due to a conflicts of the pyCLI library (prior to version 2.0.3) and the Debian/Ubuntu package *python-stats*

There are three ways to get rid of the error:

  1. Uninstall the *python-stats* package <python-stats> (run the command apt-get remove python-stats as user root)
  2. Upgrade pyCLI to version 2.0.3 at least.
  3. Upgrade GC3Pie, which will force an upgrade of pyCLI.

DistributionNotFound

If you get this error:

Traceback (most recent call last):
    ...
pkg_resources.DistributionNotFound: gc3pie==1.0rc2

It usually means that you didn’t run source ../bin/activate; ./setup.py develop when upgrading GC3Pie.

Please re-do the steps in the GC3Pie Upgrade instructions to fix the error.

FTPControlError: Server responded: File unavailable.

Sample error traceback:

Traceback (most recent call last):
  ...
  File "/opt/nordugrid/lib/python2.5/site-packages/arclib.py", line 1405, in Download
    def Download(self, *args): return _arclib.FTPControl_Download(self, *args)
arclib.FTPControlError: Server responded: File unavailable.

This error happens when trying to view a non-existent file with the gtail command; see Issue 111 for details.

Once a job goes into RUNNING state, its standard output log is not made immediately available to the batch system. The only solution is to try again later; if the error persists, there might be some problem with the application that is not creating its output file as expected. This would needs further investigation and you are encouraged to report to the GC3Pie mailing list about it.

ImportError: .../site-packages/_arclib.so: undefined symbol: Py_InitModule4

This error:

Traceback (most recent call last):
  ...
  File ".../site-packages/arclib.py", line 17, in swig_import_helper
    import _arclib
ImportError: .../site-packages/_arclib.so: undefined symbol: Py_InitModule4

Is due to a mismatch of the Python version and the NorduGrid ARClib library: it occurs when the arclib library has been compiled for a different Python version than the one you’re using.

NorduGrid compiles their ARC/Python library with the default Python that comes with the Linux distribution; thus, if you are using a Python version different from the default, there’s a good chance that the ARC library will not work with that Python.

A rule of thumb to determine if you are using the default Python is to run the command which python. If this outputs anything different from /usr/bin/python or /bin/python, then you are using a custom Python version. You might want to ask your systems administrator to install the version of arclib appropriate for your Python, or to revert to the default Python.

Failing that, please ask on the GC3Pie mailing list: each case is subtly different and it’s impossible to give universally-valid instructions.

ImportError: No module named pstats

This error only occurs on Debian and Ubuntu GNU/Linux:

Traceback (most recent call last):
File ".../pyCLI-2.0.2-py2.6.egg/cli/util.py", line 19, in <module>
   import pstats
ImportError: No module named pstats

To solve the issue: install the *python-profiler* package <python-profiler>:

apt-get install python-profiler # as `root` user

NoResources: Could not initialize any computational resource - please check log and configuration file.

This error:

Traceback (most recent call last):
  ...
  File ".../src/gc3libs/core.py", line 150, in submit
    raise gc3libs.exceptions.NoResources("Could not initialize any computational resource"
gc3libs.exceptions.NoResources: Could not initialize any computational resource - please check log and configuration file.

can have two different causes:

  1. You didn’t create a configuration file, or you did not list any resource in it.
  2. Some other error prevented the resources from being initialized, or the configuration file from being properly read.

Terminate called after throwing an instance of LdapQueryError

This error has been reported after some days of running the gcodeml script with the ARC backend:

terminate called after throwing an instance of 'LdapQueryError'
 what():  Thread creation in ParallelLdapQueries failed
Aborted

We believe this is a bug in the ARC libraries, but not much information is known at the moment. So, if you see an instance of this, please get in touch via the GC3Pie mailing list.

ValueError: I/O operation on closed file

Sample error traceback (may be repeated multiple times over):

Traceback (most recent call last):
  File "/usr/lib/python2.5/logging/__init__.py", line 750, in emit
    self.stream.write(fs % msg)
ValueError: I/O operation on closed file

This is discussed in Issue 182; a fix have been committed to release 1.0, so if you are seeing this error, you are running a pre-release version of GC3Pie and should Upgrade.