Document Actions

NCSA Instructions for Running CIG Software

by Sue Kientz — last modified 2007-06-25 09:47 AM

The National Center for Supercomputing Applications (NCSA) at the University of Illinois, Urbana-Champaign, offers two job queues: dque (for general purpose) and debug (higher priority but maximum 30 minutes walltime).

See specifications of NCSA's hardware resources and other details at NCSA TeraGrid Resources.

CIG is able to offer small allocations of time at NCSA and a few other locations to get you started using CIG software; fill out the application to request an allocation.

Log in to NCSA

Once you receive an allocation login (via snail-mail package a few weeks after you get your portal login), you can log in to NCSA:

    $ ssh username@tg-login.ncsa.teragrid.org

Set up your environment

Before submitting batch jobs, make sure the permissions on your home directory and $HOME/.pbs_spool directory have execute permissions for other users. This is needed by PBS. When the job is running, PBS stores your stdout and stderr files temporarily in $HOME/.pbs_spool. When the job completes, it transfers them to the directory from which you submitted your job. Execute these commands once:
```
   $ mkdir $HOME/.pbs_spool
   $ chmod o+x $HOME $HOME/.pbs_spool 
```
TeraGrid uses SoftEnv to manage your software environment, e.g., the PATH and LD_LIBRARY_PATH environment variables. The setting is stored in the file ~/.soft. In order to run CitcomS, add the following lines to the beginning of your ~/.soft. Note that the order of the lines is important.
```
   CIGHOME = /usr/projects/CIG

   PATH += $CIGHOME/CitcomS/bin
   PATH += $CIGHOME/python-2.4.4/bin

   ### uncomment the next line if you want to compile CitcomS yourself
   #PATH += $CIGHOME/autotools/bin

   +phdf5-1.6.4-r1
   @teragrid
```
After modifying your ~/.soft, remember to "resoft" it to update your settings.
```
    $ resoft
```

Submit your job

Additional input parameters are needed for the batch scheduler. For example, adding these parameters to your .cfg input file will submit the job, named hello, to the "dque" queue with a maximum run-time of 2 hours.

    [CitcomS.job]
    queue = dque
    name = hello
    walltime = 2*hour

On NCSA, available queues include:

dque (for general purpose)
debug (higher priority and maximum 30 minutes walltime)

Additional queues for large or long jobs are available upon request.

More information about the queue structure can be found at the NCSA Mercury User Guide.

You can use "minute," "hour," and "day" to set the walltime.

Monitor your job

After you've submitted your job, you can monitor its status with the "qsat" command. You can view the status of the queue with the "showq" command. Again, more information can be found at the NCSA Mercury User Guide.

Sometimes you might need to remove a pending job from the queue or kill a running job:

    $ qdel      <jobid>        # Removes pending or running job.
    $ qsig -s 9 <jobid>        # Sends (sig)kill immediately to running job.

CIG

Sections