Document Actions

SDSC Instructions for Running CIG Software

by Sue Kientz — last modified 2007-06-25 09:47 AM

The San Diego Supercomputer Center (SDSC) at University of California, San Diego, offers one queue for all batch jobs, each of which is limited to an 18-hour wallclock time.

See specifications of SDSC's hardware resources and other details at SDSC User Support.

CIG is able to offer small allocations of time at SDSC and a few other locations to get you started using CIG software; fill out the application to request an allocation.

Log in to SDSC

Once you receive an allocation login (via snail-mail package a few weeks after you get your portal login), you can log in to SDSC:

    $ ssh username@tg-login.sdsc.teragrid.org

Set up your environment

Before submitting batch jobs, make sure the permissions on your home directory and $HOME/.pbs_spool directory have execute permissions for other users. This is needed by PBS. When the job is running, PBS stores your stdout and stderr files temporarily in $HOME/.pbs_spool. When the job completes, it transfers them to the directory from which you submitted your job. Execute these commands once:
```
   $ mkdir $HOME/.pbs_spool
   $ chmod o+x $HOME $HOME/.pbs_spool 
```
TeraGrid uses SoftEnv to manage your software environment, like the PATH and LD_LIBRARY_PATH environment variables. The setting is stored in the file ~/.soft. In order to run CitcomS, add the following lines into your ~/.soft. Note that the order of the lines is important.
```
    CIGHOME = /gpfs/projects/tg_community/CIG

    PATH += $CIGHOME/CitcomS/bin
    PATH += /usr/local/apps/python-2.4.3/bin

    INTEL_HOME = /usr/local/apps/intel/compiler8
    +intel8
    +mpich-gm-1.2.5..10-intel-r2
    +phdf5-1.6.2-r2

    @teragrid-basic 
```
After modifying your ~/.soft, remember to "resoft" it to update your settings.
```
    $ resoft
```

Submit your job

Additional input parameters are needed for the batch scheduler. For example, adding these parameters to your .cfg input file will submit a job, named hello, to the "dque" queue with a maximum run-time of 2 hours.

    [CitcomS.job]
    queue = dque
    name = hello
    walltime = 2*hour

On SDSC, only one queue "dque" is normally available for all batch jobs, each of which are limited to an 18-hour wallclock time. You can use "minute" or "hour" to set the walltime.

Monitor your job

After you've submitted your job, you can monitor its status with the "qstat" command. You can view the status of the queue with the "showq" command. More information can be found at the SDSC IA-64 Cluster User Guide.

Sometimes you might need to remove a pending job from the queue or kill a running job:

    $ qdel      <jobid>        # Removes pending or running job.
    $ qsig -s 9 <jobid>        # Sends (sig)kill immediately to running job.

CIG

Sections