Migrating from IBM LoadLeveller to PBS
To help users migrate from IBM's LoadLeveller (LL -- used by IU's Big Red and Libra clusters) to the PBS job submission system (used by the AVIDD cluster), the following tables show commands, environment variables and resource specifications of two batch systems side by side.
If you are a PBS user, looking for equivalent options on LL, then please refer to the Migrating from PBS to IBM LoadLeveller page.
If you're not familiar with either LL or PBS, and are new to Big Red, then please refer to the Working on Big Red primer.
Commonly used commands
| LoadLeveler | PBS | |
|---|---|---|
| Job submission | llsubmit <scriptfile> | qsub <scriptfile> |
| Job deletion | llcancel <job_id> | qdel <job_id> |
| Job status (for user) | llq -u <username> | qstat -u <username> |
| Extended Job status | llq -l <job_id> | qstat -f <job_id> |
| Hold a job temporarily | llhold <job_id> | qhold <job_id> |
| Resume job on hold | llhold -r <job_id> | qrls <job_id> |
| List of usable queues | llclass | qstat -Q |
| GUI for batch system | xloadl | xpbs |
Environment variables
| LoadLeveler | PBS | |
|---|---|---|
| Job ID | $LOADL_STEP_ID | $PBS_JOBID |
| Submission Directory | $LOADL_STEP_INITDIR | $PBS_O_WORKDIR |
| Processor list | $LOADL_PROCESSOR_LIST (See note below) |
$PBS_NODEFILE |
Important Note about $LOADL_PROCESSOR_LIST:
For jobs using more than 128 tasks (processors/cores),
the value of $LOADL_PROCESSOR_LIST will be set to NULL. If your job uses
more than 128 tasks, then you should run:
/opt/ibmll/LoadL/full/bin/ll_get_machine_list > /tmp/machinelist.$LOADL_STEP_IDand then use the temporary file (/tmp/machinelist.$LOADL_STEP_ID) created to get the list of machines assigned to your job.
Resource specifications
| LoadLeveler | PBS | |
|---|---|---|
| Queue | #@ class=<queue> | #PBS -q <queue> |
| Nodes | #@ node=<#> | #PBS -l nodes=<#> |
| Processors | #@ tasks_per_node=<#> | #PBS -l ppn=<#> |
| Wall clock limit | #@ wall_clock_limit=<hh:mm:ss> | #PBS -l walltime=<hh:mm:ss> |
| Standard output file | #@ output=<file> | #PBS -o <file> |
| Standard error | #@ error=<file> | #PBS -e <file> |
| Copy Environment | #@ environment=COPY_ALL | #PBS -V |
| Notification event | #@ notification=start| error|complete|never| always |
#PBS -m abe |
| Email address | #@ notify_user=<email> | #PBS -M <email> |
| Job name | #@ job_name=<name> | #PBS -N <name> |
| Job restart | #@ restart=<yes|no> | #PBS -r <y|n> |
| Job type | #@ job_type=<type> | N/A |
| Initial directory | #@ initialdir=<directory> | N/A |
| Node usage | #@ node_usage=not_shared | N/A? |
| Memory requirement | #@ requirements=(Memory >= NumKiloBytes) | N/A |
For more information, check out the LLGuide document for BigRed and Libra, and the PBS Guide for AVIDD.
Common Moab scheduler commands
The BigRed, AVIDD and Libra clusters all use the Moab job scheduler. Frequently used Moab scheduler commands include:
| Command | LoadLeveler & PBS |
|---|---|
| Show currently running/queued jobs | showq | less |
| Check a job's status | checkjob <jobid> or checkjob -v <jobid> (for details) |
| Show when your job might start | showstart <job_id> |
| Show fairshare information (priority, etc.) | diagnose -f | less |
| Check a nodes status | checknode <nodename> |
| Show current reservations | showres |
For more information on Moab, refer to the Moab Workload Manager User's Manual.
