Requesting Resources

Overview

  • Tutorial: 30 min

    Objectives:

    • Learn how to write a PBS job script for Gadi.

    • Learn how to launch a job in Gadi.

  1. Which project are you using?

  2. Which job queue are you planning to use?

  3. How many CPU cores are required for your task?

  4. How many GPUs do you need?

  5. What is the estimated runtime of your program?

  6. Which modules are necessary to execute the program?

  7. What script or command will you use to run the program?

Batch jobs

A batch job is a non-interactive job submitted to the scheduler (like PBS or SLURM) to run at a later time. It executes your code or script without requiring your direct involvement during execution.

 1#!/bin/bash
 2
 3#PBS -P vp91
 4#PBS -q normal
 5#PBS -l ncpus=48
 6#PBS -l mem=10GB
 7#PBS -l storage=gdata/vp91+scratch/vp91
 8#PBS -l walltime=00:02:00
 9#PBS -N testScript
10
11module load python3/3.11.0
12module load papi/7.0.1
13
14. /g/data/vp91/Training-Venvs/intro-to-numba/bin/activate
15
16which python

 1#!/bin/bash
 2
 3#PBS -P vp91
 4#PBS -q gpuvolta
 5#PBS -l ncpus=12
 6#PBS -l ngpus=1
 7#PBS -l mem=10GB
 8#PBS -l storage=gdata/vp91+scratch/vp91
 9#PBS -l walltime=00:02:00
10#PBS -N gpuScript
11
12module load python3/3.11.0
13module load papi/7.0.1
14
15. /g/data/vp91/Training-Venvs/intro-to-numba/bin/activate
16
17nvidia-smi

  1. P - Gadi project (sometimes called account) to use

  2. q - Gadi queue to use

  3. ncpus - Total number of cores requested

  4. ngpus - Total number of GPUs requested

  5. mem - Total memory requested

  6. l - Total wall time for which the resources are provisioned

  7. N - Name of the job

For more PBS Directives please check the Gadi document and for more details on the different Gadi queues please check out the corresponding Gadi document .

All the Python code are available in the directory python/src` while all the job scripts are available in the directory python/jobScripts. To submit a job use the command

1qsub cpu.pbs

and to know the status of your job use the command

1qstat <jobid>

To know get the details about the job use the command

1qstat -swx <jobid>

Interactive Jobs

An interactive job allows you to interact directly with the HPC system and the job while it’s running. This means you have a command-line shell (e.g., terminal) on the compute node where you can run commands in real-time.

1qsub -I -q normal  -P vp91 -l walltime=00:10:00,ncpus=48,mem=10GB

Key Points

  1. Multiple PBS directives are available request a job.

  2. Gadi uses some custom directives.

  3. There are two modes to request a job - batched and interactive.