The VU MIF HPC cluster uses SLURM to manage computing resources and execute jobs.
If you have used a personal computer before, think of SLURM as a workload manager. Instead of running applications directly, you ask SLURM for resources (CPU cores, memory, GPUs, and runtime), and SLURM finds a suitable compute node to run your job.
SLURM is responsible for:
Most HPC work starts with creating a job script and submitting it using:
sbatch job.sh
A typical HPC workflow looks like this:
sbatch<note tip> Unlike a personal computer, computational workloads are normally not executed directly on the login node.
Use SLURM to run calculations on compute nodes. </note>
The VU MIF HPC cluster currently provides:
| Partition | Purpose | Maximum Runtime |
|---|---|---|
main | General-purpose CPU workloads | 7 days |
gpu | GPU-accelerated workloads | 48 hours |
<note>
This guide focuses on the main and gpu partitions, which are used by most users.
The power partition is intended for specialized IBM Power9 workloads and is not covered here.
</note>
<note tip>
Most users should start with the main partition.
Use the gpu partition only when your application requires NVIDIA GPUs.
</note>
<note tip> If no runtime limit is specified, SLURM uses a default runtime limit of 2 hours. </note>
Create a file named hello.sh:
nano hello.sh
Paste:
#!/bin/bash #SBATCH --job-name=hello #SBATCH --partition=main #SBATCH --output=hello.out #SBATCH --time=00:01:00 echo "Hello from HPC!" hostname date
Submit the job:
sbatch hello.sh
Example output:
Submitted batch job 123456
Many applications require CPU cores, memory, or GPUs.
Example:
#!/bin/bash #SBATCH --job-name=myjob #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=8G #SBATCH --time=01:00:00 python3 script.py
| Directive | Description |
|---|---|
–partition=main | Use the CPU partition |
–partition=gpu | Use the GPU partition |
–cpus-per-task=4 | Request 4 CPU cores |
–mem=8G | Request 8 GB RAM |
–time=01:00:00 | Request 1 hour runtime |
–output=file.out | Save output to a file |
–job-name=myjob | Assign a name to the job |
Users working in Waldur-managed projects may need to specify an allocation account:
#SBATCH --account=alloc_xxxx_project
Example:
#!/bin/bash #SBATCH --account=alloc_xxxx_project #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=8G #SBATCH --time=01:00:00 python3 script.py
<note important>
VU MIF users using the default allocation do not need to specify the –account parameter.
Only users assigned to a project allocation should use –account.
</note>
Submit a job:
sbatch job.sh
SLURM returns a Job ID:
Submitted batch job 123456
The Job ID uniquely identifies your job.
Show your running and pending jobs:
squeue -u $USER
Example:
JOBID PARTITION NAME USER ST TIME NODES NODELIST(REASON) 123456 main test user R 00:10 1 compute07
A job may remain in PENDING state when:
Show detailed status:
squeue -j JOBID
Display information about completed jobs:
sacct -j JOBID
More detailed output:
sacct -j JOBID --format=JobID,State,Elapsed,AllocCPUS
Example:
JobID State Elapsed AllocCPUS ------------ ---------- ------------ ---------- 123456 COMPLETED 00:01:05 4
While a job is running:
sstat -j JOBID.batch --format=JobID,AveCPU,AveRSS,MaxRSS
Example:
JobID AveCPU AveRSS MaxRSS ------------ ---------- ---------- ---------- 123456.batch 00:00:01 500M 620M
Where:
| Field | Description |
|---|---|
AveCPU | Average CPU time consumed by job tasks |
AveRSS | Average memory usage |
MaxRSS | Maximum memory usage |
After a job finishes, you can compare elapsed runtime with total CPU usage:
sacct -j JOBID --format=JobID,AllocCPUS,Elapsed,CPUTime
Example:
JobID AllocCPUS Elapsed CPUTime ------------ ---------- ---------- ---------- 123456 4 00:00:10 00:00:38
| Field | Description |
|---|---|
AllocCPUS | Number of allocated CPU cores |
Elapsed | Real (wall-clock) runtime |
CPUTime | Total CPU time used by all allocated CPU cores |
AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:10
The application only used approximately one CPU core, even though four cores were allocated.
AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:38
The application actively used multiple CPU cores.
A fully utilized four-core job running for ten seconds would produce approximately:
AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:40
<note tip>
A quick way to verify that your application is using multiple CPU cores is to compare CPUTime with Elapsed.
If CPUTime is several times larger than Elapsed, the workload is utilizing multiple CPU cores concurrently.
</note>
Cancel a running or pending job:
scancel JOBID
Example:
scancel 123456
Sometimes it is useful to work interactively instead of submitting a batch script.
Start an interactive shell:
srun --pty $SHELL
When finished:
exit
To reserve GPU resources:
#!/bin/bash #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=4 #SBATCH --mem=16G #SBATCH --time=01:00:00 nvidia-smi
The parameter –gres=gpu:1 requests one GPU.
The command:
nvidia-smi
displays information about the GPU allocated to the job.
Users working within a project allocation may also need:
#SBATCH --account=alloc_xxxx_project
| Command | Purpose |
|---|---|
sbatch job.sh | Submit a batch job |
squeue -u $USER | View your jobs |
sacct -j JOBID | View completed jobs |
sstat -j JOBID.batch | Monitor running jobs |
scancel JOBID | Cancel a job |
sinfo | Show cluster partitions |
The typical workflow for running a job is:
sbatchsqueuesacct or sstat<note tip> Start with small jobs and modest resource requests. Once your application works correctly, increase CPU, memory, or GPU allocations as needed. </note>
main for most workloads.gpu only when your application requires NVIDIA GPUs.Most VU MIF users do not.
Only users assigned to a project allocation should specify:
#SBATCH --account=alloc_xxxx_project
SLURM applies a default runtime limit of 2 hours.
The most common reasons are:
Check:
squeue -j JOBID
By default:
slurm-JOBID.out
in the directory where the job was submitted.
You can change this using:
#SBATCH --output=myjob.out
Possible causes:
Check:
squeue -j JOBID
Possible causes:
Check the output file:
cat slurm-JOBID.out
Also review:
sacct -j JOBID
Verify:
–partition=gpu is used–gres=gpu:N is specifiedNow that you know how to submit and monitor jobs, learn about resource quotas, project allocations, and Waldur-managed projects:
You may also revisit:
<note> This guide was prepared and updated by the VU MIF HPC team with the assistance of Microsoft Copilot. All procedures and examples should be validated against the current HPC environment, software stack, and operational policies. </note>