Table of Contents
First Successful Job (15 Minutes)
This guide walks you through the basic HPC workflow:
- Connect to the HPC system
- Understand the difference between login and compute nodes
- Create and submit your first job
- Monitor job execution
- View job results
- Request CPU and memory resources
- Learn how project allocations work
- Run a multi-core Python application
- Verify CPU utilization
- Submit a GPU job
If you can complete this guide, you are ready to start running your own workloads on the HPC cluster.
Before You Start
Make sure that:
- You have completed Registration
- You have an active HPC account
- You have an SSH client installed
- You can connect to the HPC login node
Need access first?
Step 1: Connect to the Cluster
Open a terminal and connect to the HPC login node:
ssh username@hpc.mif.vu.lt
Replace username with your HPC username.
Verify that you are connected:
hostname pwd
You should see the hostname of the login node and your home directory path.
<note important> Your SLURM account is created automatically after your first successful login. This process may take up to 5 minutes.
Until the account is created, job submission may fail with account-related errors. </note>
For detailed instructions see:
Step 2: Understanding Login Nodes and Compute Nodes
Unlike a personal computer, computational workloads are not normally executed directly from the login node. Instead, jobs are submitted to SLURM, which schedules them on available compute nodes.
The login node is intended for:
- Editing files
- Compiling code
- Managing data
- Preparing and submitting jobs
The compute nodes are used for:
- CPU-intensive calculations
- Simulations
- Data analysis
- Machine learning and AI workloads
<note important> Do not run computationally intensive applications directly on the login node. Submit them through SLURM so they can be executed on the appropriate compute resources. </note>
Step 3: Create a Working Directory
Create a directory for this tutorial:
mkdir getting_started cd getting_started
Verify your location:
pwd
Step 4: Create Your First Job Script
Create a new file:
nano hello.sh
Paste the following content:
#!/bin/bash #SBATCH --job-name=hello #SBATCH --partition=main #SBATCH --output=hello.out #SBATCH --time=00:01:00 echo "Hello from HPC!" hostname date
Save the file and exit the editor.
Step 5: Submit the Job
Submit the script to the scheduler:
sbatch hello.sh
Example output:
Submitted batch job 123456
The returned number is your Job ID.
Step 6: Monitor the Job
Check the status of your jobs:
squeue -u $USER
When the job disappears from the queue, it has finished.
Step 7: View the Results
Display the output file:
cat hello.out
Example output:
Hello from HPC! compute01 Mon Nov 17 14:32:05 EET 2026
Congratulations! You have successfully executed your first HPC job.
Step 8: Request Resources for a Job
Real HPC workloads usually require specific CPU, memory, and runtime allocations. These resources are requested using SLURM directives.
Create a new script:
nano resources.sh
Paste the following content:
#!/bin/bash #SBATCH --job-name=resources-demo #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=4G #SBATCH --time=00:05:00 #SBATCH --output=resources.out echo "Job ID:" echo $SLURM_JOB_ID echo "" echo "Partition:" echo $SLURM_JOB_PARTITION echo "" echo "Allocated CPUs:" echo $SLURM_CPUS_PER_TASK echo "" echo "Finished."
Submit the job:
sbatch resources.sh
Understanding Resource Requests
| Directive | Description |
|---|---|
–partition=main | Run the job in the standard CPU partition |
–partition=gpu | Run the job in the GPU partition |
–cpus-per-task=4 | Request 4 CPU cores |
–mem=4G | Request 4 GB of RAM |
–time=00:05:00 | Request up to 5 minutes of runtime |
–output=resources.out | Save output to a file |
<note tip>
Most CPU workloads should use the main partition.
GPU workloads should use the gpu partition.
</note>
<note tip> If no runtime limit is specified, SLURM uses a default limit of 2 hours. However, specifying a realistic runtime is recommended because it helps the scheduler start jobs more efficiently. </note>
<note tip> Request only the resources you actually need. Smaller jobs generally start faster and help ensure fair sharing of cluster resources. </note>
Check the Job Output
Once the job completes:
cat resources.out
Step 9: Using Project Allocations
Some users are assigned project allocations that provide additional CPU, GPU, memory, or storage resources.
If your project administrator provides an allocation name, add it to your job script:
#!/bin/bash #SBATCH --job-name=project-demo #SBATCH --partition=main #SBATCH --account=alloc_xxxx_project #SBATCH --cpus-per-task=4 #SBATCH --mem=8G #SBATCH --time=01:00:00 hostname
The allocation name determines which project's resources are used to run the job.
<note important>
VU MIF users using the default allocation do NOT need to specify the –account parameter.
Only users associated with a project allocation should use the –account directive.
</note>
Example:
#SBATCH --account=alloc_xxxx_project
Step 10: Run a Multi-Core Python Job
Python is one of the most commonly used programming languages on HPC systems. This example performs a matrix multiplication, a common operation in scientific computing, machine learning, and data analysis.
Create a Python Script
Create a file named matrix.py:
nano matrix.py
Paste:
import os import numpy as np from time import perf_counter cpus = os.environ.get("SLURM_CPUS_PER_TASK", "unknown") print(f"CPU cores allocated by SLURM: {cpus}") size = 2000 print(f"Creating two {size}x{size} matrices...") a = np.random.rand(size, size) b = np.random.rand(size, size) start = perf_counter() c = a @ b end = perf_counter() print("Matrix multiplication completed") print(f"Result shape: {c.shape}") print(f"Execution time: {end - start:.2f} seconds")
Create a SLURM Job Script
Create a new file:
nano python-job.sh
Paste:
#!/bin/bash #SBATCH --job-name=python-demo #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=2G #SBATCH --time=00:05:00 #SBATCH --output=python-demo.out export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK export OPENBLAS_NUM_THREADS=$SLURM_CPUS_PER_TASK export MKL_NUM_THREADS=$SLURM_CPUS_PER_TASK python3 matrix.py
Submit the job:
sbatch python-job.sh
Check the Results
After the job finishes:
cat python-demo.out
Example output:
CPU cores allocated by SLURM: 4 Creating two 2000x2000 matrices... Matrix multiplication completed Result shape: (2000, 2000) Execution time: 1.82 seconds
Optional: Load a Python Module
Depending on the cluster software environment, a specific Python version may need to be loaded first:
module avail Python
Load one of the available versions shown by the system:
module load Python/<version>
Verify:
python3 --version
Step 11: Verify CPU Usage
While a job is running, you can monitor resource utilization with:
sstat -j JOBID.batch --format=JobID,AveCPU,AveRSS,MaxRSS
Example:
JobID AveCPU AveRSS MaxRSS ------------ ---------- ---------- ---------- 123456.batch 00:00:01 500M 620M
Where:
| Field | Description |
|---|---|
AveCPU | Average CPU time consumed by job tasks |
AveRSS | Average memory usage |
MaxRSS | Maximum memory usage |
You can identify your Job ID with:
squeue -u $USER
Compare Elapsed Time and CPU Time
After the job finishes:
sacct -j JOBID --format=JobID,AllocCPUS,Elapsed,CPUTime
Example:
JobID AllocCPUS Elapsed CPUTime ------------ ---------- ---------- ---------- 123456 4 00:00:02 00:00:07
| Field | Description |
|---|---|
AllocCPUS | Number of allocated CPU cores |
Elapsed | Wall-clock runtime (real time) |
CPUTime | Total CPU time used by all allocated CPU cores |
How to Interpret the Results
Single-threaded application:
AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:10
The application used approximately one CPU core, even though four cores were allocated.
Multi-threaded application:
AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:38
The application actively used multiple CPU cores.
In an ideal case, a fully utilized four-core job running for ten seconds would show:
Elapsed = 00:00:10 CPUTime = 00:00:40
<note tip>
A quick way to verify that your application is using multiple CPU cores is to compare CPUTime with Elapsed. If CPUTime is several times larger than Elapsed, the workload is utilizing multiple CPU cores concurrently.
</note>
<note tip> Requesting multiple CPU cores with:
#SBATCH –cpus-per-task=4
does not automatically make an application run in parallel. The application must also support multithreading and be configured to use the allocated cores. </note>
Step 12: Submit a GPU Job
To run a job on a GPU node:
nano gpu-demo.sh
Paste:
#!/bin/bash #SBATCH --job-name=gpu-demo #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=4 #SBATCH --mem=16G #SBATCH --time=01:00:00 nvidia-smi
Submit the job:
sbatch gpu-demo.sh
The command nvidia-smi displays information about the GPU allocated to the job.
Users working within a project allocation may also need:
#SBATCH --account=alloc_xxxx_project
Interactive GPU Session
To start an interactive shell on a GPU node:
srun -p gpu --gres=gpu:1 --pty $SHELL
After the session starts:
nvidia-smi
Interactive sessions are useful for testing software, debugging, and experimenting before submitting batch jobs.
Useful Commands
| Command | Purpose |
|---|---|
pwd | Show current directory |
ls | List files |
cd | Change directory |
sbatch job.sh | Submit a job |
squeue -u $USER | View your jobs |
sacct -j JOBID | View completed job information |
sstat -j JOBID.batch | Monitor resource usage of a running job |
scancel JOBID | Cancel a job |
cat file | Display file contents |
Typical Resource Requests
| Workload | Partition | CPUs | Memory | GPUs |
|---|---|---|---|---|
| Learning and testing | main | 1-2 | 1-4 GB | None |
| Data analysis | main | 4-8 | 8-32 GB | Optional |
| Scientific simulations | main | 8-64 | 16-128 GB | Optional |
| Machine learning and AI | gpu | 4-16 | 16-64 GB | 1 or more |
Next Steps
You now know how to:
- Connect to the HPC cluster
- Create job scripts
- Submit jobs
- Monitor job execution
- Verify resource utilization
- Run Python applications
- Use project allocations
- Submit GPU jobs
Continue with:
Common Problems
Permission denied
Verify that:
- Your account has been activated
- You are using the correct username
- Your SSH credentials are correct
sbatch: command not found
You are probably running the command on your local computer instead of the HPC login node.
Invalid account or account/partition combination specified
Possible causes:
- Your SLURM account is still being created.
- The specified allocation does not exist.
- You do not have permission to use the specified allocation.
Job remains in PENDING state
Resources may currently be unavailable.
Check your jobs:
squeue -u $USER
GPU job does not start
Verify that:
- The job uses
–partition=gpu - The job requests GPUs using
–gres=gpu:N - GPU resources are available
- Your allocation includes GPU access
Python command not found
Verify that Python is installed in the environment or load the appropriate Python module:
module avail Python
Where to Go Next
Recommended learning path:
<note> This guide was prepared and updated by the VU MIF HPC team with the assistance of Microsoft Copilot. All procedures and examples should be validated against the current HPC environment, software stack, and operational policies. </note>
