====== First Successful Job (15 Minutes) ====== This guide walks you through the basic HPC workflow: - Connect to the HPC system - Understand the difference between login and compute nodes - Create and submit your first job - Monitor job execution - View job results - Request CPU and memory resources - Learn how project allocations work - Run a multi-core Python application - Verify CPU utilization - Submit a GPU job If you can complete this guide, you are ready to start running your own workloads on the HPC cluster. ===== Before You Start ===== Make sure that: * You have completed [[hpc:registration|Registration]] * You have an active HPC account * You have an SSH client installed * You can connect to the HPC login node Need access first? * [[hpc:registration|Register for HPC Access]] ===== Step 1: Connect to the Cluster ===== Open a terminal and connect to the HPC login node: ssh username@hpc.mif.vu.lt Replace ''username'' with your HPC username. Verify that you are connected: hostname pwd You should see the hostname of the login node and your home directory path. Your SLURM account is created automatically after your first successful login. This process may take up to 5 minutes. Until the account is created, job submission may fail with account-related errors. For detailed instructions see: * [[hpc:connection|Connection]] ===== Step 2: Understanding Login Nodes and Compute Nodes ===== Unlike a personal computer, computational workloads are not normally executed directly from the login node. Instead, jobs are submitted to SLURM, which schedules them on available compute nodes. The login node is intended for: * Editing files * Compiling code * Managing data * Preparing and submitting jobs The compute nodes are used for: * CPU-intensive calculations * Simulations * Data analysis * Machine learning and AI workloads Do not run computationally intensive applications directly on the login node. Submit them through SLURM so they can be executed on the appropriate compute resources. ===== Step 3: Create a Working Directory ===== Create a directory for this tutorial: mkdir getting_started cd getting_started Verify your location: pwd ===== Step 4: Create Your First Job Script ===== Create a new file: nano hello.sh Paste the following content: #!/bin/bash #SBATCH --job-name=hello #SBATCH --partition=main #SBATCH --output=hello.out #SBATCH --time=00:01:00 echo "Hello from HPC!" hostname date Save the file and exit the editor. ===== Step 5: Submit the Job ===== Submit the script to the scheduler: sbatch hello.sh Example output: Submitted batch job 123456 The returned number is your Job ID. ===== Step 6: Monitor the Job ===== Check the status of your jobs: squeue -u $USER When the job disappears from the queue, it has finished. ===== Step 7: View the Results ===== Display the output file: cat hello.out Example output: Hello from HPC! compute01 Mon Nov 17 14:32:05 EET 2026 Congratulations! You have successfully executed your first HPC job. ===== Step 8: Request Resources for a Job ===== Real HPC workloads usually require specific CPU, memory, and runtime allocations. These resources are requested using SLURM directives. Create a new script: nano resources.sh Paste the following content: #!/bin/bash #SBATCH --job-name=resources-demo #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=4G #SBATCH --time=00:05:00 #SBATCH --output=resources.out echo "Job ID:" echo $SLURM_JOB_ID echo "" echo "Partition:" echo $SLURM_JOB_PARTITION echo "" echo "Allocated CPUs:" echo $SLURM_CPUS_PER_TASK echo "" echo "Finished." Submit the job: sbatch resources.sh ===== Understanding Resource Requests ===== ^ Directive ^ Description ^ | ''--partition=main'' | Run the job in the standard CPU partition | | ''--partition=gpu'' | Run the job in the GPU partition | | ''--cpus-per-task=4'' | Request 4 CPU cores | | ''--mem=4G'' | Request 4 GB of RAM | | ''--time=00:05:00'' | Request up to 5 minutes of runtime | | ''--output=resources.out'' | Save output to a file | Most CPU workloads should use the ''main'' partition. GPU workloads should use the ''gpu'' partition. If no runtime limit is specified, SLURM uses a default limit of 2 hours. However, specifying a realistic runtime is recommended because it helps the scheduler start jobs more efficiently. Request only the resources you actually need. Smaller jobs generally start faster and help ensure fair sharing of cluster resources. ===== Check the Job Output ===== Once the job completes: cat resources.out ===== Step 9: Using Project Allocations ===== Some users are assigned project allocations that provide additional CPU, GPU, memory, or storage resources. If your project administrator provides an allocation name, add it to your job script: #!/bin/bash #SBATCH --job-name=project-demo #SBATCH --partition=main #SBATCH --account=alloc_xxxx_project #SBATCH --cpus-per-task=4 #SBATCH --mem=8G #SBATCH --time=01:00:00 hostname The allocation name determines which project's resources are used to run the job. VU MIF users using the default allocation do NOT need to specify the ''--account'' parameter. Only users associated with a project allocation should use the ''--account'' directive. Example: #SBATCH --account=alloc_xxxx_project ===== Step 10: Run a Multi-Core Python Job ===== Python is one of the most commonly used programming languages on HPC systems. This example performs a matrix multiplication, a common operation in scientific computing, machine learning, and data analysis. ==== Create a Python Script ==== Create a file named ''matrix.py'': nano matrix.py Paste: import os import numpy as np from time import perf_counter cpus = os.environ.get("SLURM_CPUS_PER_TASK", "unknown") print(f"CPU cores allocated by SLURM: {cpus}") size = 2000 print(f"Creating two {size}x{size} matrices...") a = np.random.rand(size, size) b = np.random.rand(size, size) start = perf_counter() c = a @ b end = perf_counter() print("Matrix multiplication completed") print(f"Result shape: {c.shape}") print(f"Execution time: {end - start:.2f} seconds") ==== Create a SLURM Job Script ==== Create a new file: nano python-job.sh Paste: #!/bin/bash #SBATCH --job-name=python-demo #SBATCH --partition=main #SBATCH --cpus-per-task=4 #SBATCH --mem=2G #SBATCH --time=00:05:00 #SBATCH --output=python-demo.out export OMP_NUM_THREADS=$SLURM_CPUS_PER_TASK export OPENBLAS_NUM_THREADS=$SLURM_CPUS_PER_TASK export MKL_NUM_THREADS=$SLURM_CPUS_PER_TASK python3 matrix.py Submit the job: sbatch python-job.sh ==== Check the Results ==== After the job finishes: cat python-demo.out Example output: CPU cores allocated by SLURM: 4 Creating two 2000x2000 matrices... Matrix multiplication completed Result shape: (2000, 2000) Execution time: 1.82 seconds ==== Optional: Load a Python Module ==== Depending on the cluster software environment, a specific Python version may need to be loaded first: module avail Python Load one of the available versions shown by the system: module load Python/ Verify: python3 --version ===== Step 11: Verify CPU Usage ===== While a job is running, you can monitor resource utilization with: sstat -j JOBID.batch --format=JobID,AveCPU,AveRSS,MaxRSS Example: JobID AveCPU AveRSS MaxRSS ------------ ---------- ---------- ---------- 123456.batch 00:00:01 500M 620M Where: ^ Field ^ Description ^ | ''AveCPU'' | Average CPU time consumed by job tasks | | ''AveRSS'' | Average memory usage | | ''MaxRSS'' | Maximum memory usage | You can identify your Job ID with: squeue -u $USER ==== Compare Elapsed Time and CPU Time ==== After the job finishes: sacct -j JOBID --format=JobID,AllocCPUS,Elapsed,CPUTime Example: JobID AllocCPUS Elapsed CPUTime ------------ ---------- ---------- ---------- 123456 4 00:00:02 00:00:07 ^ Field ^ Description ^ | ''AllocCPUS'' | Number of allocated CPU cores | | ''Elapsed'' | Wall-clock runtime (real time) | | ''CPUTime'' | Total CPU time used by all allocated CPU cores | ==== How to Interpret the Results ==== Single-threaded application: AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:10 The application used approximately one CPU core, even though four cores were allocated. Multi-threaded application: AllocCPUS = 4 Elapsed = 00:00:10 CPUTime = 00:00:38 The application actively used multiple CPU cores. In an ideal case, a fully utilized four-core job running for ten seconds would show: Elapsed = 00:00:10 CPUTime = 00:00:40 A quick way to verify that your application is using multiple CPU cores is to compare ''CPUTime'' with ''Elapsed''. If ''CPUTime'' is several times larger than ''Elapsed'', the workload is utilizing multiple CPU cores concurrently. Requesting multiple CPU cores with: #SBATCH --cpus-per-task=4 does not automatically make an application run in parallel. The application must also support multithreading and be configured to use the allocated cores. ===== Step 12: Submit a GPU Job ===== To run a job on a GPU node: nano gpu-demo.sh Paste: #!/bin/bash #SBATCH --job-name=gpu-demo #SBATCH --partition=gpu #SBATCH --gres=gpu:1 #SBATCH --cpus-per-task=4 #SBATCH --mem=16G #SBATCH --time=01:00:00 nvidia-smi Submit the job: sbatch gpu-demo.sh The command ''nvidia-smi'' displays information about the GPU allocated to the job. Users working within a project allocation may also need: #SBATCH --account=alloc_xxxx_project ==== Interactive GPU Session ==== To start an interactive shell on a GPU node: srun -p gpu --gres=gpu:1 --pty $SHELL After the session starts: nvidia-smi Interactive sessions are useful for testing software, debugging, and experimenting before submitting batch jobs. ===== Useful Commands ===== ^ Command ^ Purpose ^ | ''pwd'' | Show current directory | | ''ls'' | List files | | ''cd'' | Change directory | | ''sbatch job.sh'' | Submit a job | | ''squeue -u $USER'' | View your jobs | | ''sacct -j JOBID'' | View completed job information | | ''sstat -j JOBID.batch'' | Monitor resource usage of a running job | | ''scancel JOBID'' | Cancel a job | | ''cat file'' | Display file contents | ===== Typical Resource Requests ===== ^ Workload ^ Partition ^ CPUs ^ Memory ^ GPUs ^ | Learning and testing | main | 1-2 | 1-4 GB | None | | Data analysis | main | 4-8 | 8-32 GB | Optional | | Scientific simulations | main | 8-64 | 16-128 GB | Optional | | Machine learning and AI | gpu | 4-16 | 16-64 GB | 1 or more | ===== Next Steps ===== You now know how to: * Connect to the HPC cluster * Create job scripts * Submit jobs * Monitor job execution * Verify resource utilization * Run Python applications * Use project allocations * Submit GPU jobs Continue with: * [[hpc:lustre|Shared File System (Lustre)]] * [[hpc:slurm|Batch Job Execution (SLURM)]] * [[hpc:resources|Resource Allocations and Project Requests]] ===== Common Problems ===== ==== Permission denied ==== Verify that: * Your account has been activated * You are using the correct username * Your SSH credentials are correct ==== sbatch: command not found ==== You are probably running the command on your local computer instead of the HPC login node. ==== Invalid account or account/partition combination specified ==== Possible causes: * Your SLURM account is still being created. * The specified allocation does not exist. * You do not have permission to use the specified allocation. ==== Job remains in PENDING state ==== Resources may currently be unavailable. Check your jobs: squeue -u $USER ==== GPU job does not start ==== Verify that: * The job uses ''--partition=gpu'' * The job requests GPUs using ''--gres=gpu:N'' * GPU resources are available * Your allocation includes GPU access ==== Python command not found ==== Verify that Python is installed in the environment or load the appropriate Python module: module avail Python ===== Where to Go Next ===== Recommended learning path: - [[hpc:registration|Register for HPC Access]] - [[hpc:connection|Connection]] - [[hpc:getting_started|First Successful Job]] - [[hpc:lustre|Shared File System (Lustre)]] - [[hpc:slurm|Batch Job Execution (SLURM)]] - [[hpc:resources|Resource Allocations and Project Requests]] - [[hpc:waldur|Waldur User Guide]] ---- This guide was prepared and updated by the VU MIF HPC team with the assistance of Microsoft Copilot. All procedures and examples should be validated against the current HPC environment, software stack, and operational policies.