en:hpc
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
en:hpc [2022/04/19 11:44] – [Login] grikiete | en:hpc [2022/07/04 08:32] – [Batch Processing of Tasks (SLURM)] grikiete | ||
---|---|---|---|
Line 66: | Line 66: | ||
* **For other users (non-members of the VU community)** - you must fill in the [[https:// | * **For other users (non-members of the VU community)** - you must fill in the [[https:// | ||
- | ====== | + | ====== |
You need to use SSH applications (ssh, putty, winscp, mobaxterm) and Kerberos or SSH key authentication to connect to **HPC**. | You need to use SSH applications (ssh, putty, winscp, mobaxterm) and Kerberos or SSH key authentication to connect to **HPC**. | ||
Line 72: | Line 72: | ||
If **Kerberos** is used: | If **Kerberos** is used: | ||
- | * Log in to the Linux environment in a VU MIF classroom or public terminal with your VU MIF username and password; | + | * Log in to the Linux environment in a VU MIF classroom or public terminal with your VU MIF username and password or login to **uosis.mif.vu.lt** with your VU MIF username and password using **ssh** or **putty**. |
- | * or login to **uosis.mif.vu.lt** with your VU MIF username and password using ssh or putty. | + | |
* Check if you have a valid Kerberos key (ticket) with the **klist** command. If the key is not available or has expired, the **kinit** command must be used. | * Check if you have a valid Kerberos key (ticket) with the **klist** command. If the key is not available or has expired, the **kinit** command must be used. | ||
* Connect to the **hpc** node with the command **ssh hpc** (password must not be required). | * Connect to the **hpc** node with the command **ssh hpc** (password must not be required). | ||
- | * The **first time** you log in, you must wait **5 minutes** and then you can start to use HPC. | ||
- | Jeigu naudojate | + | If **SSH keys** are used (e.g. if you need to copy big files): |
- | * Jei neturite | + | * If you don't have SSH keys, you can find instructions on how to create them in a Windows |
- | * Prieš naudodami šį būdą, reikia bent kartą prisijungti su Kerberos | + | * Before you can use this method, you need to log in with Kerberos |
- | * Prisijungiate su **ssh**, **sftp**, **scp**, **putty**, **winscp** | + | * Connect with **ssh**, **sftp**, **scp**, **putty**, **winscp** |
+ | |||
+ | The **first time** you connect, you **will not** be able to run **SLURM jobs** for the first **5 minutes**. After that, SLURM account will be created. | ||
+ | |||
+ | ====== Lustre - Shared File System ====== | ||
+ | |||
+ | VU MIF HPC shared file system is available in the directory ''/ | ||
+ | |||
+ | The system creates directory ''/ | ||
+ | |||
+ | The files in this file system are equally accessible on all compute nodes and on the **hpc** node. | ||
+ | |||
+ | Please use these directories only for their purpose and clean them up after calculations. | ||
+ | |||
+ | ====== HPC Partition ====== | ||
+ | |||
+ | ^Partition ^Time limit ^RAM | ||
+ | ^main | ||
+ | ^gpu ^48h | ||
+ | ^power | ||
+ | |||
+ | The time limit for tasks is **2h** in all partitions if it has not been specified. The table shows the maximum time limit. | ||
+ | |||
+ | The **RAM** column gives the amount of RAM allocated to each reserved **CPU** core. | ||
+ | |||
+ | ====== Batch Processing of Tasks (SLURM) ====== | ||
+ | |||
+ | To use computing resources of the HPC, you need to create task scenarios (sh or csh). | ||
+ | |||
+ | Example: | ||
+ | |||
+ | <code shell mpi-test-job.sh> | ||
+ | # | ||
+ | #SBATCH -p main | ||
+ | #SBATCH -n4 | ||
+ | module load openmpi | ||
+ | mpicc -o mpi-test mpi-test.c | ||
+ | mpirun mpi-test | ||
+ | </ | ||
+ | |||
+ | Pateikus paraišką ITAPC ir gavus teigimą sprendimą, reikia susikurti naudotoją https:// | ||
+ | |||
+ | After submission and confirmation of your application to the ITOAC services, you need to create a user at https:// | ||
+ | |||
+ | <code shell mpi-test-job.sh> | ||
+ | # | ||
+ | #SBATCH --account=alloc_xxxx_projektas | ||
+ | #SBATCH -p main | ||
+ | #SBATCH -n4 | ||
+ | #SBATCH --time=minutes | ||
+ | module load openmpi | ||
+ | mpicc -o mpi-test mpi-test.c | ||
+ | mpirun mpi-test | ||
+ | </ | ||
+ | |||
+ | |||
+ | Jame kaip specialūs komentarai yra nurodymai užduočių vykdytojui. | ||
+ | |||
+ | -p short - į kokią eilę siųsti | ||
+ | |||
+ | -n4 - kiek procesorių rezervuoti (**PASTABA: | ||
+ | |||
+ | Užduoties pradinis einamasis katalogas yra dabartinis katalogas (**pwd**) | ||
+ | |||
+ | Suformuotą scenarijų siunčiame su komanda sbatch | ||
+ | |||
+ | '' | ||
+ | |||
+ | kuri gražina pateiktos užduoties numerį **JOBID**. | ||
+ | |||
+ | Laukiančios arba vykdomos užduoties būseną galima sužinoti su komanda squeue | ||
+ | |||
+ | '' | ||
+ | |||
+ | Su komanda scancel galima nutraukti užduoties vykdymą arba išimti ją iš eilės | ||
+ | |||
+ | '' | ||
+ | |||
+ | Jeigu neatsimenate savo užduočių **JOBID**, tai galite pasižiūrėti su komanda **squeue** | ||
+ | |||
+ | '' | ||
+ | |||
+ | Užbaigtų užduočių **squeue** jau neberodo. | ||
+ | |||
+ | Jeigu nurodytas procesorių kiekis nėra pasiekiamas, | ||
+ | |||
+ | Vykdomos užduoties išvestis (**output**) yra įrašoma į failą **slurm-JOBID.out**. Jei nenurodyta kitaip, tai ir klaidų (error) išvestis yra įrašoma į tą patį failą. Failų vardus galima pakeisti su komandos **sbatch** parametrais -o (nurodyti išvesties failą) ir -e (nurodyti klaidų failą). | ||
+ | |||
+ | Daugiau apie SLURM galimybes galite paskaityti [[https:// | ||
- | Pirmą kartą prisijungus jūs **negalėsite** paleisti **SLURM užduočių** per pirmąsias **5 minutes**. Po to jums bus automatiškai sukurtas **SLURM naudotojas** ir priskirti **resursų limitai**. | ||
en/hpc.txt · Last modified: 2024/02/21 12:50 by rolnas