en:hpc
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
en:hpc [2022/07/05 10:40] – [GPU Tasks(SLURM)] grikiete | en:hpc [2024/02/21 12:45] – [Singularity] rolnas | ||
---|---|---|---|
Line 16: | Line 16: | ||
====== Software ====== | ====== Software ====== | ||
- | In **main** and **gpu** partitions there are installed [[https:// | + | In **main** and **gpu** partitions there are installed [[https:// |
You can check the list of OS package with the command '' | You can check the list of OS package with the command '' | ||
+ | |||
+ | ===== Singularity ===== | ||
With the command [[https:// | With the command [[https:// | ||
Line 42: | Line 44: | ||
$ rm -rf /tmp/python | $ rm -rf /tmp/python | ||
</ | </ | ||
+ | |||
+ | ==== Hadoop ==== | ||
There are ready-made scripts to run your **hadoop** tasks using the [[https:// | There are ready-made scripts to run your **hadoop** tasks using the [[https:// | ||
+ | |||
+ | ==== JupyterHub ==== | ||
With [[https:// | With [[https:// | ||
Line 60: | Line 66: | ||
====== Registration ====== | ====== Registration ====== | ||
- | * **For VU MIF network users** - HPC can be used without additional registration if the available resources are enough (monthly limit - **100 CPU-h and 6 GPU-h**). Once this limit has been reached, you can request more by filling in [[https:// | + | * **For VU MIF network users** - HPC can be used without additional registration if the available resources are enough (monthly limit - **500 CPU-h and 60 GPU-h**). Once this limit has been reached, you can request more by filling in [[https:// |
* **For users of the VU computer network** - you must fill in the [[https:// | * **For users of the VU computer network** - you must fill in the [[https:// | ||
Line 77: | Line 83: | ||
If **SSH keys** are used (e.g. if you need to copy big files): | If **SSH keys** are used (e.g. if you need to copy big files): | ||
- | * If you don't have SSH keys, you can find instructions on how to create them in a Windows environment **[[duk: | + | * If you don't have SSH keys, you can find instructions on how to create them in a Windows environment **[[en:duk: |
* | * | ||
* | * | ||
Line 197: | Line 203: | ||
</ | </ | ||
- | ====== GPU Tasks(SLURM) ====== | + | ====== GPU Tasks (SLURM) ====== |
To use GPU you need to specify additionally < | To use GPU you need to specify additionally < | ||
Line 208: | Line 214: | ||
</ | </ | ||
- | ====== | + | ====== |
- | Ubuntu 18.04 LTS yra **2.1.1** | + | Ubuntu 18.04 LTS is the packet of **2.1.1** OpenMPI |
- | Norint pasinaudoti naujesne | + | To use the newer version |
< | < | ||
module load openmpi/4.0 | module load openmpi/4.0 | ||
</ | </ | ||
- | prieš vykdant | + | before running |
- | ===== MPI programų kompiliavimas | + | ===== MPI Compiling Programs |
- | Paprastos | + | An example of a simple |
< | < | ||
Line 226: | Line 232: | ||
$ mpif90 -o foo foo.f | $ mpif90 -o foo foo.f | ||
</ | </ | ||
- | ===== MPI programų vykdymas | + | ===== Implementation of MPI Programmes |
- | MPI programos startuojamos su **mpirun** | + | MPI programs are started with **mpirun** |
- | Paprasta | + | A simple |
< | < | ||
Line 236: | Line 242: | ||
</ | </ | ||
- | Tai naudos visus paskirtus procesorius, | + | All allocated processors will be used according to the number ordered. If you want to use less, you can specify the -np quantity parameter in **mpirun**. |
- | Daugiau apie instaliuotą | + | **ATTENTION** It is strictly forbidden to use more CPU than you have reserved, as this may affect the performance of other tasks. |
- | ====== Užduočių efektyvumas ====== | + | Find more information on [[https:// |
- | * Prašome išnaudoti ne mažiau 50% užsakyto CPU kiekio. | + | ====== Task Efficiency ====== |
- | * Naudoti daugiau CPU, nei užsakyta, nepadidins efektyvumo, nes jūsų užduotis galės naudoti tik tiek CPU, kiek buvo užsakyta. | + | |
- | * Jeigu naudosite parametrą '' | + | |
- | ====== Resursų limitai ====== | + | * Please use at least 50% of the ordered CPU quantity. |
+ | * Using more CPUs than ordered will not improve performance, | ||
+ | * If you use the '' | ||
- | Jeigu jūsų užduotys nestartuoja su priežastimi **AssocGrpCPUMinutesLimit** arba **AssocGrpGRESMinutes**, | + | ====== The Limits of Resources ====== |
- | tai pasitikrinkite ar užduotims dar liko neišnaudotų CPU/GPU resursų iš (mėnesio) limito. | + | |
- | Peržiūrėti kiek išnaudota resursų | + | If your tasks don't start because of **AssocGrpCPUMinutesLimit** or **AssocGrpGRESMinutes**, |
+ | |||
+ | //The first way to see how much resources are used:// | ||
< | < | ||
Line 257: | Line 264: | ||
</ | </ | ||
- | kur **USERNAME** | + | Where the **USERNAME** |
- | Atkreipkite dėmesį, kad naudojimas pateikiamas minutėmis, o į valandas konvertuoti reikia dalinant iš 60. | + | **NOTE** Usage of resources is given in minutes, divide the number by 60 to get hours. |
- | Kitas būdas pažiūrėti limitus ir jų išnaudojimą | + | //The second way to see how much resources are used:// |
< | < | ||
Line 267: | Line 274: | ||
</ | </ | ||
- | kur **USERNAME** MIF naudotojo vardas. Arba parametre | + | Where **USERNAME** |
+ | * **GrpTRESRaw** - how much is used. | ||
+ | * **GrpTRESMins** - what is the limit. | ||
+ | * **GGRTRESRunMins** - the remaining resources for tasks that are still running. | ||
+ | |||
+ | ====== The Links ====== | ||
+ | * [[waldur|HPC Waldur portal description]] | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | * [[http:// | ||
+ | * [[pagalba@mif.vu.lt]] - registration of the **HPC** problems. | ||
en/hpc.txt · Last modified: 2024/02/21 12:50 by rolnas