Differences

This shows you the differences between two versions of the page.

--- en:hpc [2022/07/04 09:48] – [Batch Processing of Tasks (SLURM)] grikiete
+++ en:hpc [2022/09/12 12:55] – [Registration] grikiete
@@ Line 60: / Line 60: @@
 ====== Registration ======
-  * **For VU MIF network users** - HPC can be used without additional registration if the available resources are enough (monthly limit - **100 CPU-h and 6 GPU-h**). Once this limit has been reached, you can request more by filling in [[https://forms.office.com/Pages/ResponsePage.aspx?id=ghrFgo1UykO8-b9LfrHQEidLsh79nRJAvOP_wV9sgmdUM0ZMR1FINFg3TzVaNlhDSEhUN1A3QTlVUC4u|ITOAC service request form]].
+  * **For VU MIF network users** - HPC can be used without additional registration if the available resources are enough (monthly limit - **800 CPU-h and 60 GPU-h**). Once this limit has been reached, you can request more by filling in [[https://forms.office.com/Pages/ResponsePage.aspx?id=ghrFgo1UykO8-b9LfrHQEidLsh79nRJAvOP_wV9sgmdUM0ZMR1FINFg3TzVaNlhDSEhUN1A3QTlVUC4u|ITOAC service request form]].
   * **For users of the VU computer network** - you must fill in the [[https://forms.office.com/Pages/ResponsePage.aspx?id=ghrFgo1UykO8-b9LfrHQEidLsh79nRJAvOP_wV9sgmdUM0ZMR1FINFg3TzVaNlhDSEhUN1A3QTlVUC4u|ITOAC service request form]] to get access to MIF HPC. After the confirmation of your request, you must create your account in [[https://hpc.mif.vu.lt|Waldur portal]]. More details read [[waldur|here]].
@@ Line 167: / Line 167: @@
 More about SLURM opportunities you can read [[https://slurm.schedmd.com/quickstart.html|Quick Start User Guide]].
+====== Interactive Tasks (SLURM) ======
+Interactive tasks can be done with the //srun// command:
+<code>
+$ srun --pty $SHELL
+</code>
+The above command will connect you to the compute node environment assigned to SLURM and allow you to directly run and debug programs on it.
+After the commands are done disconnect from the compute node with the command
+<code>
+$ exit
+</code>
+If you want to run graphical programs, you need to connect to **ssh -X** to **uosis.mif.vu.lt** and **hpc**:
+<code>
+$ ssh -X uosis.mif.vu.lt
+$ ssh -X hpc
+$ srun --pty $SHELL
+</code>
+In **power** cluster interactive tasks can be performed with
+<code>
+$ srun -p power --mpi=none --pty $SHELL
+</code>
+====== GPU Tasks (SLURM) ======
+To use GPU you need to specify additionally <code>--gres gpu:N</code> where N is desired GPU amount.
+With ''nvidia-smi'' in the task you can check the GPU amount that was dedicated.
+Example of an interactive task with 1 GPU:
+<code>
+$ srun -p gpu --gres gpu --pty $SHELL
+</code>
+====== Introduction to OpenMPI ======
+Ubuntu 18.04 LTS is the packet of **2.1.1** OpenMPI version.
+To use the newer version **4.0.1** you need to use
+<code>
+module load openmpi/4.0
+</code>
+before running MPI commands.
+===== MPI Compiling Programs =====
+An example of a simple MPI program is in the directory ''/scratch/lustre/test/openmpi''. **mpicc** (**mpiCC**, **mpif77**, **mpif90**, **mpifort**) is a framework for C (C++, F77, F90, Fortran) compilers that automatically adds the necessary **MPI** include and library files to the command line.
+<code>
+$ mpicc -o foo foo.c
+$ mpif77 -o foo foo.f
+$ mpif90 -o foo foo.f
+</code>
+===== Implementation of MPI Programmes =====
+MPI programs are started with **mpirun** or **mpiexec**. You can learn more about them with the **man mpirun** or **man mpiexec** command.
+A simple (SPMD) program can be started with the following mpirun command line.
+<code>
+$ mpirun foo
+</code>
+All allocated processors will be used according to the number ordered. If you want to use less, you can specify the -np quantity parameter in **mpirun**. It is not recommended to use less CPU than reserved for a longer time period, as unused CPUs remain free.
+**ATTENTION** It is strictly forbidden to use more CPU than you have reserved, as this may affect the performance of other tasks.
+Find more information on [[https://www.open-mpi.org|OpenMPI]].
+====== Task Efficiency ======
+  * Please use at least 50% of the ordered CPU quantity.
+  * Using more CPUs than ordered will not improve performance, as your task will only be able to use the CPUs ordered.
+  * If you use the ''--mem=X'' parameter, the task can reserve more **CPUs** in proportion to the amount of memory it wants. For example: if you order ''--mem=14000'' in the **main** queue, at least 2 CPUs will be reserved, unless other parameters specify more. If your task uses less than this, it will be an ineffective use of resources. In addition, it may run slower because it may use other memory than the executing CPU.
+====== The Limits of Resources ======
+If your tasks don't start because of **AssocGrpCPUMinutesLimit** or **AssocGrpGRESMinutes**, you need to check if there are any unused CPU/GPU resources left from your monthly limit.
+//The first way to see how much resources are used://
+<code>
+sreport -T cpu,mem,gres/gpu cluster AccountUtilizationByUser Start=0101 End=0131 User=USERNAME
+</code>
+Where the **USERNAME** - is your MIF user name. **Start** and **End** show the start and end days of the current month. You can specify them also by ''$(date +%m01)'' and ''$(date +%m31)''.
+**NOTE** Usage of resources is given in minutes, divide the number by 60 to get hours.
+//The second way to see how much resources are used://
+<code>
+sshare -l -A USERNAME_mif -p -o GrpTRESRaw,GrpTRESMins,TRESRunMins
+</code>
+Where **USERNAME** is your MIF user name. Or specify the account whose usage you want to see in **-A**. The data is also displayed in minutes:
+  * **GrpTRESRaw** - how much is used.
+  * **GrpTRESMins** - what is the limit.
+  * **GGRTRESRunMins** - the remaining resources for tasks that are still running.
+====== The Links ======
+  * [[waldur|HPC Waldur portal description]]
+  * [[https://mif.vu.lt/lt3/en/about/structure/it-research-center#ordering|ITOAC service ordering]]
+  * [[https://slurm.schedmd.com/quickstart.html|Quick Start User Guide (SLURM)]]
+  * [[https://docs.qlustar.com/Qlustar/11.0/HPCstack/hpc-user-manual.html|HPC User Manual (Qlustar)]]
+  * [[http://www.mcs.anl.gov/research/projects/mpi/|MPI standart]]
+  * [[pagalba@mif.vu.lt]] - registration of the **HPC** problems.