IT wiki

VU MIF STSC

User Tools

Site Tools


en:hpc

This is an old revision of the document!


Description of the Equipment

A High Performance Computing (HPC) is a specially designed network of computers capable of running applications that can exchange data efficiently.

VU MIF HPC consists of a supercomputer from the clusters (the first number is the actual and available amount):

Title Nodes CPU GPU RAM HDD Network Notes
main 35/36 48 0 384GiB 0 1Gbit/s, 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband CPU
gpu 3/3 40 8 512GB/32GB 7TB 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband CPU NVIDIA DGX-1
power 2/2 32 4 1024GB/32GB 1.8TB 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband IBM Power System AC922

Total 40/41 nodes, 1912 CPU cores with 17TB RAM, 32 GPU with 1TB RAM.

The processor below = CPU = core - a single core of the processor (with all hyperthreads if they are turned on).

Software

In main and gpu partitions there are installed Qlustar 11 OS. It is based on Ubuntu 18.04 LTS. In power partition there is installed Ubuntu 18.04 LTS.

You can check the list of OS package with the command dpkg -l (in login node hpc or in power nodes).

With the command singularity it is possible to make use of ready-made copies of container files in directories /apps/local/hpc, /apps/local/nvidia, /apps/local/intel, /apps/local/lang or to download from singularity and docker online repositories. You can also create your own singularity containers using the MIF cloud service.

You can prepare your container with singularity, for example:

$ singularity build --sandbox /tmp/python docker://python:3.8
$ singularity exec -w /tmp/python pip install package
$ singularity build python.sif /tmp/python
$ rm -rf /tmp/python

Similarly, you can use R, Julia or other containers that do not require root privileges to install packages.

If you want to add OS packages to the singularity container, you need root/superuser privileges. With fakeroot, we simulate them, and copy the required library libfakeroot-sysv.so into the container, for example:

$ singularity build --sandbox /tmp/python docker://ubuntu:18.04
$ cp /libfakeroot-sysv.so /tmp/python/
$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get update
$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get install python3.8 ...
$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get clean
$ rm -rf /tmp/python/libfakeroot-sysv.so /tmp/python/var/lib/apt/lists (you can clean up more of what you don't need)
$ singularity build python.sif /tmp/python
$ rm -rf /tmp/python

There are ready-made scripts to run your hadoop tasks using the Magpie set in the directory /apps/local/bigdata.

With JupyterHub you can run calculations with the python command line in a web browser and use the JupyterLab environment. If you install your own JupyterLab environment in your home directory, you need to install the additional batchspawner package - this will start your environment, for example:

$ python3.7 -m pip install --upgrade pip setuptools wheel
$ python3.7 -m pip install --ignore-installed batchspawner jupyterlab

Alternatively, you can use a container that you made via JupyterHub. In that container, you need to install the batchswapner and jupyterlab packages, and to create a script ~/.local/bin/batchspawner-singleuser with execution permissions (chmod +x ~/.local/bin/batchspawner-singleuser).

#!/bin/sh
exec singularity exec --nv myjupyterlab.sif batchspawner-singleuser "$@"

Registration

  • For VU MIF network users - HPC can be used without additional registration if the available resources are enough (monthly limit - 100 CPU-h and 6 GPU-h). Once this limit has been reached, you can request more by filling in ITOAC service request form.
  • For users of the VU computer network - you must fill in the ITOAC service request form to get access to MIF HPC. After the confirmation of your request, you must create your account in Waldur portal. More details read here.
  • For other users (non-members of the VU community) - you must fill in the ITOAC service request form to get access to MIF HPC. After the confirmation of your request, you must come to VU MIF Didlaukio str. 47, Room 302/304 to receive your login credentials. Please arranged the exact time by phone + 370 5219 5005. With these credentials you are able to create an account in Waldur portal. More details read here.

Prisijungimas

Norint prisijungti prie PST reikia naudoti SSH programas (ssh, putty, winscp, mobaxterm) ir Kerberos arba SSH raktų autentikaciją.

Jeigu naudojate Kerberos:

  • Prisijungiate į Linux aplinką VU MIF klasėje arba viešame terminale savo VU MIF naudotojo vardu ir slaptažodžiu
  • arba su ssh ar putty programa prisijungiate į uosis.mif.vu.lt savo VU MIF naudotojo vardu ir slaptažodžiu.
  • Su komanda klist patikrinkite ar turite galiojantį Kerberos raktą (ticket). Jei rakto nėra arba jo galiojimo laikas pasibaigęs, tai reikia įvykdyti komandą kinit.
  • Jungiatės į mazgą hpc su komanda ssh hpc (turi nereikalauti slaptažodžio).
  • Pirmą kartą prisijungus turi praeiti 5min, kad galėtumėte naudotis PST.

Jeigu naudojate SSH raktus (pvz: jei norite kopijuoti didelius failus):

  • Jei neturite SSH raktų, instrukciją kaip juos susikurti Windows aplinkoje rasite čia
  • Prieš naudodami šį būdą, reikia bent kartą prisijungti su Kerberos ir PST failų sistemoje susikurti ~/.ssh katalogą ir į jį įdėti savo ssh viešąjį raktą (OpenSSH formatu) į failą ~/.ssh/authorized_keys.
  • Prisijungiate su ssh, sftp, scp, putty, winscp ar kitomis ssh protokolą palaikančiomis programomis į hpc.mif.vu.lt su savo ssh privačiu raktu nurodydami savo VU MIF naudotojo vardą (turi nereikalauti prisijungimo slaptažodžio, bet gali reikalauti jūsų ssh privataus rakto slaptažodžio).

Pirmą kartą prisijungus jūs negalėsite paleisti SLURM užduočių per pirmąsias 5 minutes. Po to jums bus automatiškai sukurtas SLURM naudotojas ir priskirti resursų limitai.

en/hpc.1650367685.txt.gz · Last modified: 2022/04/19 11:28 by grikiete

Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki