This is an old revision of the document!
Description of the Equipment
A Distributed Computing Network (DCN) is a specially designed network of computers capable of running applications that can exchange data efficiently.
VU MIF PST consists of a supercomputer from the clusters (the first number is the actual and available amount):
Title | Nodes | CPU | GPU | RAM | HDD | Network | Notes |
---|---|---|---|---|---|---|---|
main | 35/36 | 48 | 0 | 384GiB | 0 | 1Gbit/s, 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband | CPU |
gpu | 3/3 | 40 | 8 | 512GB/32GB | 7TB | 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband | CPU NVIDIA DGX-1 |
power | 2/2 | 32 | 4 | 1024GB/32GB | 1.8TB | 2x10Gbit/s, 4xEDR(100Gbit/s) infiniband | IBM Power System AC922 |
Total 40/41 nodes, 1912 CPU cores with 17TB RAM, 32 GPU with 1TB RAM.
The processor below = CPU = core - a single core of the processor (with all hyperthreads if they are turned on).
Software
main and gpu are installed Qlustar 11 operating system (OS) with Linux core. It is created Ubuntu 18.04 LTS based. power is installed Ubuntu 18.04 LTS.
You can check the list of OS package with the command dpkg -l
(in log in node hpc or in power nodes).
With the command singularity it is possible to make use of ready-made copies of container files in directories /apps/local/hpc
, /apps/local/nvidia
, /apps/local/intel
, /apps/local/lang
or to download from singularity and docker online directories. You can also create your own singularity containers using the MIF cloud service.
With singularity you can prepare your container, for example:
$ singularity build --sandbox /tmp/python docker://python:3.8 $ singularity exec -w /tmp/python pip install package $ singularity build python.sif /tmp/python $ rm -rf /tmp/python
Similarly, you can use R, Julia or other containers that do not require root privileges to install packages.
If you want to add OS packages to the singularity container, you need root/superuser privileges. With fakeroot, we simulate them, and copy the required library libfakeroot-sysv.so
into the container, for example:
$ singularity build --sandbox /tmp/python docker://ubuntu:18.04 $ cp /libfakeroot-sysv.so /tmp/python/ $ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get update $ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get install python3.8 ... $ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get clean $ rm -rf /tmp/python/libfakeroot-sysv.so /tmp/python/var/lib/apt/lists (you can clean up more of what you don't need) $ singularity build python.sif /tmp/python $ rm -rf /tmp/python
There are ready-made scripts to run your hadoop tasks using the Magpie set in the directory /apps/local/bigdata
.
With JupyterHub galite interneto naršyklės pagalba vykdyti skaičiavimus su python komandų eilute ir pasinaudoti JupyterLab aplinka. Jeigu savo namų kataloge instaliuosite savo JupyterLab aplinką, tai reikia instaliuoti papildomai batchspawner
paketą - tada jums startuos jūsų aplinką, pvz:
$ python3.7 -m pip install --upgrade pip setuptools wheel $ python3.7 -m pip install --ignore-installed batchspawner jupyterlab
Taip pat jūs galite pasinaudoti savo pasidarytu konteineriu per JupyterHub. Tame konteineryje reikia instaliuoti batchswapner
ir jupyterlab
paketus bei sukurti script'ą ~/.local/bin/batchspawner-singleuser
su vykdymo teisėmis (chmod +x ~/.local/bin/batchspawner-singleuser
)
#!/bin/sh exec singularity exec --nv myjupyterlab.sif batchspawner-singleuser "$@"