Differences

This shows you the differences between two versions of the page.

--- en:hpc [2022/04/14 13:18] – grikiete
+++ en:hpc [2022/04/14 13:33] – grikiete
@@ Line 3: / Line 3: @@
 A Distributed Computing Network (DCN) is a specially designed network of computers capable of running applications that can exchange data efficiently.
-VU MIF PST consists of a supercomputer from the clusters (the first number is the actual and available amount ):
+VU MIF PST consists of a supercomputer from the clusters (the first number is the actual and available amount):
 ^Title ^Nodes ^CPU ^GPU ^RAM        ^HDD    ^Network ^Notes|
@@ Line 10: / Line 10: @@
 ^power       ^2/2    ^32  ^4   ^1024GB/32GB ^1.8TB ^2x10Gbit/s, 4xEDR(100Gbit/s) infiniband ^[[https://www.ibm.com/products/power-systems-ac922|IBM Power System AC922]]|
-Iš viso	**40/41** mazgų, **1912** CPU cores su **17TB** RAM, **32** GPU su **1TB** RAM.
+Total	**40/41** nodes, **1912** CPU cores with **17TB** RAM, **32** GPU with **1TB** RAM.
-Toliau tekste procesorius = CPU = core - procesoriaus vienas branduolys (su visomis hypergijomis, jei jos yra įjungtos).
+The processor below = CPU = core - a single core of the processor (with all hyperthreads if they are turned on).
+====== Software ======
+**main** and **gpu** are installed [[https://docs.qlustar.com/Qlustar/11.0/HPCstack/hpc-user-manual.html|Qlustar 11]] operating system (OS) with Linux core. It is created Ubuntu 18.04 LTS based. **power** is installed Ubuntu 18.04 LTS.
+You can check the list of OS package with the command ''dpkg -l'' (in log in node **hpc** or in **power** nodes).
+Su komanda [[https://sylabs.io/guides/3.2/user-guide/index.html|singularity]] galima pasinaudoti jau paruoštomis konteinerių failų kopijomis kataloguose ''/apps/local/hpc'', ''/apps/local/nvidia'', ''/apps/local/intel'', ''/apps/local/lang'' arba pačiam atsisiųsti iš singularity ir docker internetinių katalogų. Pasinaudojus MIF cloud paslauga galima susikurti ir savo singularity konteinerius.
+Su singularity galite paruošti savo konteinerį, pvz:
+<code shell>
+$ singularity build --sandbox /tmp/python docker://python:3.8
+$ singularity exec -w /tmp/python pip install paketas
+$ singularity build python.sif /tmp/python
+$ rm -rf /tmp/python
+</code>
+Analogiškai galima pasinaudoti R, Julia ar kitais konteineriais, kuriuose paketams instaliuoti nereikia naudotojo root teisių.
+Jeigu norima papildyti singularity kontainerį OS paketais, tam reikia root/superuser teisių. Su fakeroot jas simuliuojame, o jam reikalingą biblioteką ''libfakeroot-sysv.so'' kopijuojame į konteinerį, pvz:
+<code shell>
+$ singularity build --sandbox /tmp/python docker://ubuntu:18.04
+$ cp /libfakeroot-sysv.so /tmp/python/
+$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get update
+$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get install python3.8 ...
+$ fakeroot -l /libfakeroot-sysv.so singularity exec -w /tmp/python apt-get clean
+$ rm -rf /tmp/python/libfakeroot-sysv.so /tmp/python/var/lib/apt/lists (galima ir daugiau išvalyti, ko nereikia)
+$ singularity build python.sif /tmp/python
+$ rm -rf /tmp/python
+</code>
+Kataloge ''/apps/local/bigdata'' yra paruošti scenarijai pasileisti savo **hadoop** užduotis pasinaudojant [[https://github.com/LLNL/magpie|Magpie]] rinkiniu.
+Su [[https://hpc.mif.vu.lt/hub/|JupyterHub]] galite interneto naršyklės pagalba vykdyti skaičiavimus su python komandų eilute ir pasinaudoti [[https://jupyter.org|JupyterLab]] aplinka. Jeigu savo namų kataloge instaliuosite savo JupyterLab aplinką, tai reikia instaliuoti papildomai ''batchspawner'' paketą - tada jums startuos jūsų aplinką, pvz:
+<code shell>
+$ python3.7 -m pip install --upgrade pip setuptools wheel
+$ python3.7 -m pip install --ignore-installed batchspawner jupyterlab
+</code>
+Taip pat jūs galite pasinaudoti savo pasidarytu konteineriu per JupyterHub. Tame konteineryje reikia instaliuoti ''batchswapner'' ir ''jupyterlab'' paketus bei sukurti script'ą ''~/.local/bin/batchspawner-singleuser'' su vykdymo teisėmis (''chmod +x ~/.local/bin/batchspawner-singleuser'')
+<code shell>
+#!/bin/sh
+exec singularity exec --nv myjupyterlab.sif batchspawner-singleuser "$@"
+</code>