en:hpc
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
en:hpc [2022/04/14 13:18] – grikiete | en:hpc [2022/04/15 06:09] – [Software] grikiete | ||
---|---|---|---|
Line 3: | Line 3: | ||
A Distributed Computing Network (DCN) is a specially designed network of computers capable of running applications that can exchange data efficiently. | A Distributed Computing Network (DCN) is a specially designed network of computers capable of running applications that can exchange data efficiently. | ||
- | VU MIF PST consists of a supercomputer from the clusters (the first number is the actual and available amount ): | + | VU MIF PST consists of a supercomputer from the clusters (the first number is the actual and available amount): |
^Title ^Nodes ^CPU ^GPU ^RAM ^HDD ^Network ^Notes| | ^Title ^Nodes ^CPU ^GPU ^RAM ^HDD ^Network ^Notes| | ||
Line 10: | Line 10: | ||
^power | ^power | ||
- | Iš viso **40/ | + | Total **40/ |
- | Toliau tekste procesorius | + | The processor below = CPU = core - a single core of the processor |
+ | |||
+ | ====== Software ====== | ||
+ | |||
+ | In **main** and **gpu** partitions there are installed [[https:// | ||
+ | |||
+ | You can check the list of OS package with the command '' | ||
+ | |||
+ | With the command [[https:// | ||
+ | |||
+ | With singularity you can prepare your container, for example: | ||
+ | <code shell> | ||
+ | $ singularity build --sandbox /tmp/python docker:// | ||
+ | $ singularity exec -w /tmp/python pip install package | ||
+ | $ singularity build python.sif / | ||
+ | $ rm -rf / | ||
+ | </ | ||
+ | Similarly, you can use R, Julia or other containers that do not require root privileges to install packages. | ||
+ | |||
+ | If you want to add OS packages to the singularity container, you need root/ | ||
+ | <code shell> | ||
+ | $ singularity build --sandbox /tmp/python docker:// | ||
+ | $ cp / | ||
+ | $ fakeroot -l / | ||
+ | $ fakeroot -l / | ||
+ | $ fakeroot -l / | ||
+ | $ rm -rf / | ||
+ | $ singularity build python.sif / | ||
+ | $ rm -rf / | ||
+ | </ | ||
+ | |||
+ | There are ready-made scripts to run your **hadoop** tasks using the [[https:// | ||
+ | |||
+ | With [[https:// | ||
+ | |||
+ | <code shell> | ||
+ | $ python3.7 -m pip install --upgrade pip setuptools wheel | ||
+ | $ python3.7 -m pip install --ignore-installed batchspawner jupyterlab | ||
+ | </ | ||
+ | |||
+ | Alternatively, | ||
+ | <code shell> | ||
+ | #!/bin/sh | ||
+ | exec singularity exec --nv myjupyterlab.sif batchspawner-singleuser " | ||
+ | </ |
en/hpc.txt · Last modified: 2024/02/21 12:50 by rolnas