administration

administration:beegfs_client

2021-01-25T14:05:02+02:00

====== BeeGFS Client Setup ====== * ''apt install beegfs-helperd beegfs-client'' * Change '/etc/beegfs/beegfs-client.conf': sysMgmtdHost = bmanage-cp * ''mkdir -p /services/scratch'' * Change '/etc/beegfs/beegfs-mounts.conf': /services/scratch/ /etc/beegfs/beegfs-client.conf * ''systemctl restart beegfs-helperd beegfs-client'' * Finally, you need to make sure that beegfs is pruned in ''/etc/updatedb.conf'': ''sed -i '$ s/\"$/\ beegfs\"/' /etc/updatedb.conf'' …

administration:emergency_oar_restart

2020-04-17T12:42:48+02:00

[[administration:index|Administration]] ====== Emergency OAR Restart ====== Sometimes OAR locks up completely. No new jobs are scheduled, and existing jobs stay in the "Waiting" state even after being killed. When this happens, the next thing you have to try (on edgar) is to run the following: - systemctl stop oar-server - systemctl stop postgresql / systemctl stop pgsql - open htop, F6 Filter "Almighty", send SIGKILL to running instances - systemctl restart postgresql - system…

administration:gpu_cluster_doc

2020-11-13T17:05:07+02:00

[[administration:index|Back to Administration]] ====== GPU cluster documentation ====== ===== Basics ===== Currently the GPU cluster is ran as a separate set of resources accessible from the **edgar** machine. **edgar** runs an **OAR scheduler**. The command **oarnodesetting** is the entry point for most operations, such as:


oarnodesetting -h gpuhost38 -s Alive
# WARNING: kills existing jobs, does NOT reschedule idempotent
oarnodesetting -h gpuhost38 -s Absent

# no new jobs can be s…



administration:index
2021-08-09T17:25:20+02:00
====== Administration ======

This page covers topics related to machine administration.
Some pages are only accessible to [[:system_administrators|System Administrators]].

  * **Introduction**
    * [[administration:recap|Quick recap]]
  * **Setup**
    * [[administration:os_deployment:index|Installing new machines]]
    * [[administration:new_user_setup|New User setup]]
    * [[administration:using_docker|Set up Docker]]
    * [[administration:beegfs_client|BeeGFS Client Setup]]
    * [[admin…


administration:new_user_setup
2021-01-09T12:45:44+02:00
====== New User setup ======

Below are described some steps to setup a new user account.

===== Default folders =====

Here is the command to create the default folders for a given user when no physical connexion can be made to the Thoth subnetwork.

   * ''su ''
   * ''cd /home/''
   * ''xdg-user-dirs-update''

===== Remote SSH keys setup =====

When a new member '''' starts remotely, admins should grant them remote ssh access to ''''. The procedure is d…


administration:nvidia_driver_reinstall
2020-11-17T14:51:00+02:00
====== Updating the NVIDIA drivers ======


Before (re)installing a (new) driver, do ''init 3''. Post-installation, do ''init 5''.

You can check the installed packages (''ii'') using ''dpkg --list | grep nvidia''

You can see on the [[http://edgar/monika|GPU cluster status page]] which node may be failing because of a NVIDIA driver.

===== Available options =====

There are 4 options available to fix a failing node. The first option is always the preferred one.

  - NVIDIA driver reinstallation…


administration:raid_setup
2019-12-09T14:43:25+02:00
====== Tutorial on how to set up a RAID array of two disks (scratch) ======

This tutorial intend to serve as a walk-through for setting up a RAID array of two disks in the team's desktop machines. This RAID array will work as a **scratch** in your computer.

First, you should check that disks (HW) are correctly connected to the computer. Once you check that the disks are connected, you could check that they are recognized in your machine by running fdisk -l. You should be able to locate two add…


administration:recap
2020-11-17T11:19:42+02:00
====== Quick Recap for System Administrators ======

Here is a quick summary of our role as system administrators.We are responsible for the team's fleet of desktop machines, the GPU cluster, as well as the huge amount of data stored in the different scratches.

We have several tasks that can be summarized as follows: setup, monitoring, fixing and ordering.

===== Setup =====

  * Whenever a new member joins the team, we assign him a desktop machine, based on the availability (See machines.ods i…


administration:underclock_nvidia_headless
2019-06-06T17:21:50+02:00
Back to Administration



Force X server even without monitors:
nvidia-xconfig -a --allow-empty-initial-configuration --cool-bits=31 --use-display-device=“DFP-0” --connected-monitor=“DFP-0”



administration:using_docker
2018-07-26T15:30:31+02:00
DOCKER

Here are some informations about how to install and configure Docker, so that it can be run by users on their desktop machines without potential safety issues.

Installation

You can check this page to see how to install Docker on our Ubuntu machines:


administration:web_servers
2019-09-02T15:19:02+02:00
Web servers

There are two web-facing servers in Thoth.

The first is the public webpage “”. It is a virtual machine hosted by the MI. The content is drawn from the following NFS mount:

/home/wwwlear

The second web server is “pascal”. Is it physically in Jakob's office (armoire).