Add a new rundir on a fast file system

At installation step user default rundir is mapped to /home/hpcgadmin/<user_name>/path_date/path_task_id.

In some use cases, the default rundir is not the best location for computation as it is using user home based on NFS mounted filesystem. A preferable location can be a fast filesystem mounted both on head node and on all compute nodes. This file system can be for example FEFS or Lustre or GPFS …etc…

Solution: Create a new dedicated rundir mapped to /fefs/hpcgadmin/<user_name>/rundir/path_date/path_task_id.

Step by step:

Assuming that FEFS is already mounted as /fefs both on head node and compute node.

1. Extract HPC Gateway configuration from Mongo database (follow this link)

2. Move to the extracted configuration into the clusters collection:

[hpcgadmin@myhead hpcgadmin]$ cd /opt/hpcg/repo/conf/current/clusters

3. Edit your clusters.json file as following from your head node on Linux (to avoid Windows CR+LF)

hpcgadmin@myhead clusters]$ cat 5989b02e051bc0eb6848a08b.json
{
    "_id": {
        "$oid": "5989b02e051bc0eb6848a08b"
    },
    "cost": 0,
    "description": "Cluster tst02",
    "headnode": {
        "id": "5989b02bec7ff6242653763b",
        "name": "tst02"
    },
    "modificationDate": 1504000544,
    "name": "tst02",
    "projects": [],
    "rundirs": [
        {
            "description": "The rundir will be automatically created in the HPC Gateway home file system (/home/hpcgadmin/hpcgateway)",
            "label": "Automatic creation in HPC Gateway home",
            "num": 2,
            "root": "/home/hpcgadmin/hpcgateway/rundir/@@__HPCG_USER__@@/@@__HPCG_PATH_DATE__@@/task_@@__HPCG_TASK_NUM__@@"
        },
        {
            "description": "The rundir will be automatically created in the FEFS fast file system (/fefs/hpcgadmin/hpcgateway)",
            "label": "Automatic creation in FEFS fast file system",
            "num": 1,
            "root": "/fefs/hpcgadmin/hpcgateway/rundir/@@__HPCG_USER__@@/@@__HPCG_PATH_DATE__@@/task_@@__HPCG_TASK_NUM__@@"
        }
    ],
    "scheduler": {
.....

You should copy the exiting rundir block and paste it after a comma. You can then modify it according to your needs:

  • description is used for tip when you fly other the label
  • label is used as label/title to add a new option line
  • order is unique sequentiel number to organize rundir display order
  • root is the real location of the run directory, you can here use several internal variables to have for your use case /home_local/my_user/rundir/path_date/task_num

4. Inject HPC Gateway configuration from Mongo database (follow this link)

5. Create new rundir root

[hpcgadmin@myhead hpcgadmin]$ mkdir -p /fefs/hpcgadmin/hpcgateway/rundir
[hpcgadmin@myhead hpcgadmin]$ chmod a+rwxt /fefs/hpcgadmin/hpcgateway/rundir

6. Create Mount point runnable to authorize exploration of new created rundir Open AdminDasboard and add a mountpoint as following (don't forget to check runnable):

7. Open the HPC Gateway Web interface (logout/login if connected) and check that a new line has appeared in the left bottom rundir panel choices of jobsubmiter: