Screencast Video Tutorials


Individual applications - batch and interactive - are primarily utilised through the Gateway Application Management tool. This section illustrates how to use embedded applications through several common packages used in the HPC community.

  • OpenFOAM to solve a computational fluid dynamic (CFD) problem.
  • ParaView to visualise the result of the preceding OpenFOAM calculation.
  • DL_POLY Classic to solve a molecular dynamics (MD) problem.
  • VMD to visualise the result of the DL_POLY calculation.

—-

OpenFOAM Main

The Gateway at the FSE data centre includes an application named OpenFOAM Main. This runs the OpenFOAM application using 3 stages:

  1. Prolog: Runs commands provided by the user in serial mode only. Normally used to create the mesh and set parameters and characteristics of the model to be solved. The mesh decomposition is done implicitly by the application.
  2. Execute: Runs the solver(s) specified by the user. These executables are run in parallel mode.
  3. Epilog: For automated post-processing, such as reconstructing the decomposed solution directories into a single result directory.


The video for this tutorial is located here. Right-click to open in a new tab.

The full list of inputs are given in the following table.

Application OpenFOAM Main
Version 0.3.1
Location of all initial input directories ssf:/work/shared/OpenFOAM/tutorials/city/0
ssf:/work/shared/OpenFOAM/tutorials/city/constant
ssf:/work/shared/OpenFOAM/tutorials/city/system
Control start time of the simulation firstTime
Simulation end time 400
Time step of the simulation 1
Decomposition time range 0
Clean previous time directories if found True
OpenFOAM script for prolog phase surfaceFeatureExtract
blockMesh
rm -r 400
snappyHexMesh -overwrite
OpenFOAM script for execute phase simpleFoam -parallel
OpenFOAM script for epilog phase reconstructPar -latestTime

ParaView

ParaView is a graphical application used for data analysis and visualisation, and is widely used in the CAE (Computer-Aided Engineering) market.

The video for this tutorial is located here. Right-click to open in a new tab.

The full list of inputs are given in the following table.

Application ParaView
Version 0.1
Set dimensions of application display <optional>
Paraview python startup script <see below>
Name of a data file <can be left blank>

ParaView can run a Python script at startup to perform some initial setup. The following script will read in the result files and create an initial image. A more complete script can be found under the tutorial directory as shown in the video.

#### import the simple module from the paraview
from paraview.simple import *

# open OpenFOAM initialisation file
datafoam = OpenFOAMReader(FileName='./data.foam')

# get animation scene
GetAnimationScene().GoToLast()


Follow the keystrokes in the video to see how to adjust the displayed results. Left-click and drag the mouse in the 3D image panel to change the viewing angle.


DL_POLY Classic

 Under development

The video for this tutorial is located here. Right-click to open in a new tab.


VMD

 Under development

The video for this tutorial is located here. Right-click to open in a new tab.



The following sections walk through four different ways to submit a batch application in Gateway.

  1. Run directly from a shell terminal using the command line interface.
  2. Run with a Gateway submit command at the CLI within the shell terminal.
  3. Run the script from the Job Submitter tool in the Gateway web desktop.
  4. Create and run a Gateway Application Method from the web desktop.

This progression is illustrated using the DL_POLY Classic molecular dynamics application.


Running from the terminal

At its simplest Gateway can be used to run applications directly from a shell terminal on the server (cluster head node, login node, etc.). With the Remote Terminal add-on a user can open a direct shell session on the available server. Users who prefer to continue working with a direct Linux command line can continue to work this way with the following benefits from Gateway:

  • Single authentication to the web portal
  • Implicit login to the remote server
  • Multiple windows on the web desktop
  • File upload/download through the Gateway File Explorer

The video for this tutorial is located here. Right-click to open in a new tab.


Running with Gateway command line submission

The identical application job script used in the first stage above can be directly submitted with a command provided in the Gateway environment.

  • Task can now be followed in the Gateway Task Monitor in addition to the batch job monitor
  • Task is recorded in the Gateway database for analytics
source /opt/hpcg/core/etc/profile.sh

$HPCG_HOME/core/bin/batch/hpcg_submit.py --hpcg_name=run_from_gsub job.sh

The video for this tutorial is located here. Right-click to open in a new tab.


Run application with Gateway Job Submitter

Gateway includes a tool named Job Submitter that can be used to run any user-defined script.

  • The user script no longer needs to include the batch header. Batch control is done through the Execution Environment area of the Job Submitter panel.
  • Existing scripts can be uploaded from the server or client desktop.
  • Job profiles can be saved to preserve different initialisations of the same basic script and batch settings.
  • All jobs run are stored by Gateway and can be monitored with its Task Monitor tool.

The video for this tutorial is located here. Right-click to open in a new tab.


Create and run a Gateway application method

Gateway Application Management provides a catalogue of embedded application methods. These methods bring a wide range of benefits:

  • End-users no longer work with scripts directly, making task execution more robust and traceable.
  • The simplified and purpose-built input profile panel allows more users to be able to prepare and run workloads with these applications.
  • Standardise on validated procedures across the organisation.

The video for this tutorial is located here. Right-click to open in a new tab.

The following instructions describe the steps within this video.

  1. Open the original Linux command line script. This will be used as the basis of the Gateway app development.
  2. Open the Gateway Application Editor. Set general descriptive entries in the Main section and save to create app.
  3. Move to Inputs tab. Two input variables are relevant from the initial script.
    Files are a required input. Gateway can be automatically transfer or link these into the eventual run directory.
    Steps allows the user to easily change the simulation duration. Although this is not defined as required a default value will be required. Otherwise the script as-is will not correctly modify the CONTROL file. A maximum value is also defined to limit an overall run duration.
  4. Outputs are not used currently.
  5. The Prolog phase is created and activated to enable automated ingest of the input files. The script is executed as a short serial job prior to the main Execute phase.
  6. The Execute phase is initiated by adding the whole initial script. We then modify this to adapt to the Gateway app mechanism.
    Remove the batch header as this is built automatically from the app submission profile.
    There is a specific Environment panel to define the settings for different machines while keeping the same prolog/exec/epilog script logic. The list of available systems is determined by the local Gateway community. We will only use one cluster named 'ssf'. Export the variable name for the DL_POLY executable so that it is usable within the execute phase script.
  7. Return to Execute phase.
    Remove the environment settings which have been moved as above.
    Remove the preparation of the run directory as this is done automaticaly by Gateway. You can still create and select the run directory in the Gateway file explorer. You can retain directory manipulation within the apps scripts, though note that all files transferred in by the Gateway will go to the declared run directory only.
    Remove the input file transfer commands.
  8. The original script had a hard-coded value for the MPI parallel count. This should be generalised to use the value defined in the batch app execution environment. The variable HPCG_BATCH_MPI_PROCS is used for this purpose.
  9. Save the application.
  10. Run the application.
    Give a name for this job.
    Select the cluster to use. Only one is available since the Environment was activated only for the cluster 'ssf'.
    Select the input files. For this model we need 4 inputs as seen in the original script.
    Change the number of iterations.
    Set the run directory.
    Define the the number of nodes and cores to use. This requires the Advanced settings on this cluster, though this is site-dependent.
  11. Name this job profile and save for future traceability and re-use.
  12. Submit the job.
  13. Track the job in the Task Monitor.
    Notice that there are two batch jobs - one for prolog and one for execute. Prolog only transfers in the files since we did not give it any script content. Check this is the file explorer by opening the run directory.
    Refresh the File Explorer and open OUTPUT to see simulation progress. Observe the settings provided for parallelism and step count.
  14. When REVCON restart file is present then the simulation has completed. Check OUTPUT again.
  15. Go back to the editor. Add an icon image and save.
  16. Open the Application Management. Find the new application.
    This will still need to be released for other users to run.


Add a Data Monitor to an Application Method

The Data Monitor is a feature that allows for dynamic tracking of particular application output data. The Monitor is one of the four phases of applications methods that are createdwithin Gateway using the Application Editormonitor script is added as one of the phases of the application method in the Application Editor.

This tutorial extends the method that was embedded in an earlier exercise.

The video for this tutorial is located here. Right-click to open in a new tab.



Defining and running a Process Map

The video for this tutorial is located here. Right-click to open in a new tab.



Accelerated remote visualisation with Fujitsu RVEC

Fujitsu Gateway includes and add-on function for accelerated remote visualisation named RVEC. This is a client-server system using a Fujitsu proprietary compression protocol to transfer the complete remote desktop image to the screen of a client device. It directly addresses the challenges of effectively using highly-technical 3D graphical applications across a network, particularly 3D CAD and CAE (computer-aided engineering) tools.

Remote graphical systems hosting an RVEC software server are defined and published within the Gateway RVEC Management add-on.

The video for this tutorial is located here. Right-click to open in a new tab.


Collaborating with RVEC

One of the distinctive functions of RVEC is the ability to establish a collaborative session with other users.

  • Once a RVEC session is open the user first enables collaboration mode.
  • Other users then select Join collaboration in their RVEC Management panel.
  • The initiator of the session will then see a prompt to allow access from those users. Access may be active or passive (Read only).

The video for this tutorial is located here. Right-click to open in a new tab.