Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
fujitsu:hpcgateway:guides:internals:task_submission [2017/09/28 14:53]
127.0.0.1 external edit
fujitsu:hpcgateway:guides:internals:task_submission [2018/03/26 12:29] (current)
fujitsu
Line 1: Line 1:
 ====== Task submission mechanism ====== ====== Task submission mechanism ======
-{{  :fujitsu:​hpcgateway:​guides:​internals:​Torii_submission_mechanism.png?​direct&​800 ​ }}+ 
 +{{  ​.:torii_submission_mechanism.png?​direct&​800 ​ }}
  
 The picture illustrates the whole mechanism of the task submission: The picture illustrates the whole mechanism of the task submission:
-   * the forge of the scripts + 
-   ​* the submission //​as-the-user//​ +  ​* the forge of the scripts 
-   ​* the batch submission+  * the submission //​as-the-user//​ 
 +  * the batch submission
  
 ===== Forge the submission script ===== ===== Forge the submission script =====
Line 17: Line 19:
 The submission script is a standalone shell script that can be submitted to the batch scheduler. The submission script is a standalone shell script that can be submitted to the batch scheduler.
  
-The submission script is standalone, it does not have any dependency on another file (like the application script). \\ +The submission script is standalone, it does not have any dependency on another file (like the application script). \\ The submission script contains all the mechanisms to configure the environment for the application script and the application script itself.
-The submission script contains all the mechanisms to configure the environment for the application script and the application script itself+
- +
-The submission script is part of the scheduler configuration and it is stored on GridFS (ex: ''​clusters/​rnd01/​forge''​) \\ +
-See ''​script_execute''​ and ''​script_common''​ for the templates.+
  
 +The submission script is part of the scheduler configuration and it is stored on GridFS (ex: ''​clusters/​rnd01/​forge''​) \\ See ''​script_execute'' ​ and ''​script_common'' ​ for the templates.
  
 ==== Application script ==== ==== Application script ====
Line 31: Line 30:
  
 The application script is part of the application definition and it is stored on GridFS (ex: ''​applications/​675A865A765A65765A/​app_execute''​) The application script is part of the application definition and it is stored on GridFS (ex: ''​applications/​675A865A765A65765A/​app_execute''​)
- 
  
 ==== Task parameters and inputs ==== ==== Task parameters and inputs ====
Line 39: Line 37:
 The parameters and inputs are stored in the task JSON that it stored on the DB. The parameters and inputs are stored in the task JSON that it stored on the DB.
  
-The parameters and inputs are user to replace the markup that appear in the submission scripts and the application scripts (ex: ''​@@__HPCG_RUN_DIR__@@''​).+The parameters and inputs are user to replace the markup that appear in the submission scripts and the application scripts (ex: ''​@@__HPCG_RUN_DIR__ @@''​).
  
-    ​"​inputs":​ [ +<​code>​ 
-        +  ​"​inputs":​ [ 
-            "​autoCheck":​ false,  +      
-            "​description":​ "The time the test script will sleep before exiting",​  +          "​autoCheck":​ false, 
-            "​editable":​ true,  +          "​description":​ "The time the test script will sleep before exiting",​ 
-            "​hidden":​ false,  +          "​editable":​ true, 
-            "​label":​ "Sleep value",​  +          "​hidden":​ false, 
-            "​labelVisible":​ true,  +          "​label":​ "Sleep value",​ 
-            "​name":​ "​sleep",​  +          "​labelVisible":​ true, 
-            "​order":​ 1,  +          "​name":​ "​sleep",​ 
-            "​required":​ true,  +          "​order":​ 1, 
-            "​strict":​ false,  +          "​required":​ true, 
-            "​type":​ "​int",​  +          "​strict":​ false, 
-            "​valid":​ true,  +          "​type":​ "​int",​ 
-            "​value":​ 20 +          "​valid":​ true, 
-        },  +          "​value":​ 20 
-        +      }, 
-            "​autoCheck":​ false,  +      
-            "​description":​ "​Select input files that will be copied in the run directory",​  +          "​autoCheck":​ false, 
-            "​editable":​ true,  +          "​description":​ "​Select input files that will be copied in the run directory",​ 
-            "​files":​ [ +          "​editable":​ true, 
-                +          "​files":​ [ 
-                    "​action":​ "​COPY",​  +              
-                    "​mods":​ "​----------",​  +                  "​action":​ "​COPY",​ 
-                    "​path":​ "/​home/​hpcgadmin/​test",​  +                  "​mods":​ "​----------",​ 
-                    "​server":​ { +                  "​path":​ "/​home/​hpcgadmin/​test",​ 
-                        "​id":​ "​59b1302f6eea4a7e979b2c56",​  +                  "​server":​ { 
-                        "​name":​ "​rnd01"​ +                      "​id":​ "​59b1302f6eea4a7e979b2c56",​ 
-                    },  +                      "​name":​ "​rnd01"​ 
-                    "​size":​ 42 +                  }, 
-                +                  "​size":​ 42 
-            ],  +              
-            "​hidden":​ false,  +          ], 
-            "​label":​ "Input files",​  +          "​hidden":​ false, 
-            "​labelVisible":​ true,  +          "​label":​ "Input files",​ 
-            "​name":​ "​input_files",​  +          "​labelVisible":​ true, 
-            "​order":​ 2,  +          "​name":​ "​input_files",​ 
-            "​required":​ true,  +          "​order":​ 2, 
-            "​type":​ "​file",​  +          "​required":​ true, 
-            "​valid":​ true +          "​type":​ "​file",​ 
-        +          "​valid":​ true 
-    ]+      
 +  ] 
 +</​code>​
  
 ==== Cluster environment ==== ==== Cluster environment ====
Line 92: Line 92:
 ===== Submission mediator ===== ===== Submission mediator =====
  
-The submission mediator is a python script that does the effective submission of the submission script to the batch system. +The submission mediator is a python script that does the effective submission of the submission script to the batch system. The mediator waits for the completion of the submission and returns the job ID. 
-The mediator waits for the completion of the submission and returns the job ID.+