Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
fujitsu:hpcgateway:guides:internals:picoms [2018/03/26 13:32]
fujitsu
fujitsu:hpcgateway:guides:internals:picoms [2018/03/26 14:13]
fujitsu
Line 18: Line 18:
   * the template tasks it can use as model for the new tasks it,    * the template tasks it can use as model for the new tasks it, 
   * the tasks it has spawned previously ​   * the tasks it has spawned previously ​
- 
- 
-=== Runlog structure === 
  
 The runlog is the center of the dialog between Gateway and the picom script. The runlog is the center of the dialog between Gateway and the picom script.
Line 74: Line 71:
 </​code>​ </​code>​
  
 +==== Context ====
 +
 +The executions of the picom tasks and its sub-tasks is influenced by the //context// of the execution.
 +The context is keep the trace of the executions and it propagated to the sub-tasks of the picom task.
 +
 +=== Step and maxStep ===
 +
 +The //step// correspond to the different executions of the picom script.
 +
 +The step is limited by a field //maxStep// in the context (default is 100).
 +
 +=== Depth and maxDepth ===
 +
 +The //depth// is the depth of the task hierarchy.
 +The picom task that is created by the user has a depth 0 and its immediate children have a depth 1.
 +  ​
 +The depth is limited by a field //​maxDepth//​ in the context (default is 3).
 +
 +The depth and its limitation are crucial in the recursive picoms to avoid the picoms to run wild.
 +
 +===== Client API =====
 +
 +==== Submitting a Picom Task ====
 +
 +<​code>​
 +
 +</​code>​
 +
 +==== Creating a new Picom ====
 +
 +The code below is an example of Picom creation from the client API.
 +
 +<​code>​
 +# step 0 - define the picom script
 +def picom_body(ex):​
 +    """​
 +    The function defining the content of the Picom
 +    :param ex: the picom executor
 +    :return:
 +    """​
 +
 +    sleep = picom_task.get_input('​sleep'​)
 +    ​
 +    if ex.context.step > 5:
 +        ex.stop()
 +
 +    if not ex.tasks.last_tasks:​
 +        for i in range(0, 1):
 +            task = ex.tasks.new(template='​test task', name='​task ' + str(i), submit=True)
 +            task.set_input('​sleep',​ sleep)
 +            task.context.num = i
 +    else:
 +        i = ex.tasks.last_tasks[-1].context.num
 +        for last_task in ex.tasks.last_tasks:​
 +            task = ex.tasks.new(template=last_task,​ name='​task ' + str(i), submit=True)
 +            task.clear_dependencies()
 +            task.add_dependency(last_task)
 +
 +            task.context.num = i
 +            i += 1
 +    return
 +
 +    ​
 +t = Torii()
 +
 +# step 0 - create the picom to edit
 +name = '​test'​
 +version = '​0.1'​
 +picom = t.picoms.new(name=name,​ version=version)
 +
 +# define the script
 +picom.set_script(picom_body)
 +
 +# prepare the task template
 +application = t.applications.get(name='​Test App 001')
 +template = t.tasks.new(name = 'test task', description = 'wait for a while',​ application=application)
 +picom.templateTasks = [template]
 +
 +# define the icon
 +picom.icon = ICON
 +
 +# save the picom
 +picom.update()  ​
 +  ​
 +</​code>​
 +
 +There are two major elements in this example. ​
 +
 +First, the script that is set using a function reference. The following section explains how the function is transformed in a script.
 +
 +Second, we give a template task to the picom for the script to have a model for spawning new tasks. ​
 +
 +
 +=== Picom script ​ and picom executor===
 +
 +The code below correspond to the script generated by the client API in the former example.
 +The client API add to the function that was passed as argument a '​main'​ body and the creation of 
 +a //picom executor// that mimics the behavior of a Torii client.
 +
 +The idea is to have exactly the same syntax for the picom script than the one we would use in the client API.
 +In particular the picom executor offers a pseudo-task service (OfflineTaskService) that allows the picom to spawn new tasks.
 +
 +In fact the picom executor read the context, templates and picom descriptions from the runlog and automatically generate the 
 +Json files of the tasks-to-spawned in the step directory.
 +
 +Keep in mind that the picom execution is completely offline. ​
 +The executor cannot get any data from Gateway except the data that was written in the runlog. ​
 +
 +<​code>​
 +from torii import PicomExecutor
 +
 +def picom_body(ex):​
 +    """​
 +    The function defining the content of the Picom
 +    :param ex: the picom executor
 +    :return:
 +    """​
 +
 +    sleep = picom_task.get_input('​sleep'​)
 +    ​
 +    if ex.context.step > 5:
 +        ex.stop()
 +
 +    if not ex.tasks.last_tasks:​
 +        for i in range(0, 1):
 +            task = ex.tasks.new(template='​test task', name='​task ' + str(i), submit=True)
 +            task.set_input('​sleep',​ sleep)
 +            task.context.num = i
 +    else:
 +        i = ex.tasks.last_tasks[-1].context.num
 +        for last_task in ex.tasks.last_tasks:​
 +            task = ex.tasks.new(template=last_task,​ name='​task ' + str(i), submit=True)
 +            task.clear_dependencies()
 +            task.add_dependency(last_task)
 +
 +            task.context.num = i
 +            i += 1
 +    return
 +
 +
 +if __name__ == "​__main__":​
 +
 +    ex =  PicomExecutor()
 +    ​
 +    picom_body(ex)
 +
 +</​code>​