Changes between Version 3 and Version 4 of ComputeStartExample7

Timestamp:

2010-11-11T14:58:23+01:00 (14 years ago)

Author:

george

Comment:

--

ComputeStartExample7

@@ -1 +1 @@
 = Job Monitoring =
 
-Job execution is monitored by reading a log file of the job. A job consists of a number of steps which consists of a number of operations ([http://www.molgenis.org/wiki/ComputeStartDataModel MCF Data Model]). A step of the job can be started when all operations of a previous step are finished. Operations in a step can be executed in parallel. Every operation writes to a job log file. Monitoring is implemented by reading job log files. Monitoring is called from [http://www.molgenis.org/svn/sandbox/molgenis_processing/3.3.galaxy/ScriptbasedComputePlatform/src/scriptserver/PipelineThread.java PipelineThread] after step is submitted for execution:
+Job execution is monitored by reading a log file of the job. A job consists of a number of steps which consists of a number of operations ([http://www.molgenis.org/wiki/ComputeStartDataModel MCF Data Model]). A step of the job can be started when all operations of a previous step are finished. Operations in a step can be executed in parallel. Every operation writes to a job log file. Monitoring is implemented by reading job log files. Monitoring is called from [http://www.molgenis.org/svn/sandbox/molgenis_processing/3.3.galaxy/ScriptbasedComputePlatform/src/scriptserver/PipelineThread.java PipelineThread] after the step is submitted for execution:
 
 {{{
@@ -33 +33 @@
 }}}
 
-    
+In actual reading of the remote logging file is done using gridgain in the next method of [http://www.molgenis.org/svn/sandbox/molgenis_processing/3.3.galaxy/ScriptbasedComputePlatform/src/monitor/LoggingReader.java LoggingReader]
+ 
+{{{
+public void checkStepStatus()
+    {
+        Future<RemoteResult> future = exec.submit(new RemoteLoggingReader(log_location));
+
+        RemoteResult back = null;
+
+        try
+        {
+            back = future.get();
+        }
+        
+        ... 
+
+        String logging = new String(back.getData());
+
+        summary.scripts_started = 0;
+        summary.scripts_finished = 0;
+        summary.scripts_all = currentStep.getNumberOfScripts();
+
+        for (int i = 0; i < summary.scripts_all; i++)
+        {
+            String script_id = currentStep.getScript(i).getID();
+
+            int index_started = logging.indexOf(script_id + _STARTED);
+            int index_finished = logging.indexOf(script_id + _FINISHED);
+
+            if (index_started > 0) summary.scripts_started++;
+            if (index_finished > 0) summary.scripts_finished++;
+        }
+
+        if (summary.scripts_finished == summary.scripts_all)
+            isStepFinished = true;
+       ...
+
+}}}
+