Difference between revisions of "JEX"
(→Installation) |
|||
| (5 intermediate revisions by 2 users not shown) | |||
| Line 1: | Line 1: | ||
| − | == | + | == Summary == |
| − | CoGe is in the process of moving all analysis to the job execution framework. The job execution framework allows analysis to run in a | + | Yerba is the job execution engine used by CoGe. CoGe is in the process of moving all analysis to the job execution framework. |
| + | The job execution framework allows analysis to run in a heterogeneous environment. An analysis can then be run in an environment which best suits its computational needs. | ||
| + | An analysis will be run as a workflow which may contain multiple tasks to be run. | ||
[[Image:JEX Diagram1.png|center|500px|High level diagram illustrating how tools on CoGe have jobs processed.]]<br> | [[Image:JEX Diagram1.png|center|500px|High level diagram illustrating how tools on CoGe have jobs processed.]]<br> | ||
| − | |||
| − | |||
| − | |||
| − | |||
== Installation == | == Installation == | ||
| − | For installation instructions please refer to the [https://github.com/LyonsLab/Yerba | + | For installation instructions please refer to the [https://github.com/LyonsLab/Yerba git repository] or the README.md in the repository.<br> |
To clone the repository run the command: | To clone the repository run the command: | ||
| Line 46: | Line 44: | ||
To update the database the command can then be ran with the update argument. | To update the database the command can then be ran with the update argument. | ||
coge/scripts/update-jobs coge.conf --update | coge/scripts/update-jobs coge.conf --update | ||
| + | |||
| + | == Reference == | ||
| + | *[https://docs.google.com/document/d/1zQ-oQx8JxMPKzGsrXU5h39Thd46aX6vJkW-MV_8v3GI/edit?usp=sharing Functional Design Document] | ||
Latest revision as of 12:50, 3 May 2017
Contents
Summary
Yerba is the job execution engine used by CoGe. CoGe is in the process of moving all analysis to the job execution framework. The job execution framework allows analysis to run in a heterogeneous environment. An analysis can then be run in an environment which best suits its computational needs. An analysis will be run as a workflow which may contain multiple tasks to be run.
Installation
For installation instructions please refer to the git repository or the README.md in the repository.
To clone the repository run the command:
git clone https://github.com/LyonsLab/Yerba.git
Usage
Yerba has two components the job daemon yerbad and the command line client yerba. The job daemon is started started when the server is first booted. Additionally, cctool's catalog and worker pool need to be running. yerba' is a command line interface to the job engine to check the status of the job engine. More features are planned in the future.
yerbad
To (manually) start the job engine you need to start each script.
sudo start catalog_server sudo start work_queue_pool sudo start yerba
yerba
This command currently only checks for a response from the job engine.
$YERBA_DIR/bin/yerba
Another method to check the status of the job engine is: status yerba
CoGe yerba utilities
CoGe has several utilities that provide functionality beyond the scope of the job engine.
update_jobs
This script keeps the CoGe jobs table up to date. By default the command will perform a dry run and not update the database.
coge/scripts/update-jobs coge.conf
To update the database the command can then be ran with the update argument.
coge/scripts/update-jobs coge.conf --update
