bartender.schedulers.dirac#

Attributes#

dirac_status_map

logger

Classes#

DiracScheduler

DIRAC scheduler.

Functions#

_relative_output_dir(→ pathlib.Path)

Return description.output_dir relative to user's home directory.

_extract_text_file(→ None)

Extract text file from tarfile.

Module Contents#

bartender.schedulers.dirac.dirac_status_map: dict[str, bartender.db.models.job_model.State]#
bartender.schedulers.dirac.logger#
class bartender.schedulers.dirac.DiracScheduler(config: bartender.schedulers.dirac_config.DiracSchedulerConfig)#

Bases: bartender.schedulers.abstract.AbstractScheduler

DIRAC scheduler.

[More info](http://diracgrid.org/).

Parameters:

config (bartender.schedulers.dirac_config.DiracSchedulerConfig) –

config: bartender.schedulers.dirac_config.DiracSchedulerConfig#
wms_client#
monitoring#
async submit(description: bartender.schedulers.abstract.JobDescription) → str#

Submit a job description for running.

The job submitted to DIRAC will do the following:

  1. Download description.job_dir / input.tar from grid storage.

  2. Unpack input.tar.

  3. Run description.command

    • Capturing stdout and stderr as stdout.txt and stderr.txt files.

    • Capturing return code in return_code file.

  4. Pack output.tar.

    • Excluding files from input.tar.

  5. Upload output.tar to grid storage as description.job_dir / output.tar.

Parameters:

description (bartender.schedulers.abstract.JobDescription) – Description for a job.

Raises:

RuntimeError – When submission fails.

Returns:

Identifier that can be used later to interact with job.

Return type:

str

async state(job_id: str) → bartender.db.models.job_model.State#

Get state of a job.

Once job is completed, then scheduler can forget job.

Parameters:

job_id (str) – Identifier of job.

Raises:

RuntimeError – When getting job status fails.

Returns:

State of job.

Return type:

bartender.db.models.job_model.State

async states(job_ids: list[str]) → list[bartender.db.models.job_model.State]#

Get state of jobs.

Once a job is completed, then scheduler can forget job.

Parameters:

job_ids (list[str]) – Identifiers of jobs.

Raises:

RuntimeError – When getting jobs status fails.

Returns:

States of jobs.

Return type:

list[bartender.db.models.job_model.State]

async cancel(job_id: str) → None#

Cancel a queued or running job.

Once a queued job is cancelled, then the scheduler can forget job.

Parameters:

job_id (str) – Identifier of job.

Raises:

RuntimeError – When cancelling of job fails.

Return type:

None

async close() → None#

Close scheduler.

Return type:

None

async logs(job_id: str, job_dir: pathlib.Path) → Tuple[str, str]#

Get stdout and stderr of a job.

If job has not completed, then will raise an exception. If job completed succesfully, then stdout,txt and stderr.txt are read from job_dir. If job failed, then stdout and stderr are read from grid storage.

Logs of a failed job includes logs of unpacking/packing the input and output files.

The logs can be fetched on command line with dirac-wms-job-get-output <job id>.

Parameters:

  • job_id (str) – Identifier of job.

  • job_dir (pathlib.Path) – Directory where stdout.txt and stderr.txt files of job are stored.

Raises:

RuntimeError – When fetching of logs fails.

Returns:

Tuple of stdout and stderr.

Return type:

Tuple[str, str]

async _job_script(description: bartender.schedulers.abstract.JobDescription, scriptdir: pathlib.Path) → pathlib.Path#
Parameters:
Return type:

pathlib.Path

_job_script_content(description: bartender.schedulers.abstract.JobDescription) → str#
Parameters:

description (bartender.schedulers.abstract.JobDescription) –

Return type:

str

_command_script(description: bartender.schedulers.abstract.JobDescription) → str#
Parameters:

description (bartender.schedulers.abstract.JobDescription) –

Return type:

str

async _jdl_script(description: bartender.schedulers.abstract.JobDescription, scriptdir: pathlib.Path) → str#
Parameters:
Return type:

str

bartender.schedulers.dirac._relative_output_dir(description: bartender.schedulers.abstract.JobDescription) → pathlib.Path#

Return description.output_dir relative to user’s home directory.

user home directory is /<vo>/user/<initial>/<user> to write /tutoVO/user/c/ciuser/bartenderjobs/job1/input.tar OutputPath must be bartenderjobs/job1

Parameters:

description (bartender.schedulers.abstract.JobDescription) – Description of job.

Returns:

.description.output_dir relative to user’s home directory.

Return type:

pathlib.Path

async bartender.schedulers.dirac._extract_text_file(tar: tarfile.TarFile, src: str, target: pathlib.Path) → None#

Extract text file from tarfile.

Parameters:
Return type:

None