scheduler

JobScheduler classes

class scheduler.JobScheduler(source: PathFinder)

Base class for scheduling a merge job

Initialize the JobScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

run() → None

Run the Job scheduler.

Returns:

None

write_json(chunk) → str

Write a JSON dictionary to a file and return the file name.

Parameters:

chunk – MergeChunk object to write

Returns:

Name of the written JSON file

abstract write_script() → list

Write the job script

Returns:

List of the generated script file name(s)

class scheduler.JustinScheduler(source: PathFinder)

Job scheduler for JustIN merge jobs

Initialize the JustinScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

justin_cmd(tier: int, site: str, cvmfs_dir: str | None = None) → str

Create the JustIN command for submitting a merge job.

Parameters:

• tier – Merge pass (1 or 2)

• site – Site to run the job

• cvmfs_dir – CVMFS directory where config files are located

Returns:

JustIN command string

upload_cfg() → None

Make a tarball of the configuration files and upload them to cvmfs

Returns:

Path to the uploaded configuration directory

write_script() → list

Write the job scripts for submitting JustIN merge jobs.

Returns:

Name of the generated script file(s)

class scheduler.LocalScheduler(source: PathFinder)

Job scheduler for local merge jobs

Initialize the JobScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

write_json(chunk) → str

Write a JSON dictionary to a file and return the file name.

Parameters:

chunk – MergeChunk object to write

Returns:

Name of the written JSON file

write_script() → list

Write the job script for running local merge jobs.

Returns:

Name of the generated script file