scheduler

JobScheduler classes

class scheduler.JobScheduler(source: PathFinder)[source]

Base class for scheduling a merge job

Initialize the JobScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

run() → None[source]

Run the Job scheduler.

Returns:

None

write_json(chunk) → str[source]

Write a JSON dictionary to a file and return the file name.

Parameters:

chunk – MergeChunk object to write

Returns:

Name of the written JSON file

abstract write_script(tier: int) → str[source]

Write the job script for a given tier.

Parameters:

tier – Pass number (1 or 2)

Returns:

Name of the generated script file

class scheduler.JustinScheduler(source: PathFinder)[source]

Job scheduler for JustIN merge jobs

Initialize the JustinScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

upload_cfg() → None[source]

Make a tarball of the configuration files and upload them to cvmfs

Returns:

Path to the uploaded configuration directory

write_script(tier: int) → str[source]

Write the job script for a given tier.

Parameters:

tier – Pass number (1 or 2)

Returns:

Name of the generated script file

class scheduler.LocalScheduler(source: PathFinder)[source]

Job scheduler for local merge jobs

Initialize the JobScheduler with a source of files to merge.

Parameters:

source – PathFinder object to provide input files

write_json(chunk) → str[source]

Write a JSON dictionary to a file and return the file name.

Parameters:

chunk – MergeChunk object to write

Returns:

Name of the written JSON file

write_script(tier: int) → str[source]

Write the job script for a given tier.

Parameters:

tier – Pass number (1 or 2)

Returns:

Name of the generated script file