bout_runners.submitter.abstract_cluster_submitter¶

Contains the abstract cluster submitter class.

Classes

AbstractClusterSubmitter(job_name, …)

The abstract cluster class of the submitters.

class bout_runners.submitter.abstract_cluster_submitter.AbstractClusterSubmitter(job_name: Optional[str] = None, store_dir: Optional[pathlib.Path] = None, submission_dict: Optional[Dict[str, Optional[str]]] = None, processor_split: Optional[bout_runners.submitter.processor_split.ProcessorSplit] = None)[source]¶

The abstract cluster class of the submitters.

Attributes

_cluster_specificdict: Dict containing the commands for cancelling a job, releasing a job and submit a job for the inherited object
_job_namestr: Getter and setter variable for job_name
_log_and_error_basePath: Base for the path for the .log and .err files
_store_dirPath: Getter and setter variable for store_dir
_submission_dictdict: Dict containing walltime, mail, queue and account info
_releasedbool: Getter variable for waiting_for
_waiting_fortuple of str: Getter variable for released
job_namestr: Set the properties of self.job_name.
releasedbool: Return whether the job has been released to the cluster.
store_dirPath: Set the properties of self.store_dir.
waiting_fortuple of str: Return the waiting for list as a tuple.

Methods

_populate_std_out_and_std_err()	Populate std_out and std_err
get_return_code(sacct_str)	Return the exit code if any
create_submission_string(command, waiting_for)	Create the submission string
get_days_hours_minutes_seconds_from_str(time_str)	Return days, hours, minutes, seconds from the string
add_waiting_for(waiting_for_id)	Add a waiting for id to the waiting for list
kill()	Kill a job if it exists
release()	Release job if held
submit_command(command)	Submit a command
raise_error()	Raise and error from the subprocess in a clean way

Set the member data.

Parameters

job_namestr or None

Name of the job If None, a timestamp will be given as job_name

store_dirPath or None

Directory to store the script If None, the caller directory will be used as the store directory

submission_dictNone or dict of str of None or str

Dict containing optional submission options One the form

>>> {'walltime': None or str,
...  'mail': None or str,
...  'queue': None or str,
...  'account': None or str}

These options will not be used if the submission_dict is None

processor_splitProcessorSplit or None

Object containing the processor split If None, default values will be used

Attributes

job_id: Return the process id.
job_name: Set the properties of self.job_name.
released: Return whether the job has been released to the cluster.
return_code: Return the return code.
std_err: Return the standard error.
std_out: Return the standard output.
store_dir: Set the properties of self.store_dir.
waiting_for: Return the waiting for list as a tuple.

Methods

`add_waiting_for`(waiting_for_id)	Add a waiting for id to the waiting for list.
`completed`()	Return the completed status.
`create_submission_string`(command, waiting_for)	Create the submission string.
`errored`([raise_error])	Return True if the process errored.
`extract_job_id`(std_out)	Return the job_id.
`get_days_hours_minutes_seconds_from_str`(time_str)	Return days, hours, minutes, seconds from the string.
`kill`()	Kill a job if it exists.
`raise_error`()	Raise and error from the subprocess in a clean way.
`release`()	Release job if held.
`reset`()	Reset the submitter.
`submit_command`(command)	Submit a command.
`wait_until_completed`([raise_error])	Wait until the process has completed.
`write_python_script`(path, function[, args, …])	Write python function to file.

add_waiting_for(waiting_for_id: Union[str, None, Iterable[str]]) → None[source]¶

Add a waiting for id to the waiting for list.

This will waiting for list will be written to the submission string upon creation

Parameters

waiting_for_idNone or list of str: Id to the job waiting for

abstract completed() → bool¶: Return the completed status.

abstract create_submission_string(command: str, waiting_for: Tuple[str, …]) → str[source]¶

Create the submission string.

Parameters

commandstr: The command to submit
waiting_fortuple of str: Tuple of ids that this job will wait for

errored(raise_error: bool = False) → bool¶

Return True if the process errored.

Parameters

raise_errorbool: Whether or not to raise errors

Returns

bool: True if the process returned a non-zero code

abstract static extract_job_id(std_out: Optional[str]) → str[source]¶

Return the job_id.

Parameters

std_outstr or None: The standard output from the local submitter which submits the job

Returns

job_idstr: The job id

static get_days_hours_minutes_seconds_from_str(time_str: str) → Tuple[int, int, int, int][source]¶

Return days, hours, minutes, seconds from the string.

Parameters

time_strstr

Must be on the format

>>> 'hh:mm:ss'

or

>>> 'd-hh:mm:ss'

Returns

daysint: Number of days in the time string
hoursint: Number of hours in the time string
minutesint: Number of minutes in the time string
secondsint: Number of seconds in the time string

Raises

ValueError: If the string is malformatted

property job_id¶

Return the process id.

Returns

self._status[“job_id”]int or None: The process id if a process has been called, else None

property job_name¶

Set the properties of self.job_name.

Returns

str: The job name

kill() → None[source]¶: Kill a job if it exists.

raise_error() → None[source]¶

Raise and error from the subprocess in a clean way.

Raises

RuntimeError: If an error was caught

release() → None[source]¶: Release job if held.

property released¶

Return whether the job has been released to the cluster.

Returns

bool: True if the job is not held in the cluster

reset() → None¶: Reset the submitter.

property return_code¶: Return the return code.

property std_err¶

Return the standard error.

Returns

self._status[“std_err”]str or None: The standard error None if the process has not completed

property std_out¶

Return the standard output.

Returns

self._status[“std_out”]str or None: The standard output None if the process has not completed

property store_dir¶

Set the properties of self.store_dir.

Returns

Path: Path to the store directory

submit_command(command: str) → None[source]¶

Submit a command.

Parameters

commandstr: Command to submit

Notes

All submitted jobs are held Release with self.release See [1] for details

References

1: https://community.openpbs.org/t/ignoring-finished-dependencies/1976

wait_until_completed(raise_error: bool = True) → None¶

Wait until the process has completed.

Parameters

raise_errorbool: Whether or not to raise errors

property waiting_for¶

Return the waiting for list as a tuple.

Returns

tuple of str: The waiting for list as a tuple

static write_python_script(path: pathlib.Path, function: Callable, args: Optional[Tuple[Any, …]] = None, kwargs: Optional[Dict[str, Any]] = None) → None¶

Write python function to file.

Parameters

pathPath: Absolute path to store the python file which holds the function and its arguments
functionfunction: The function to call
argstuple: The positional arguments
kwargsdict: The keyword arguments