bout_runners.submitter.pbs_submitter¶
Contains the PBS submitter class.
Classes
|
The PBS submitter class. |
-
class
bout_runners.submitter.pbs_submitter.
PBSSubmitter
(job_name: Optional[str] = None, store_directory: Optional[pathlib.Path] = None, submission_dict: Optional[Dict[str, Optional[str]]] = None, processor_split: Optional[bout_runners.submitter.processor_split.ProcessorSplit] = None)[source]¶ The PBS submitter class.
Examples
>>> submitter = PBSSubmitter(job_name, store_path) >>> submitter.submit_command("echo 'Hello'") >>> submitter.wait_until_completed() >>> submitter.std_out Hello
- Attributes
- __dequeuedbool
Whether or not the job has been dequeued from the queue
Methods
_wait_for_std_out_and_std_err()
Wait until the process completes if a process has been started
get_return_code(sacct_str)
Return the exit code if any
get_return_code(trace)
Return the exit code if any
has_dequeue(trace)
Return whether or not the job has been removed from the queue
structure_time_to_pbs_format(time_str)
Structure the time string to a PBS time string
completed()
Return the completed status
create_submission_string(command, waiting_for)
Return the PBS script as a string
get_trace()
Return the trace from
tracejob
reset()
Reset dequeued, released, waiting_for and status dict
Set the member data.
- Parameters
- job_namestr or None
Name of the job If None, a timestamp will be given as job_name
- store_directoryPath or None
Directory to store the script If None, the caller directory will be used as the store directory
- submission_dictNone or dict of str of None or str
Dict containing optional submission options One the form
>>> {'walltime': None or str, ... 'account': None or str, ... 'queue': None or str, ... 'mail': None or str}
These options will not be used if the submission_dict is None
- processor_splitProcessorSplit or None
Object containing the processor split If None, default values will be used
- Attributes
job_id
Return the process id.
job_name
Set the properties of self.job_name.
released
Return whether the job has been released to the cluster.
return_code
Return the return code.
std_err
Return the standard error.
std_out
Return the standard output.
store_dir
Set the properties of self.store_dir.
waiting_for
Return the waiting for list as a tuple.
Methods
add_waiting_for
(waiting_for_id)Add a waiting for id to the waiting for list.
Return the completed status.
create_submission_string
(command, waiting_for)Return the PBS script as a string.
errored
([raise_error])Return True if the process errored.
extract_job_id
(std_out)Return the job_id.
get_days_hours_minutes_seconds_from_str
(time_str)Return days, hours, minutes, seconds from the string.
get_return_code
(trace)Return the exit code if any.
Return the trace from
tracejob
.has_dequeue
(trace)Return whether or not the job has been removed from the queue.
kill
()Kill a job if it exists.
Raise and error from the subprocess in a clean way.
release
()Release job if held.
reset
()Reset dequeued, released, waiting_for and status dict.
structure_time_to_pbs_format
(time_str)Structure the time string to a PBS time string.
submit_command
(command)Submit a command.
wait_until_completed
([raise_error])Wait until the process has completed.
write_python_script
(path, function[, args, …])Write python function to file.
-
add_waiting_for
(waiting_for_id: Union[str, None, Iterable[str]]) → None¶ Add a waiting for id to the waiting for list.
This will waiting for list will be written to the submission string upon creation
- Parameters
- waiting_for_idNone or list of str
Id to the job waiting for
-
create_submission_string
(command: str, waiting_for: Tuple[str, …]) → str[source]¶ Return the PBS script as a string.
- Parameters
- commandstr
The command to submit
- waiting_fortuple of str
Tuple of ids that this job will wait for
- Returns
- job_scriptstr
The script to be submitted
-
errored
(raise_error: bool = False) → bool¶ Return True if the process errored.
- Parameters
- raise_errorbool
Whether or not to raise errors
- Returns
- bool
True if the process returned a non-zero code
-
static
extract_job_id
(std_out: Optional[str]) → str[source]¶ Return the job_id.
- Parameters
- std_outstr or None
The standard output from the local submitter which submits the job
- Returns
- job_idstr
The job id
- Raises
- RuntimeError
If the job_id cannot be found
-
static
get_days_hours_minutes_seconds_from_str
(time_str: str) → Tuple[int, int, int, int]¶ Return days, hours, minutes, seconds from the string.
- Parameters
- time_strstr
Must be on the format
>>> 'hh:mm:ss'
or
>>> 'd-hh:mm:ss'
- Returns
- daysint
Number of days in the time string
- hoursint
Number of hours in the time string
- minutesint
Number of minutes in the time string
- secondsint
Number of seconds in the time string
- Raises
- ValueError
If the string is malformatted
-
static
get_return_code
(trace: str) → Optional[int][source]¶ Return the exit code if any.
- Parameters
- tracestr
Trace obtained from the tracejob command
- Returns
- return_codeNone or int
Return code obtained from the cluster
-
get_trace
() → str[source]¶ Return the trace from
tracejob
.- Returns
- tracestr
Trace obtained from the
tracejob
An empty string is will be returned if no job_id exist
-
static
has_dequeue
(trace: str) → bool[source]¶ Return whether or not the job has been removed from the queue.
- Parameters
- tracestr
Trace obtained from the tracejob command
- Returns
- bool
True if the job has been removed from the queue
-
property
job_id
¶ Return the process id.
- Returns
- self._status[“job_id”]int or None
The process id if a process has been called, else None
-
property
job_name
¶ Set the properties of self.job_name.
- Returns
- str
The job name
-
kill
() → None¶ Kill a job if it exists.
-
raise_error
() → None¶ Raise and error from the subprocess in a clean way.
- Raises
- RuntimeError
If an error was caught
-
release
() → None¶ Release job if held.
-
property
released
¶ Return whether the job has been released to the cluster.
- Returns
- bool
True if the job is not held in the cluster
-
property
return_code
¶ Return the return code.
-
property
std_err
¶ Return the standard error.
- Returns
- self._status[“std_err”]str or None
The standard error None if the process has not completed
-
property
std_out
¶ Return the standard output.
- Returns
- self._status[“std_out”]str or None
The standard output None if the process has not completed
-
property
store_dir
¶ Set the properties of self.store_dir.
- Returns
- Path
Path to the store directory
-
static
structure_time_to_pbs_format
(time_str: str) → str[source]¶ Structure the time string to a PBS time string.
- Parameters
- time_strstr
Must be on the format >>> ‘hh:mm:ss’ or >>> ‘d-hh:mm:ss’
- Returns
- str
The time string formatted as >>> ‘hh:mm:ss’
-
submit_command
(command: str) → None¶ Submit a command.
- Parameters
- commandstr
Command to submit
Notes
All submitted jobs are held Release with self.release See [1] for details
References
-
wait_until_completed
(raise_error: bool = True) → None¶ Wait until the process has completed.
- Parameters
- raise_errorbool
Whether or not to raise errors
-
property
waiting_for
¶ Return the waiting for list as a tuple.
- Returns
- tuple of str
The waiting for list as a tuple
-
static
write_python_script
(path: pathlib.Path, function: Callable, args: Optional[Tuple[Any, …]] = None, kwargs: Optional[Dict[str, Any]] = None) → None¶ Write python function to file.
- Parameters
- pathPath
Absolute path to store the python file which holds the function and its arguments
- functionfunction
The function to call
- argstuple
The positional arguments
- kwargsdict
The keyword arguments