bout_runners.metadata.metadata_reader

Module containing the MetadataReader class.

Functions

drop_ids(func)

Return a function which remove excessive ids.

Classes

MetadataReader(db_connector, drop_id)

Class for reading the metadata from the database.
class bout_runners.metadata.metadata_reader.MetadataReader(db_connector: bout_runners.database.database_connector.DatabaseConnector, drop_id: Optional[str] = 'keep_run_id')[source]

Class for reading the metadata from the database.

Examples

>>> from pathlib import Path
>>> from bout_runners.database.database_connector import DatabaseConnector
>>> db_connector = DatabaseConnector('test', Path())
>>> metadata_reader = MetadataReader(db_connector)
>>> metadata_reader.get_parameters_metadata()
   bar.id  bar.foo  ... parameters.baz_id  parameters.foo_id
0       1        1  ...                 1                  1
1       2       10  ...                 1                  2
2       2       10  ...                 1                  1

[3 rows x 16 columns]

>>> metadata_reader.get_all_metadata()
   run.id  ...                  system_info.version
0       1  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
1       2  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
2       3  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
3       4  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
4       5  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
5       6  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
6       7  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019

[7 rows x 43 columns]

>>> metadata_reader.drop_id = 'all_id'
>>> metadata_reader.get_all_metadata()
  run.latest_status  ...                  system_info.version
0          complete  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
1          complete  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
2          complete  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
3          complete  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
4             error  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
5           running  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019
6         submitted  ...  #1 SMP Thu Oct 17 19:31:58 UTC 2019

[7 rows x 28 columns]

Attributes
__db_readerDatabaseConnector

The connection to the database

__table_namestuple

Getter variable for table_names

__table_column_dictdict of tuple

Getter variable for table_column_dict

__table_connectionsdict of tuple

Getter variable for table_connections

__sorted_columnstuple

Getter variable for sorted_columns

table_namestuple

Set the properties of self.table_names.

table_column_dictdict of tuple

Set the properties of self.table_column_dict.

table_connectionsdict of tuple

A dict where the keys are tables, and the values are tuples of tables connected to the key table

sorted_columnstuple

Set the properties of self.sorted_columns.

date_columnstuple

Columns containing dates

drop_idNone or str

Specifies what id columns should be dropped when obtaining the metadata

Methods

get_all_metadata()

Return all of the run metadata

get_parameters_metadata()

Return only the parameter part of the run metadata

get_join_query(from_statement, columns, alias_columns, table_connections)

Return the query string of a SELECT query with INNER JOIN

__get_parameters_query()

Return the parameters query string

__get_sorted_columns()

Return all columns sorted

__get_table_connections()

Return a dict containing the table connections

__get_all_table_names()

Return all the table names in the schema

__get_table_column_dict()

Return all the column names of the specified tables

Set the database to use.

Parameters
db_connectorDatabaseConnector

The connection to the database

drop_idNone or str

Specifies what id columns should be dropped when obtaining the metadata - None : No columns will be dropped - ‘parameters’ : All columns containing parameters ids

will be dropped

  • ‘keep_run_id’ : Only the run.id of the id columns will be kept

  • ‘all_id’ : All id columns will be removed

Attributes
sorted_columns

Set the properties of self.sorted_columns.

table_column_dict

Set the properties of self.table_column_dict.

table_connection

Set the properties of self.table_connections.

table_names

Set the properties of self.table_names.

Methods

get_all_metadata(*args, **kwargs)

Drop columns inplace.

get_join_query(from_statement, columns, …)

Return the query string of a SELECT query with INNER JOIN.

get_parameters_metadata(*args, **kwargs)

Drop columns inplace.
get_all_metadata(*args, **kwargs) → pandas.core.frame.DataFrame[source]

Drop columns inplace.

Parameters
selfobject

Self reference to the instance the function is belonging to Must contain self.drop_id

argstuple

Arguments belonging to the input function

kwargsdict

Keyword arguments to the input function

Returns
data_frameDataFrame

The DataFrame where the ids has been dropped

static get_join_query(from_statement: str, columns: Sequence[str], alias_columns: Sequence[str], table_connections: Dict[str, Tuple[str, …]]) → str[source]

Return the query string of a SELECT query with INNER JOIN.

Parameters
from_statementstr

The statement after the FROM keyword in the query I.e.

>>> f'SELECT * FROM {from_statement}'
columnsarray_like

The columns to select from the tables I.e.

>>> f'SELECT {columns} FROM *'
alias_columnsarray_like

The name of the columns in the resulting table I.e.

>>> f'SELECT {columns[0]} AS {alias_columns[0]} FROM *'
table_connectionsdict

A dict where the keys are the table names, and the values are tuples containing table names connected to the key table as described in the note above

Returns
querystr

The SQL-string which can be used to query where table in databases are joined through INNER JOIN operations

Notes

The tables in table_connection is assumed to be joined by id`s. I.e. `table_a is connected to table_b by table_b having a column named table_a_id which corresponds to the id column of table_a

get_parameters_metadata(*args, **kwargs) → pandas.core.frame.DataFrame[source]

Drop columns inplace.

Parameters
selfobject

Self reference to the instance the function is belonging to Must contain self.drop_id

argstuple

Arguments belonging to the input function

kwargsdict

Keyword arguments to the input function

Returns
data_frameDataFrame

The DataFrame where the ids has been dropped

property sorted_columns

Set the properties of self.sorted_columns.

Returns
self.__sorted_columnstuple

A tuple of the column names as they will be sorted in the all_metadata DataFrame

property table_column_dict

Set the properties of self.table_column_dict.

Returns
self.__table_column_dictdict of tuple

A dict where the keys are table names, and the values are corresponding column names

property table_connection

Set the properties of self.table_connections.

Returns
self.__table_connectionsdict of tuple

A dict where the keys are tables, and the values are tuples of tables connected to the key table

property table_names

Set the properties of self.table_names.

Returns
self.__table_namestuple

A tuple containing all names of the tables

bout_runners.metadata.metadata_reader.drop_ids(func: Callable) → Callable[source]

Return a function which remove excessive ids.

Parameters
funcfunction

A function returning a DataFrame

Returns
dropfunction

The function dropping the ids