Api

The Api class is used to authenicate with the belvis Api and retrieve timeseries data. It is used by the Tenant class for this purpose, but can also be used stand-alone to investigate the timeseries available in a Belvis instance.

Initialisation

We must specify the server (and port) and the Belvis tenant that will give us the data. For example:

import belvys
api = belvys.Api("belvisserver01:8040", "pfm_gas")

We can also initialise from a configuration file, with the belvys.Api.from_file() class method. For the example above, the contents of the corresponing yaml-file is

server: http://belvisserver:8040   # Server and port of the rest API. 
tenant: pfm_gas                    # Name of tenant on this server.

(download yaml 📄)

Authentication

Before we can get any data from the belvis server, we need to authenicate with by providing username and password:

# Continuation of previous example.
api.access_from_usr_pwd('Myfirst.Lastname', 'my_5tr0ngp@ssw0rd')

Warning

Never store your username and password in a python file - e.g. use environment variables instead, and access those from the built-in os.environ dictionary.

Usage

The following methods can be used to get timeseries data, if the timeseries ID (tsid) is known.

Metadata

.metadata() returns a dictionary describing the timeseries, see the example below. The most relevant metadata are highlighted. With the instanceToken, the portfolio ID is meant.

# Continuation of previous example.
api.metadata(23840744)

{'id': '23840744',
 'dataExchangeNumber': None,
 'instanceName': 'Household customers',
 'instanceToken': 'B2C_household',
 'instanceType': 'book',
 'measurementUnit': 'MW',
 'meteringCodeOnExport': None,
 'obisCode': None,
 'obisCodeOnMsconsExport': None,
 'parameterName': 'General',
 'specificationName': 'Buch.Wirkl.Saldo',
 'specificationNumber': 5163,
 'timeRange': '2016-01-01T06:00:00.000Z--2026-01-01T05:00:00.000Z',
 'timeSeriesName': 'Procurement forward volume in MW',
 'timeStampOfLastSaving': '2022-09-02T11:57:41.000Z',
 'virtualMeteringCode': None}

Series

This is the main method of this class; .series() returns actual timeseries data. It includes a physical (pint) unit if one is found in the metadata, see below.

# Continuation of previous example.
import pandas as pd
api.series(23840744, pd.Timestamp('2022-09-30'), pd.Timestamp('2022-10-02'))

ts
2022-09-29 23:00:00+00:00    17.802
2022-09-30 00:00:00+00:00    17.802
2022-09-30 01:00:00+00:00    17.802
2022-09-30 02:00:00+00:00    17.802
...                          ...
2022-10-01 02:00:00+00:00    17.802
2022-10-01 03:00:00+00:00    17.802
2022-10-01 04:00:00+00:00    17.802
2022-10-01 05:00:00+00:00    36.935
2022-10-01 06:00:00+00:00    36.935
2022-10-01 07:00:00+00:00    36.935
...                          ...
2022-10-01 18:00:00+00:00    36.935
2022-10-01 19:00:00+00:00    36.935
2022-10-01 20:00:00+00:00    36.935
2022-10-01 21:00:00+00:00    36.935
2022-10-01 22:00:00+00:00    36.935
Freq: H, Length: 48, dtype: pint[MW]

A few things to note here about the data as it is returned by the Belvis API:

  • Timestamps are localized to the UTC timezone. A conversion to the correct (in this case “Europe/Berlin”) timezone is necessary.

  • Timestamps are right-bound. In the example above, the first timestamp is 2022-09-29 23:00:00+00:00, which is the same as 2022-09-30 01:00:00+02:00 in the Europe/Berlin timezone. The first hour of 2022-09-30 is thus denoted with the 01:00 o’clock timestamp, which is when that hour ends, not when it starts.

  • A peculiarity of the gas market can also be seen: daily values do not apply from midnight to midnight, but rather from 06:00 to 06:00. The values change with the timestamp 2022-10-01 05:00:00+00:00, which is 2022-10-01 07:00:00+02:00 in the Europe/Berlin timezone, which in Belvis denotes the hour starting at 06:00 (see previous point).

More series

For convenience, the method .series_from_tsname() combines looking up the timeseries id with fetching the data. It is a thin wrapper around .series().

Timeseries IDs

For most methods above, the timeseries ID is needed. This is a number uniquely identifying a timeseries (including the portfolio that contains it) in the Belvis database.

In order to find the tsid, several methods are available, depending on how much information is known about the timeseries.

  • all_ts(): if only the portfolio is known, this method can be used. It returns a dictionary with the names and IDs of all the timeseries in it.

  • find_tsids(): if the portfolio and part of the timeseries name is known, this method can be used. It returns the same kind of dictionary as all_ts().

  • find_tsid(): if the portfolio and the exact timeseries name are known, this method is used. It returns the corresponding integer ID number.

Cache

To avoid repeating the (potentially time-costly) task of finding the timeseries ID, the class keeps a cache. When a file is provided during the initialisation, it is used to persistently store the cache every time it is updated. (When initialising with .from_file(), this file is automatically used for this purpose.)

Class Documentation

class belvys.Api(server: str, tenant: str, cache: Dict[str, Dict[str, int]] | None = None, filepath: Path | None = None)

A class to interact with a Belvis Rest API server.

access_from_usr_pwd(usr: str, pwd: str) → None

Specify username and password to access Belvis API.

Parameters:

  • usr (str) – Username.

  • pwd (str) – Password.

all_ts(pfid: str) → Dict[str, int]

Use API to find all timeseries in a Belvis portfolio and store in cache. May take a long time if portfolio has many timeseries.

Parameters:

pfid (str) – Id (= short name) of Belvis portfolio.

Returns:

Dictionary with found timeseries. Keys are the timeseries names, the values are the timeseries ids.

Return type:

Dict[str, int]

find_tsid(pfid: str, tsname: str) → int

Find Belvis id number of timeseries.

Parameters:

  • pfid (str) – Id (=short name) of portfolio as found in Belvis.

  • tsname (str) – Full name of timeseries.

Returns:

id of found timeseries.

Return type:

int

find_tsids(pfid: str, tsname: str = '', case_sensitive: bool = True, allow_partial: bool = True) → Dict[str, int]

Use API to find specific timeseries in a Belvis portfolio. May take a long time if portfolio has many timeseries.

Parameters:

  • pfid (str) – Id (= short name) of Belvis portfolio.

  • tsname (str, optional (default: return all timeseries).) – Name of timeseries.

  • case_sensitive (bool, optional (default: True)) – If True, do a case sensitive search for the timeseries name.

  • allow_partial (bool, optional (default: True)) – If True, also returns timeseries if the name partially matches.

Returns:

Dictionary with found timeseries. Keys are the timeseries names, the values are the timeseries ids.

Return type:

Dict[str, int]

classmethod from_file(filepath: str | Path) → Api

Load api data from file.

Parameters:

filepath (str | pathlib.Path) – Path to load yaml api configuration from. Also the path to save api configuration (which includes the cache).

Notes

The api configuration file must have the keys ‘server’ (= string of server address including port), ‘tenant’ (= name of the belvis tenant), and possibly ‘cache’ (= dictionary).

metadata(tsid: int) → Dict

Get information about timeseries.

Parameters:

tsid (int) – Belvis id of timeseries.

Returns:

Metadata about the timeseries.

Return type:

dict

query(url: str) → Dict | List

Query connection for general information.

read_cache(pfid: str, tsname: str) → int

Retrieve belvis timeseries id from cache file. Returns None if not found. Case sensitive.

Parameters:

  • pfid (str) – Id (= short name) of portfolio.

  • tsname (str) – Name of timeseries.

Return type:

int

series(tsid: int, ts_left: Timestamp | datetime, ts_right: Timestamp | datetime, *, leftrange: str = 'exclusive', rightrange: str = 'inclusive', missing2zero: bool = True, blocking: bool = True) → Series

Return timeseries in given delivery time interval using its id.

Parameters:

  • tsid (int) – Belvis id of timeseries.

  • ts_left (pd.Timestamp | dt.datetime) –

  • ts_right (pd.Timestamp | dt.datetime) –

  • leftrange (str, optional (default: 'exclusive')) – ‘inclusive’ (‘exclusive’) to get values with timestamp that is >= (>) ts_left. Default: ‘exclusive’ because timestamps in Belvis are usually right-bound.

  • rightrange (str, optional (default: 'inclusive')) – ‘inclusive’ (‘exclusive’) to get values with timestamp that is <= (<) ts_right. Default: ‘inclusive’ because timestamps in Belvis are usually right-bound.

  • missing2zero (bool, optional (default: True)) – What to do with values that are flagged as ‘missing’. True to replace with 0, False to replace with nan.

  • blocking (bool, optional (default: True)) – If True, recalculates data that is not up-to-date before returning; might take long time or result in internal-server-error. If False, return most up-to-date data that is available without recalculating.

Returns:

with resulting information.

Return type:

pd.Series

Notes

Returns series with data as found in Belvis; no correction (e.g. for right-bounded timestamps) done.

series_from_tsname(pfid: str, tsname: str, ts_left: Timestamp | datetime, ts_right: Timestamp | datetime, *, leftrange: str = 'exclusive', rightrange: str = 'inclusive', missing2zero: bool = True, blocking: bool = True) → Series

Return timeseries in given delivery time interval, using its name and portfolio id.

Parameters:

  • pfid (int) – ID (=short name) of portfolio in Belvis.

  • tsname (str) – Name of the timeseries. Must be exact.

  • ts_left (pd.Timestamp | dt.datetime) –

  • ts_right (pd.Timestamp | dt.datetime) –

  • leftrange (str, optional (default: 'exclusive')) – ‘inclusive’ (‘exclusive’) to get values with timestamp that is >= (>) ts_left. Default: ‘exclusive’ because timestamps in Belvis are usually right-bound.

  • rightrange (str, optional (default: 'inclusive')) – ‘inclusive’ (‘exclusive’) to get values with timestamp that is <= (<) ts_right. Default: ‘inclusive’ because timestamps in Belvis are usually right-bound.

  • missing2zero (bool, optional (default: True)) – What to do with values that are flagged as ‘missing’. True to replace with 0, False to replace with nan.

  • blocking (bool, optional (default: True)) – If True, recalculates data that is not up-to-date before returning; might take long time or result in internal-server-error. If False, return most up-to-date data that is available without recalculating.

Returns:

with resulting information.

Return type:

pd.Series

Notes

  • Returns series with data as found in Belvis; no correction (e.g. for right-bounded timestamps) done.

  • If not yet cached, the .series() method is potentially a lot faster.

to_file(filepath: Path) → None

Write api configuration to file. Access method is not stored.

Parameters:

filepath (pathlib.Path) – Path of file to store data to.

write_cache(pfid: str, tsname_to_tsid: Dict[str, int]) → None

Write belvis timeseries id into cache file. Overwrites if already present.

Parameters:

  • pfid (str) – Id (= short name) of the Belvis portfolio.

  • tsname_to_tsid (Dict[str, int]) – Dictionary mapping timeseries names to their ids.