Terminology and data models

This page describes commonly used terms used in the API and the Python client.

Curve

A Curve describes any data series. The curve’s name is unique and identifies data series in the API. The Curve-model contains all the meta-information which makes up its name, such as categories, areas, unit and others.

An important attribute on the curve model is resolution (more on resolutions below). It tells you the time step (hourly, 15-minute, daily, etc.) and timezone for data in the curve.

The subscription attribute tells you what kind of subscription gives you access to the curve.

Curve types

Another essential detail is the curve_type attribute. This attribute describes what kind of data the curve stores. There different types are:

  • TIMESERIES

  • SCENARIO_TIMESERIES

  • INSTANCE

  • PERIODS

  • PERIOD_INSTANCES

  • OHLC

The curve_type tells you which operations you shall use to load data from the curve in the API.

Subscription

A Subscription describes your access to a curve. It contains information like the level of access, type of subscription, a human-readable label and more information such as the package, the area and the collection depending on the type of subscription.

Subscription access levels

Access levels for a subscription.

  • FREEMIUM – Access is included in the freemium tier

  • BLOCKED – No access

  • TRIAL – Access is granted during a trial period

  • PAYING – Access is provided through a paid subscription

  • REALTO – Access is provided with a www.realto.io subscription

  • INTERNAL – Access is provided through partner agreements

  • FREE – Access is available at no cost

  • NONE – Missing access information

Subscription types

The type of subscription.

  • COLLECTION – A subscription related to collections

  • FREE – No subscription required for provided content

  • FREEMIUM – Limited access due to no subscription defined for provided content

  • PACKAGE – A subscription associated with a package of services

  • PACKAGE_AREA – A subscription tied to a combination of a specific package and area

  • PRIVATE – A private subscription with restricted access

Subscription collection permissions

The user’s permissions for a collection.

  • r – Read-only access

  • rw – Read-write access

Instance

Some data series, such as those that contain forecasts, are not only one series, but a collection of many series. We call each of these series instances.

An Instance is identified by the combination of two attributes: An issue date (date-time) and a tag (string).

For instance, Energy Quantified’s forecasts based on the weather forecast ECMWF uses tag = 'ec' and the issue date as specified by ECMWF. So the morning forecast has these attributes:

# ECMWF deterministic forecast at midnight on 1 January 2020
instance.issued = datetime(2020, 1, 1, 0, 0, 0, 0, tz=UTC)
instance.tag = 'ec'

Time series that are instances (forecasts) have an instance attribute.

Place

The Place model is a rather generic: It represents anything that has a geographical location, and therefore it has a latitude and longitude.

Places have a type attribute describing what you may find in this place! These types are currently:

  • producer – Powerplant. Where available, you will also get a fuel attribute with the production type (wind, solar, nuclear, etc.).

  • consumer – Factory or otherwise large consumer of power

  • weatherstation – A weather station

  • river – A point on a river (used for river temperature forecasts at critical locations)

Curves may be linked to a place (for instance actual production for a nuclear power plant). And a place has a list of all curves connected to it.

Resolution, timezone and frequency

Power markets operate on contracts such as 15-minute, hourly, daily, weekly, monthly, quarterly and yearly. We call these different time intervals for frequencies.

Frequency

A frequency is a time step. We use mostly ISO-8601-style naming of frequencies, but with a few exceptions. See Duration (Wikipedia) for an excellent explanation of the format.

  • P1Y – Yearly

  • SEASON – Summer or winter

  • P3M – Quarterly

  • P1M – Monthly

  • P1W – Weekly

  • P1D – Daily

  • PT1H – Hourly

  • PT30M – 30 minutes

  • PT15M – 15 minutes

  • PT10M – 10 minutes

  • PT5M – 5 minutes

The SEASON frequency is used for gas market contracts. It starts on 1 April (summer) or 1 October (winter) and lasts six months.

Besides, the following frequency constant is used when data does not follow a fixed interval (such as tick data). It is an invalid frequency for operations that involve the Timeseries model.

  • NONE – No frequency specified (i.e. tick data)

See the Frequency enum class for more details.

Timezone

These are the most commonly used timezones. Most power markets in Europe operate in CET due to standardization and market coupling.

  • UTC – Coordinated Universal Time

  • WET – Western European Time

  • CET – Central European Time

  • EET – Eastern European Time

  • Europe/Istanbul – Turkey Time

  • Europe/Moscow – Russian/Moscow Time

  • Europe/Gas_Day – (Non-standard timezone; not in the IANA timezone database) European Gas Day at UTC-0500 (UTC-0400 during Daylight Saving Time). Starts at 06:00 in CE(S)T time. Used for the natural gas market in the European Union.

We use the pytz library for timezones.

Resolution

It is a combination of a frequency and a timezone. All time series have a resolution. Only resolutions with iterable frequencies are iterable (meaning all frequencies other than NONE).

With Energy Quantified’s Python library, you can do something like this:

>>> from energyquantified.time import (
>>>    Resolution, Frequency, UTC, get_datetime
>>> )
>>> resolution = Resolution(Frequency.P1D, UTC)
>>> begin = get_datetime(2020, 1, 1, tz=UTC)
>>> end = get_datetime(2020, 1, 5, tz=UTC)
>>> for d in resolution.enumerate(begin, end):
>>>     print(d)
2020-01-01 00:00:00+00:00
2020-01-02 00:00:00+00:00
2020-01-03 00:00:00+00:00
2020-01-04 00:00:00+00:00

Of course, you could use datetime.timedelta from the standard Python library to achieve a similar result. However, datetime.timedelta does not handle the transition from/to daylight saving time. Using the Resolution will make sure that the date-times get the right offset from UTC.

See the Resolution class for a full reference.

Aggregation and filters

Aggregation

To aggregate means to downsample data to a lower resolution. Example: Convert hourly values to daily values.

When aggregating, you must choose a strategy for how to calculate the aggregated value. The supported aggregations are:

  • AVERAGE – The mean of all input values

  • SUM – Sum of all input values

  • MIN – Find the lowest value

  • MAX – Find the highest value

Energy Quantified defaults to use AVERAGE (mean).

Class reference: Aggregation

Filters (or hour-filters)

You can also apply filters on which hours you want to include in aggregations.

In the power markets, one typically make a distinction between base and peak hours. Some weekly contracts traditionally also separate workdays from weekends. Here are some explanations:

  • BASE – All hours

  • PEAK – Peak hours (8-20). For future contracts: Peak hours (8-20) during workdays

  • OFFPEAK – Offpeak (0-8 and 20-24). For future contracts: Offpeak hours (0-8 and 20-24) during workdays and all hours during the weekend

  • WORKDAYS – Monday, Tuesday, Wednesday, Thursday, Friday

  • WEEKENDS – Saturday, Sunday

Important: When loading aggregated time series data from the API, you should keep the following in mind:

  • For weekly, monthly, quarterly and yearly resolutions, PEAK is defined as PEAK hours during WORKDAYS (8-20 during workdays). OFFPEAK is, for the same resolutions, defined as OFFPEAK hours during WORKDAYS and all hours during WEEKENDS.

  • For daily resolutions, PEAK and OFFPEAK do not make a distinction between workdays and weekends.

Class reference: Filter

Conversions

Unit

Convert data to another unit. Supported units at this moment:

  • °C for temperatures in celsius degrees

  • Degrees for angles in degrees

  • hPa for pressure in hectopascal

  • m for length in meters

  • m^2 for area in square meters

  • m^3 for volume in cubic meters

  • s for time in seconds

  • t for weight in tons

  • TW, GW, MW, kW, W for power in watt

  • TWh, GWh, MWh, kWh, Wh for energy in watt-hours

  • TWh/h, GWh/h, MWh/h, kWh/h, Wh/h for average energy in watt-hours per hour

  • therm for heat energy in therms

  • bbl for volume in barrels

  • % as percent

  • EUR, USD, GBP, NOK, SEK, DKK, CHF, CZK, HUF, PLN, BGN, HRK, RUB, RON, TRY, pence for currencies

Note: Currency conversions are not supported for timeseries with a frequency higher than P1D and not for periods.

Aggregation threshold

By default, the aggregation returns empty values whenever one or more input values are missing. You can set a threshold that defines how many values are allowed to be missing within a frame of the converted frequency. If the number of missing values is less than or equal to the threshold, aggregation is performed on the remaining non-empty values. Otherwise, an empty value is returned.

Note: By default, the threshold is set to zero. This means that an empty input value will result in an empty output value.

For example, you want to convert hourly values to daily values using the mean value. Let’s assume that some input values are missing. Instead of getting empty values, you want to get the average if a maximum of four values are missing within a day. In this case, set the threshold to four.

Time series

A Timeseries is a data series with date-times as the index. Time series in Energy Quantified’s API has a fixed interval (i.e. 15-minute, hourly, daily). For time series with varying duration per item, see [Period series](#period-series).

Example of a time series:

Date         Value
----------  ------
2020-01-01   145.2
2020-01-02   156.9
2020-01-03   167.4
2020-01-04   134.1
...

Time series data can have a varying number of values per date-time:

  • Single-value: Each date-time has one corresponding value.

  • Scenarios: Each date-time has multiple values.

  • Scenarios with mean value: Each date-time has multiple values and a mean value of those scenarios.

Scenarios are sometimes also referred to as ensembles. This terminology comes from meteorology, where forecasts with multiple scenarios are called ensemble forecasts. For instance, the ECMWF ensemble forecast has 51 scenarios, and the GFS ensemble forecast has 21 scenarios.

Period series

While Time series’ are excellent for representing fixed-interval data, some time series data can be stored and served more efficient.

For instance, there are plenty of capacity plans published in the power markets (i.e. REMIT). Another example is assumptions on installed capacity on different fuel types in the future. Such data often have the same value over an extended period, and the value changes sporadically.

So Energy Quantified created what we call a Periodseries for this, which is a collection of date-time ranges with a begin date-time, an end date-time, and a corresponding value.

The client also supports converting any such period series to a time series in your preferred resolution.

Example of a period series:

Begin       End          Value
----------  ----------  ------
2020-01-01  2020-01-05     300    #  4 days
2020-01-05  2020-02-01     125    # 27 days
2020-02-01  2020-02-13     160    # 12 days
2020-02-13  2020-02-14     220    #  1 day
...

Period-based series has two different types of periods:

  • Period with a value: Each period has one corresponding value, like in the example table above.

  • Period with a value and a capacity: Each period has a current value and a total installed capacity. These types of values appear mostly in REMIT data, where the value is the currently available production capacity, while the total installed capacity is provided for reference.

OHLC

End-of-day statistics for financial contracts. OHLC stands for open, high, low and close, and is a summary of trades for a day. OHLC is typically used to illustrate movements in the price of a financial instrument and can be seen in financial charts looking like candlesticks.

In cooperation Montel, Energy Quantified provides OHLC data for all power market regions in Europe, as well as prices for gas markets, carbon emissions (EUA), brent oil and coal (API2).

OHLC chart

Example: The Nasdaq Nordic Power Future contract for a front quarter contract (Q1).

SRMC

In the power market, the short-run marginal cost of running power plants. See short-run marginal cost on Wikipedia for a broader definition.

Energy Quantified allows SRMC calculations for gas- and coal-fired power production.

Next steps

Learn how to connect to the API and to discover data.