Skip to content

Dimensions#

A dimension is a set of labels that entries are indexed over, such as the hours in snapshot or the units in generator. This page lists the keys a dimension accepts, the closed label lists of one_of, the rule for the order of a dimension, and the members that a dimension lists under over.

dimensions#

Each key under dimensions: declares one dimension, and every dimension that the file uses anywhere is declared here.

dimensions:
  snapshot: { dtype: int, ordered: true }
  generator: { dtype: str }
parameters:
  load: { dims: [snapshot] }
variables:
  dispatch: { dims: [snapshot, generator] }
constraints:
  power_balance:
    dims: [snapshot]
    expression: sum(dispatch, over=generator) == load

snapshot has integer labels in a fixed order, and generator has string labels in no order. sum(dispatch, over=generator) removes generator, so power_balance has one row per snapshot.

The order of the keys under dimensions:, and the order of a dims: list, change nothing. mathspec sorts both by name at load, and an entry is indexed over its dimensions in that order. So dispatch above prints as \(\mathit{dispatch}_{g,t}\), and the legend lists generator first.

Key What it sets Default
dtype the type of a label: float, int, str or datetime str
one_of the labels a str dimension allows null
ordered whether the order of the labels is part of the spec (ordered) false
over the dimensions whose labels a label combines null
missing whether every combination of the members needs a label absent
description free text, which the typeset legend lists null

The file does not list the labels themselves. They arrive with the data, in the order that the data table lists them. With data attached, a piece of software that reads a spec (a tool) matches every entry indexed over the dimension to the labels.

mathspec loads a dimension that no entry uses, and for that dimension python -m mathspec check prints a message that does not refuse the file (advice).

A map from one dimension onto another is a relation, not a dimension.

one_of#

one_of lists every label that a str dimension allows. With one_of declared, mathspec checks at load the spelling of each label that a condition compares against.

dimensions:
  carrier: { dtype: str, one_of: [ccgt, ocgt, wind] }
  generator: { dtype: str }
relations:
  carrier_of: { key: generator, values: carrier }
variables:
  p: { dims: [generator], where: "carrier_of == 'wind'" }

p exists only at the generators whose carrier is wind. The relation carrier_of maps each generator to one carrier, and one_of makes wind one of three allowed labels.

A one_of list meets four rules:

  • Each label is a string, so mathspec refuses a number such as 2030 and accepts the string '2030'.
  • The list has at least one label, and each label once.
  • The dtype is str by default, so carrier: { one_of: [ccgt, ocgt, wind] } declares the same dimension. mathspec refuses a one_of with any other dtype.
  • In every other rule, the dimension is an ordinary str dimension.

The order of the list changes nothing. mathspec sorts it by name at load, so one_of: [wind, ccgt, ocgt] declares the same dimension.

At load, mathspec refuses a condition that compares the dimension, or a relation column over it, against a label that is not in the list. With carrier_of == 'gnrtr' in place of 'wind' above, mathspec prints:

Variable 'p': 'gnrtr' is not a label of 'carrier', which allows only ['ccgt', 'ocgt', 'wind']. Write one of them, or add 'gnrtr' to the one_of of 'carrier'.

mathspec lists the allowed labels and suggests two rewrites.

With data attached, a tool refuses a label outside the list, in the table of the dimension and in every relation column over it.

Only a dimension takes one_of, and a one_of key on a parameter is refused as an unknown key.

ordered#

ordered: true makes the order of the labels part of the spec. Five constructs use the order of a dimension, and each one requires ordered: true:

Construct What it uses
shift(x, along=d, offset=n), in an expression or a condition the label n positions back
sum_back(x, along=d, window=n) the last n labels
position(d) where a label sits along d
piecewise: with over: d which breakpoints are neighbours
a special ordered set (sos:) with type: 2 and along: d which labels are consecutive

The ramp constraint below steps back one snapshot, and snapshot is not declared ordered:

dimensions:
  snapshot: { dtype: int }
  generator: { dtype: str }
parameters:
  ramp_limit: { dims: [generator] }
variables:
  p: { dims: [snapshot, generator] }
constraints:
  ramp:
    dims: [snapshot, generator]
    expression: p - shift(p, along=snapshot, offset=1) <= ramp_limit
Constraint 'ramp': shift(along=snapshot) reads the order of 'snapshot', which is not declared ordered, so the order would come from the row order of the data. Declare 'snapshot: {ordered: true}' under dimensions:.
shift reads the label before along an ordered dimension The dimension snapshot is declared ordered, with labels 1, 2 and 3. The variable p is a table with three rows: generator wind at snapshot 1, 2 and 3. shift of p along snapshot with offset 1 is a table with two rows: at wind and snapshot 2 it holds p at wind and snapshot 1, and at wind and snapshot 3 it holds p at wind and snapshot 2. Snapshot 1 has no label before it, so shift has no row there and is absent. dimensions: snapshot: { dtype: int, ordered: true } p a variable generator snapshot wind1 wind2 wind3 offset=1 each label reads the one before it shift(p, along=snapshot, offset=1) an expression generator snapshot value wind2p[wind, 1] wind3p[wind, 2] no row for 1: no label before it absent: no row

The file loads once snapshot is declared { dtype: int, ordered: true }. With snapshots 1, 2 and 3 attached, shift has a row at snapshots 2 and 3 and none at 1.

An int dimension is not ordered unless it declares ordered: true.

An sos entry of type: 1 uses no order, so it needs no ordered: true.

A dimension with over: has no order, so mathspec refuses ordered: true on it.

edge='wrap' makes one shift cyclic, so that the first label steps back to the last. edge='wrap' does not change the dimension, and the dimension still needs ordered: true.

over#

over lists the dimensions (the members) whose labels a label of the dimension combines. A label is then one tuple of member labels, such as the unit u1 with the port out.

dimensions:
  unit: { dtype: str }
  port: { dtype: str }
  bus: { dtype: str }
  snapshot: { dtype: int, ordered: true }
  connection: { over: [unit, port] }
relations:
  connection_bus: { key: connection, values: bus }
parameters:
  capacity: { dims: [unit] }
  load: { dims: [bus, snapshot] }
variables:
  flow: { dims: [connection, snapshot], bounds: { lower: 0 } }
constraints:
  limit:
    dims: [connection, snapshot]
    expression: flow <= at(capacity, by=connection[unit])
  balance:
    dims: [bus, snapshot]
    expression: sum(flow, over=connection, by=connection_bus[bus]) == load

flow has one column per listed connection and snapshot, and none for a unit and a port that the data does not list together. limit reads the capacity of the unit of each connection, and balance adds each flow to the bus of its connection. The legend prints the dimension as \(\mathcal{C} \subseteq \mathcal{P} \times \mathcal{U}\), and limit prints as:

\[ \mathit{flow}_{c,t} \le \mathrm{capacity}_{\mathrm{connection.unit}(c)} \qquad \forall\, c \in \mathcal{C},\ t \in \mathcal{T} \]

over takes a list of dimensions, or a mapping of name to dimension where two members are over one dimension:

dimensions:
  region: { dtype: str }
  trade: { over: { source: region, target: region } }

A member in the list form has the name of its dimension. The order of the members changes nothing, because mathspec sorts them by name at load.

The relation of the dimension#

A dimension with members also declares a relation of the same name. Its key is the dimension, its values are the members, and it has a row for every label. So connection above is also the relation { key: connection, values: [unit, port] }, and a call reads a member as it reads any relation column:

Written Result
at(capacity, by=connection[unit]) the capacity of the unit of each connection
at(rating, by=connection[unit, port]) rating, a parameter over unit and port, at each tuple
at(flow, by=connection[unit, port]) flow at each unit and port that the dimension lists
sum(flow, over=connection, by=connection[port]) the flows of each port, added over its connections
where: "connection[unit] == 'u1'" the connections of the unit u1
where: "at(active, by=connection[unit])" the connections whose unit is in the condition active

The last at is on an operand over connection, so the dimension goes and its members arrive (at by the members of a dimension).

mathspec refuses a relation under relations: with the name of a dimension, so the file does not declare this relation a second time. A fact about a tuple that is not a member, such as the bus of a connection, is a relation keyed by the dimension, as connection_bus is above.

One dimension in every operator#

A dimension with members is one dimension wherever a dims: list or an operator names it. mathspec does not align an entry over a member with an entry over the dimension. With flow <= capacity in limit above, mathspec prints:

Constraint 'limit': an operand over ['unit'] meets one over 'connection'. 'unit' is a member of 'connection', which is one dimension here. Read the operand at each tuple: at(capacity, by=connection[unit]), or sum flow to the member: sum(flow, over=connection, by=connection[unit]).

mathspec suggests the lookup through the relation of the dimension, or the sum of flow to the member. The same refusal applies wherever the two operands meet: in a product, in a sum, and in the objective, as in sum(flow * capacity). An entry over both, such as a variable over [connection, unit], loads.

mathspec refuses the meeting unless the dims: of the entry lists both the dimension and the member. Then the two operands meet as their cross product, at every pair of a connection and a unit. With dims: [connection, unit, snapshot], flow <= capacity loads, with one row for each connection, unit and snapshot. Inside a sum, a dimension that the sum adds up is not in the dims: of the entry, so sum(flow * capacity, over=unit) is refused in that entry too. The objective and an assumption have no dims:, so mathspec refuses the meeting in each of them. It also refuses the meeting in the where: of a piecewise: entry, because mathspec repeats that condition in assumptions.

A sum over a member is refused the same way. With sum(flow, over=unit), mathspec prints:

Constraint 'balance': sum(over=unit) but the expression has only the dims ['connection', 'snapshot']. 'unit' is a member of 'connection', which is one dimension here. Write sum(..., over=connection, by=connection[port]).

mathspec suggests the sum over the dimension, grouped by the other member.

A dims: list is the domain of an entry, so a tool refuses a data row at a tuple that the dimension does not list. A where: condition then removes rows of that domain.

At load, mathspec also refuses each of these, and prints the rewrite:

Refused What you write instead
a member that has members of its own the members themselves, in one list
a dimension twice in the list form, as in over: [region, region] the mapping form, with a name for each member
a member named after a dimension that it is not over another name for the member
ordered: true, dtype: or one_of: beside over: the key on a member
shift, sum_back, position, piecewise: or sos: of type: 2 along the dimension the construct along an ordered member, as described below this table
a comparison of the dimension with a label, as in connection == 'a' a comparison of a member, as in connection[unit] == 'u1'
a call through the relation of the dimension on an operand over both the dimension and a member the member summed out of the operand first

A tuple has no order, and a member can declare one. shift and sum_back step along an ordered member of an operand that at has moved to the members. A second at reads the result back at each tuple. Here assignment lists the periods in which each unit is active, and carry reads the level of the same unit one period earlier:

dimensions:
  unit: { dtype: str }
  period: { dtype: int, ordered: true }
  assignment: { over: [unit, period] }
variables:
  level: { dims: [assignment], bounds: { lower: 0 } }
  charge: { dims: [assignment], bounds: { lower: 0 } }
constraints:
  carry:
    dims: [assignment]
    expression: level == at(shift(at(level, by=assignment[unit, period]), along=period, offset=1), by=assignment[unit, period]) + charge
\[ \mathit{level}_{a} = \mathit{level}_{\left( \mathrm{assignment.period}(a) - 1,\ \mathrm{assignment.unit}(a) \right)} + \mathit{charge}_{a} \qquad \forall\, a \in \mathcal{A} \]

carry has no row for an assignment whose unit has no assignment one period earlier, because the shifted level is absent there.

position, piecewise: and sos: take no operand that at can move. An entry that uses one of them along a member is over the members, and the set of tuples is a bare relation over unit and period. Such an entry uses the relation as where: active, and an entry over connection and period reads it as where: "at(active, by=connection[unit])".

Two dimensions that share a member#

Two dimensions with members can share one, as connection and assignment share unit below. An entry over both is their cross product, with one row for each pair of a connection and an assignment. A where: that compares the shared member keeps the pairs of one unit, which is the inner join of the two:

dimensions:
  unit: { dtype: str }
  port: { dtype: str }
  period: { dtype: int, ordered: true }
  connection: { over: [unit, port] }
  assignment: { over: [unit, period] }
variables:
  sent: { dims: [connection] }
  busy: { dims: [assignment] }
constraints:
  exclusive:
    dims: [connection, assignment]
    where: connection[unit] == assignment[unit]
    expression: sent + busy <= 1

mathspec reads each side through the relation of its own dimension, and the dims: of the entry lists both, so the comparison loads (conditions). exclusive prints as:

\[ \mathit{sent}_{c} + \mathit{busy}_{a} \le 1 \qquad \forall\, a \in \mathcal{A},\ c \in \mathcal{C} \,:\, \mathrm{connection.unit}(c) = \mathrm{assignment.unit}(a) \]

An entry over connection and period reads busy in two steps. The inner at moves busy from assignment to its members, and the outer at reads the result at the unit of each connection. The inner at is absent where assignment lists no tuple, and a sum by the same columns is 0 there:

dimensions:
  unit: { dtype: str }
  port: { dtype: str }
  period: { dtype: int, ordered: true }
  connection: { over: [unit, port] }
  assignment: { over: [unit, period] }
variables:
  sent: { dims: [connection, period] }
  busy: { dims: [assignment] }
constraints:
  only_when_busy:
    dims: [connection, period]
    expression: sent <= at(at(busy, by=assignment[unit, period]), by=connection[unit])
  idle_sends_nothing:
    dims: [connection, period]
    expression: sent <= at(sum(busy, over=assignment, by=assignment[unit, period]), by=connection[unit])
\[ \mathit{sent}_{c,p} \le \mathit{busy}_{\left( p,\ \mathrm{connection.unit}(c) \right)} \qquad \forall\, c \in \mathcal{C},\ p \in \mathcal{P} \]
\[ \mathit{sent}_{c,p} \le \sum_{a \in \mathcal{A} \,:\, \mathrm{assignment.period}(a) = p \wedge \mathrm{assignment.unit}(a) = \mathrm{connection.unit}(c)} \mathit{busy}_{a} \qquad \forall\, c \in \mathcal{C},\ p \in \mathcal{P} \]

In only_when_busy, the absent busy stands outside any sum, so the entry has no row where the unit of the connection has no assignment in that period. Its rows are those of the inner join, and sent is free elsewhere. In idle_sends_nothing, the sum has no term there and is 0. The entry keeps every connection and period, which is the left join, and holds sent at 0 where nothing matches.

The language has no outer join, which would keep the tuples of either dimension. Write such an entry over the members, as dims: [unit, port, period], with the two sets of tuples as the bare relations plugged over [unit, port] and active over [unit, period]. Then where: "plugged OR active" keeps the rows of either.

missing#

missing: on a dimension with members sets whether every combination of member labels needs a label.

missing: Result
absent, by default the data lists the tuples that exist, and a combination that it leaves out is not in the dimension
refused a tool refuses data that leaves a combination out, so the dimension is the full product of its members

With missing: refused, the legend prints the dimension with \(=\) in place of \(\subseteq\). mathspec refuses missing: on a dimension without over:.

The data contract#

With data attached, the labels of connection are one column whose values have one field per member, named after the member: {unit: u1, port: out}. The relation of the dimension takes its rows from this column, so a tool reads no table of its own for it. A tool requires three things of the column:

  • Each tuple appears once.
  • Every field is a label of its dimension.
  • Under missing: refused, every combination of member labels appears.

mathspec declares no type for the tuple, so a tool may store the label in any form that keeps the fields.

See also