Enforce consistency in objective dimensions

[tsmbland]

Description

See #581 for a description of the problem.

This PR enforces that all objectives must have an asset dimension. I've added these checks in the decorator, and modified the "comfort", "efficiency", "capital_costs" and "ALCOE" objectives accordingly, adding an asset dimension by broadcasting. The tests also needed updating.

Now, all objectives are consistent in the dimensions they return, apart from some of them which also return a timeslice dimension. However, this ends up being compressed by this line:

MUSE_OS/src/muse/investments.py

Line 289 in f94ee1b

costs = timeslice_max(costs)

So ultimately by the time the objectives reach the solver the dimensions are always the same: asset and replacement.

This seems to fix the problems described in the issue (see case study below), although I haven't really figured out why. I think it's to do with the scipy adapter (_unified_dataset and lp_constraint and lp_constraint_matrix functions), although the code is so unreadable it's hard to figure out what's going on. I'm not sure it matters though for now, as long as the issue is fixed.

To help enforce dimensions, I've created the check_dimensions helper function. This is checking both the inputs and outputs of the objective functions. I've also used this in the demand_share module, and there are probably many other places it could be used, but I'll leave it like this for now.

Case study

I've taken the default model and changed the objective to "capital_costs". I've also increased the capital costs of gasboiler 10x, so the expected outcome is clear - it should invest in heatpumps in the residential sector.

Scenario 1: original code, single objective (capital costs)

The solver fails to find a solution. This is strange as changing the objective is only supposed to change the costs vector, which shouldn't effect whether or not a solution is possible (this is all determined by the constraints)

Scenario 2: original code, adding a dummy second objective (LCOE, 0.01 weight)

Now the model runs to completion and the results are in line with expectations.

I think this works because when the objectives are combined, the asset dimension from LCOE gets picked up in the overall combined objective.

Scenario 3: new code, single objective (capital costs)

Now we get the expected result, even without adding a dummy objective

Fixes #581

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (non-breaking, back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (whatever its nature)

Key checklist

• All tests pass: $ python -m pytest
• The documentation builds and looks OK: $ python -m sphinx -b html docs docs/build

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[dalonsoa]

LGTM! It seems like a good runtime verification with minimal overhead.

[tsmbland]

@dalonsoa One thing I'm a bit unsure about is when it's better to use assert statements and when it's better to explicitly raise an error, like I'm doing here. There are lots of places in the code using assert to check dimensions, which seems like it's effectively the same, but is there an important distinction?

[dalonsoa]

There's nothing wrong with assert, but raising an error let you control better the type of error (so it is more informative to the user).

It also happens that assertions are ignored when running python in optimised mode (with the -O and -OO flags). I've never known anyone using these flags for anything, so I'm not sure if it is a big deal, but you should not trust the behaviour of your code to a feature that might be disabled by the users. In summary, assert should be used only in tests or for debugging purposes.