Fix commodity price calculations

[tsmbland]

Description

This PR aims to address a number of issues in the costs module to make the code cleaner and fix some incorrect calculations.

The main motivation for this was that models are currently particularly sensitive to the scaling of technology inputs and outputs, and often fail spectacularly if the output quantity isn't 1 (#552). In reality, we'd expect the scaling of inputs/outputs in the CommIn/CommOut files to have very little impact on the results, as long as inputs and outputs are scaled together - in other words it's the ratio that matters and not the absolute values. I've added a test to prove that this is fixed (at least for the LCOE calculation).

I've also noticed that #607 to do with the timeslice level is fixed, which is a good sign

Detailed changes below.

The results have now changed for all models, particularly commodity prices which now look a bit closer to what's expected as far as I can tell (electricity and gas prices more closely match the values in the projections file, which I assume are at least on the expected order of magnitude). I can't exactly pinpoint which changes are most responsible for changing the results as there's a lot going on. I'm actually a bit surprised that prices have changed this much as none of these models have technologies with output quantities not equal to 1. But I've double checked and can't see any mistakes in what I've done, so I have to assume it's all fine.

Note: We're currently still calculating costs at the timeslice level, so this will all change again. But it will be based on what I've done here, so this can still be reviewed and merged.

Main changes

Combining lifetime and annual LCOE into the same function

• Capital costs are calculated for the lifetime of the technology (a one time cost), whereas running costs, production and revenue are calculated for a single year
• If the "annual" method is being used, capital costs are annualised by multiplying by the capital recovery factor
• If the "lifetime" option is being used, running costs and production are broadcasted across the lifetime of the technology, multiplied by the discount factor for each year, and then summed across years
• Annual LCOE is now based on actual capacity, production and consumption data, rather than effectively assuming that these were all equal to 1

Separate functions for cost components (e.g. fuel costs, environmental costs)

• Reduces code duplication in the costs module
• Can also be used in the objectives module to avoid duplicating the calculations here.

Cost functions only take and return data for a single year

• Since Limit prices used for objectives to investment year #602, there's no need to pass data to these functions for a span of years. All costs are calculated based on data for a single year (usually the investment year).
• I'm therefore mandating that these functions are only passed data for a single year. This simplifies some aspects of the code (e.g. no interpolation), and avoids potentially unwanted broadcasting if some inputs have a year dimension and some don't
• I'm not actually checking whether the inputs have a year dimension, I'm just checking that the output doesn't have a year dimension (using a new decorator), which it would do if any arrays used in the calculation had a year dimension.
• There's no longer need for a year argument for any of the cost functions

Passing consumption data to cost functions rather than calculating it internally

• I think it gives greater separation of concerns if this data is calculated externally and passed to the function
• It also allows us to use actual consumption data in some circumstances (especially the calculation of commodity prices) rather than always using made-up data
• This leads to slightly more repetitive code in the objectives module, as consumption is now calculated here, but I think it's worth it - and this could probably be simplified later

There was a + 1 applied to the year argument for the discount factor calculations (here), which meant that the discount factor was being applied in the first year of the technology lifetime. I think this is incorrect, so I've remove this + 1 (i.e. year = 0 for the first year, so discount factor = 1)

Fixes #552
Fixes #607

Type of change

• New feature (non-breaking change which adds functionality)
• Optimization (non-breaking, back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (whatever its nature)

Key checklist

• All tests pass: $ python -m pytest
• The documentation builds and looks OK: $ python -m sphinx -b html docs docs/build

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[tsmbland]

I suspect there may be a mistake in the treatment of regions, as I've noticed that electricity prices are unexpectedly high for the example models with multiple regions. I'm looking into it...

[dalonsoa]

I'm holding the review until you have sorted the issue with the regions.

[tsmbland]

@dalonsoa As far as I can tell it's not something I've done wrong in this PR, but the treatment of regions is generally a mess. It brings me back to issue #475

I think generally the problem is this:

In arrays with an "asset" dimension (e.g. supply, consumption), "region" should always be a property of the asset (not sure if that's the correct xarray term) rather than its own dimension. This is because each asset belongs to a specific region. However, the supply data passed to the LCOE function has "region" and "asset" as separate dimension, in which all the assets in each region have been copied across both regions. Ultimately LCOE is ending up approximately twice as high in the the model with two regions, which carries through to commodity prices.

Basically, I don't think there's any good reason for any array to have both "asset" and "region" dimensions. I might try adding a check to the patched_broadcast_compat_data function and that should flag where the problems are throughout the code. But I don't know how easy it will be to fix.

I suspect the reason prices were previously similar in multi-region models was just a case of two bugs cancelling each other out. I suspect one of the bugs was the same one that was causing #607, which is now fixed, although I haven't fully figured out why.

In any case, I think you can go ahead and review this PR when you have the time.

[tsmbland]

@dalonsoa #609 builds on this (and I think simplifies things overall), so it might be worth reviewing them both together

[dalonsoa]

I've made a couple of comments/questions, but this looks really good. A much simplified and easy to follow code - and therefore, easier to figure out if things go wrong in the future.

[dalonsoa]

Assuming that the results are correct - which seem to be the case for what you say - this looks way, way better than it was before. Much clearer code, much clear execution pipeline, and much clearer explanations. Nothing to add, here, to be honest.

[dalonsoa]

I guess that coming up with tests that check some other properties of the outputs of the cost functions, besides the dimensions, is tricky, right?

[tsmbland]

Yeah it's a bit tricky. A lot of the test elsewhere just copy code directly from the respective function and check that the output is the same - which is completely pointless. I've added some tests for LCOE which demonstrate how LCOE is expected to change (or not) as the inputs are changed. I couldn't really think of similar tests for the other costs (or didn't try that hard), but since LCOE uses a lot of the individual cost components I think the coverage is still pretty good.