Known limitations of integration with Tulipa

Major

After merging !124 (merged), the integration with Tulipa will have limitations. Adding them here for reference and tracking

  1. Speed. New Tulipa model is created each time. To know what's going on, a progress bar would be nice. See #78.
  2. Tulipa assets have no memory. A new model is created from scratch with the initial configuration. This means storage levels are re-set after each optimization.
  3. Central market only runs for the bidding_window, thus ignores intertemporal constraints from non-annular storage assets. I think the current implementation of the Tulipa market could handle a longer optimization_window, though coupling.py does not.
  4. Neither market models work with curtailable bids.
  5. The Tulipa feature heavily depends on the the data model in Tulipa, and this has been changing frequently. They main problem this poses is that users need to use data from a Tulipa model that is compatible; this may make it harder to get started with annular when coming from Tulipa.

Smaller

  • Cluttered TODOs in the code - most important ones are collected here
  • Tests in test_tulipa_helpers.py::TestBuildModelWithBids are slow-ish (4s). They can be made faster with a class-scoped fixture that builds the database, and is reused across functions in the class. IIUC, because of fixture dependencies into conftest, it's not straightforward to change the scope of the fixture without changing the scope of the fixtures in conftest. I tried something along the lines of this but it'd take a bit more time and thought and I postponed for now.
  • Test coverage: reported coverage from the coverage job underestimates actual coverage b/c tulipa integration tests are not run. Running all tests, I get 84% 81%. Among new code, lowest coverage for
    • tulipa.py and build_tulipa_model.py: no tests for postprocessing logic after running julia; tests for read_outputs_from_db assume optimization window = bidding window.
    • install_tulipa
  • Inconsistencies between market implementations
    • The two functions do not expect the same arguments.
    • Tulipa does not store the market results. The base central market stores them in a csv file. Writing scheduled bids and market price should be relatively easy; generator dispatch more work because it's encoded in the Julia database. Long term it might be better to persist the Tulipa database for the entire simulation and then do downstream analytics directly off these data. On the market side, we store the entire database from Tulipa. For base, we store a simple csv file with dispatch per generator and satelltie.
  • file structure is a bit messy
    • rename src/annular/central_market to src/annular/market?
    • code for tulipa in src/annular/central_market is split somewhat arbitrarily across files
    • tests in test_market_model_module: some specifically test implementation of base market, others test market functionality (mostly TestMarketClearingSupplyBids) for both market implementations. splitting this would make the code base easier to navigate
  • potentially confusing naming for "timestep" for Tulipa and in "timestep" in sample_capacity_factors. see also #107
  • other things we observed: #146
  • Have not run mypy on the changes in the PR
Edited by Flavio Hafner