Errors while running westeros

[zsalimian]

Hi
How to tackle the following error
Regards

### ValueError Traceback (most recent call last)
in <module>
1 history = [690]
2 model_horizon = [700, 710, 720]
----> 3 scenario.add_horizon(
4 year=history + model_horizon,
5 firstmodelyear=model_horizon[0]

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\core.py in add_horizon(self, year, firstmodelyear, data)
365 existing = self.set("year").tolist()
366 if len(existing):
--> 367 raise ValueError(
368 f"Scenario has year={existing} and related values"
369 )

### ValueError: Scenario has year=[690, 700, 710, 720] and related values

[zsalimian]

Moreover, in addition I faced this error while running scenario.solve() in westeros tutorial example:

CalledProcessError Traceback (most recent call last)
in
----> 1 scenario.solve()

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\core.py in solve(self, model, solve_options, **kwargs)
547 :class:.GAMSModel.
548 """
**--> 549 super().solve(model=model, solve_options=solve_options, kwargs)
550
551 def add_macro(self, data, scenario=None, check_convergence=True, **kwargs):

**e:\users\zsalimian\documents\github\main\ixmp\ixmp\core.py in solve(self, model, callback, cb_kwargs, model_options)
1607 # Iterate until convergence
1608 while True:
-> 1609 model.run(self)
1610
1611 # Store an iteration number to help the callback

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\models.py in run(self, scenario)
117
118 try:
--> 119 result = super().run(scenario)
120 finally:
121 # Remove the optfile regardless of whether the run completed

e:\users\zsalimian\documents\github\main\ixmp\ixmp\model\gams.py in run(self, scenario)
146 # Invoke GAMS
147 cwd = self.temp_dir if self.use_temp_dir else model_file.parent
--> 148 check_call(command, shell=os.name == 'nt', cwd=cwd)
149
150 # Read model solution

e:\users\zsalimian\appdata\local\programs\python\python38\lib\subprocess.py in check_call(*popenargs, **kwargs)
362 if cmd is None:
363 cmd = popenargs[0]
--> 364 raise CalledProcessError(retcode, cmd)
365 return 0
366

CalledProcessError: Command 'gams "e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\MESSAGE_run.gms" --in="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\data\MsgData_Westeros_Electrified_baseline.gdx" --out="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\output\MsgOutput_Westeros_Electrified_baseline.gdx" --iter="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\output\MsgIterationReport_Westeros_Electrified_baseline.gdx" LogOption=4' returned non-zero exit status 2.

[zsalimian]

Last error happened while running scenario.var('OBJ')['lvl']

RuntimeError Traceback (most recent call last)
in
----> 1 scenario.var('OBJ')['lvl']

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\core.py in var(self, name, filters)
170 Filtered elements of the variable.
171 """
--> 172 return self._year_as_int(name, super().var(name, filters))
173
174 def cat_list(self, name):

e:\users\zsalimian\documents\github\main\ixmp\ixmp\core.py in var(self, name, filters, **kwargs)
1418 index names mapped list of index set elements
1419 """
-> 1420 return self._backend('item_get_elements', 'var', name, filters)
1421
1422 def equ_list(self):

e:\users\zsalimian\documents\github\main\ixmp\ixmp\core.py in _backend(self, method, *args, **kwargs)
485 still alive.
486 """
--> 487 return self.platform._backend(self, method, *args, **kwargs)
488
489 def del(self):

e:\users\zsalimian\documents\github\main\ixmp\ixmp\backend\base.py in call(self, obj, method, *args, **kwargs)
21 for the method name, e.g. 'ts_{method}'.
22 """
---> 23 return getattr(self, method)(obj, *args, **kwargs)
24
25 # Platform methods

e:\users\zsalimian\documents\github\main\ixmp\ixmp\backend\jdbc.py in item_get_elements(self, s, type, name, filters)
839
840 # Retrieve the item
--> 841 item = self._get_item(s, type, name, load=True)
842 idx_names = list(item.getIdxNames())
843 idx_sets = list(item.getIdxSets())

e:\users\zsalimian\documents\github\main\ixmp\ixmp\backend\jdbc.py in _get_item(self, s, ix_type, name, load)
1040 raise KeyError(name) from None
1041 else: # pragma: no cover
-> 1042 _raise_jexception(e)
1043
1044

e:\users\zsalimian\documents\github\main\ixmp\ixmp\backend\jdbc.py in _raise_jexception(exc, msg)
132 else:
133 msg += exc.message()
--> 134 raise RuntimeError(msg) from None
135
136

RuntimeError: unhandled Java exception: There exists no variable 'OBJ'!

[khaeru]

ValueError: Scenario has year=[690, 700, 710, 720] and related values

As with #416, the key thing is to read the documentation. Here is the documentation for add_horizon(), the function you are calling. In particular:

Raises: ValueError – If the year set of the Scenario is already populated.

Seeing this error indicates you are not running the notebook cells one-by-one starting from a fresh Jupyter kernel and clean scenario, but rather that you have already called add_horizon() at least once on this Scenario object. The tutorial code works if you run each cell, once, in order. If you make a mistake, you should try to restart the kernel.

The documentation page on prequisite knowledge & skills includes (emphasis added):

You should be able to: […] 8. Use a Jupyter notebook to run Python or R code, including: start the notebook server; open, restart, and close notebooks; create and edit cells.

The other errors, like in your second comment, likely also occur because you tried to run cells without successfully running the preceding cells. For instance, if the scenario has not been solved successfully, then scenario.var(…) will not work.

Please close this issue once you've managed to run the tutorial.

[zsalimian]

Dear Sir,
Thank you for your answer.
I changed the model_horizon from following default values
history = [690] model_horizon = [700, 710, 720]
to
history = [2000] model_horizon = [2010, 2020, 2030]
Surprisingly, it solved the above problem.

But Unfortunately, I still see the following error:
CalledProcessError Traceback (most recent call last)
in
----> 1 scenario.solve()

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\core.py in solve(self, model, solve_options, **kwargs)
547 :class:.GAMSModel.
548 """
--> 549 super().solve(model=model, solve_options=solve_options, **kwargs)
550
551 def add_macro(self, data, scenario=None, check_convergence=True, **kwargs):

e:\users\zsalimian\documents\github\main\ixmp\ixmp\core.py in solve(self, model, callback, cb_kwargs, **model_options)
1607 # Iterate until convergence
1608 while True:
-> 1609 model.run(self)
1610
1611 # Store an iteration number to help the callback

e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\models.py in run(self, scenario)
117
118 try:
--> 119 result = super().run(scenario)
120 finally:
121 # Remove the optfile regardless of whether the run completed

e:\users\zsalimian\documents\github\main\ixmp\ixmp\model\gams.py in run(self, scenario)
146 # Invoke GAMS
147 cwd = self.temp_dir if self.use_temp_dir else model_file.parent
--> 148 check_call(command, shell=os.name == 'nt', cwd=cwd)
149
150 # Read model solution

e:\users\zsalimian\appdata\local\programs\python\python38\lib\subprocess.py in check_call(*popenargs, **kwargs)
362 if cmd is None:
363 cmd = popenargs[0]
--> 364 raise CalledProcessError(retcode, cmd)
365 return 0
366

CalledProcessError: Command 'gams "e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\MESSAGE_run.gms" --in="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\data\MsgData_Westeros_Electrified_baseline.gdx" --out="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\output\MsgOutput_Westeros_Electrified_baseline.gdx" --iter="e:\users\zsalimian\documents\github\main\message_ix#install-from-source-advanced-users\message_ix\model\output\MsgIterationReport_Westeros_Electrified_baseline.gdx" LogOption=4' returned non-zero exit status 2.

[zsalimian]

I guess that may be I should introduce solve_options such as CPLEX to the super().solve(model=model, solve_options=solve_options, **kwargs) in core.py.

[khaeru]

I changed the model_horizon from following default values
history = [690] model_horizon = [700, 710, 720]
to
history = [2000] model_horizon = [2010, 2020, 2030]
Surprisingly, it solved the above problem.

This will not affect the cause of the error message you reported. I suspect you re-started/re-ran the notebook.

I should introduce solve_options such as CPLEX to the super().solve(model=model, solve_options=solve_options, **kwargs) in core.py.

Again, to be clear, you do not need to edit core.py or run it directly.

It is possible to pass solve_options when solving the model, but that is not necessary to run the tutorials.

CalledProcessError: Command 'gams […]' returned non-zero exit status 2.

Exit status 2 from GAMS indicates a compilation error: https://www.gams.com/latest/docs/UG_GAMSReturnCodes.html#UG_GAMSReturnCodes_ListOfErrorCodes

Is this a version of the GAMS file you have edited? If so, how? Can you attach the MESSAGE_run.lst file?

[khaeru]

@vahabmokari yours is not a "similar problem." The way to tell: the ModuleNotFoundError exception from your first message does not appear anywhere else in the error messages pasted by @zsalimian in the issue description and earlier comments.

Please:

• Open a new issue; don't hijack an existing, unrelated one.
• Read the GitHub documentation for writing issue descriptions, in particular how to use triple-backticks to put error message text inline:

  Like this!
  

  …and do not attach screenshots.

[panday1995]

Hi @khaeru, I also encountered CalledProcessError when processing the "austrian energy system modelling" example (I slightly extended the horizon to 2050). Then the code rised an error status of non-zero exit status 7.

I checked the GAMS link you attached above, which says it's a licensing error. However, I can indeed run the other demo "westeros". Also if I run the model with a horizon of 2010 to 2040, it worked. So I assume the maximum period that a free GAMS licenses (GAMS 29) allows is four periods (also I saw in the MESSAGEix doc that the free demon license only permit small models. Am I correct?

[khaeru]

@panday1995 that's roughly correct.

GAMS solvers for linear programs (LPs) are limited/licensed based on “problem size”. In the standard LP formulation, the decision variable x is a vector, and the limit is on the size, call it Nx, of this vector, e.g. Nx ≤ 1000.

The MESSAGE GAMS code, once translated by GAMS into a standard LP, has an Nx that depends on the number of technologies, nodes, years, etc. Increasing the number of elements in any set will increase Nx. So it is not true that the ‘year” set, specifically, must have ≤ 4 elements. Instead: in that particular model Nx < 1000 when there are 4 periods, and Nx > 1000 (or whatever threshold for the free license) when there are 5 periods. In a different, smaller model, e.g. Westeros, perhaps Nx ≤ 1000 even with 5 or more periods, so it could be solved with the free license.

Finally, thanks for looking through existing issues. As you found, even though CalledProcessError occurs, GAMS gives a different exit code. That means yours is actually a different issue than the one reported in this thread. (The code could make that a little more clear; see iiasa/ixmp#383.)

So, just as I asked @\vahabmokari above, please in the future file a new issue instead of commenting on an unrelated one.