Docker file

[ndangtt]

The current docker file should be updated:

1. Use all binaries that are accompanied in the conjure's docker image. This means that we'll no longer call those install-<software>.sh scripts in the docker file.

Questions: what are provided in the conjure image and what are missing?

2. For the missing ones, update the installation script if necessary.

[ozgurakgun]

This is how the docker image is built, you can see a list of solvers there. https://github.com/conjure-cp/conjure/blob/main/Dockerfile There is an issue with gecode so that’s commented out. We also have another version that has cplex in it but we don’t publish it due to license. I can show you how we set it up if you need cplex. Özgür Akgün

…

On Sat, 2 Nov 2024 at 9:43 pm, Nguyen Dang ***@***.***> wrote: The current docker file <https://github.com/stacs-cp/AutoIG/blob/dev/Dockerfile> should be updated: 1. Use all binaries that are accompanied in the conjure's docker image. This means that we'll no longer call those install-<software>.sh scripts in the docker file. Questions: what are provided in the conjure image and what are missing? 2. For the missing ones, update the installation script if necessary. — Reply to this email directly, view it on GitHub <#7>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABNEUTI6YN6BKC3KULR4O3Z6VBQTAVCNFSM6AAAAABRCCSGQCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGYZTAOBRGM2TEMI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[vincepick]

I am happy to look through and see which of those install-<software>.sh scripts no longer need to be called and update the current Dockerfile. Will keep you updated on how it goes.

Looks like Savilerow, Minizinc, Runsolver, and ortools are already in the Conjure base image, while iRace, Yuck, and Picat are not.

Me and @ndangtt have not discussed using cplex or gecode at all, and as far as I am aware I don't think we were planning to use either soon, so for now I will just use the version at: https://github.com/conjure-cp/conjure/blob/main/Dockerfile. If the plan was to eventually use cplex though when me and Tudor analyze AutoIG runs, I would be happy to try to get that working now as well.

[ndangtt]

thanks very much, @ozgurakgun!

@vincepick: thanks very much for checking it! I think there are a couple of things that would potentially need to be updated when switching:

• For the minizinc experiments, the installation of both yuck and picat requires setting them up properly within the minizinc's installation folder. We need to make sure that is properly done when switching, otherwise it will break the current minizinc support.
• Similarly, for ortools and chuffed, if we want minizinc to see them properly, we may need to make sure the setup is done properly.
• In general, I think it'd be a nice feature to create a number of automated test cases to ensure any changes in the setup doesn't break current MiniZinc support (and the Essence support we're adding). This would be a useful feature in the list of project tasks that you and Tudor are planning to create.

[vincepick]

I think that some automated test cases would be a great idea, and would be happy to make some over the next week. I think it would be especially good to do as we continue with development, so we can check if our developments actually work or inadvertently break something else, (an issue I have already bumped into with some of my changes which I then had to revert).

For the issue of converting the Docker, I thought it would be relatively straightforward but seems much more difficult than I originally thought. With the Way the existing AutoIG installations are made, many of the installation scripts require previous programs to already be installed in the directory, such as the installation of Yuck and Picat as you discussed. While my implementation of pointing to the Conjure binaries (seemed) to work within the container, I am not sure how to point those installation scripts in AutoIG to an existing MiniZinc within a Conjure base image binary which has not been created yet. I am not sure what the limitations of the HPC cluster are, but my understanding was that its usage was tracked by CPU hours, so if the storage size is not as big of an issue as the time spent, then perhaps I should focus on another aspect of the project, such as those automated test cases, and circle back at a later date. I would be happy to discuss this further in our next meeting, and to this comment I have attached an example of one of the errors that came up with my most recent attempt at a new Docker container (occurred when building the Docker container).

[vincepick]

@ndangtt I have made some progress on the Docker file for preventing redundant dependency installations, and am aiming to have it setup hopefully by the VIP meeting this Wednesday.

However, a related new issue I have run into is that my AutoIG container seems to be setup correctly, but not actually successful in runs. I tried running it again with a fresh version of the Repo pulled from main, but ran into some unexpected issues I haven't encountered before. The run.sh script for macc-graded first stopped running due to an error, where a file ./detailed-output/inst-1-1965286212.dzn.chuffed.runsolver wasn't generated correctly then the program failed when it wasn't found.

I added some more installs to the container and manually ran the install-runsolver script again, and believe I have bypassed that issue. However, I am now having a new issue I have never seen before in which no error is thrown, but the program just stops prematurely when running the original macc-graded example. When I run the individual python script generated, it seems to run correctly with no issues, which is why I am confused and unsure about how to go about debugging this. I cross-referenced my project directory with Tudors and checked for any potential discrepancies, but wasn't able to spot anything like a missing file or a different version of anything. I also checked the versions of the different programs (such as ensuring minizinc was updated and version 2.8.5 since I ran into issues with that before), and found nothing that stood out.

Any advice you have or tips about what direction to go in would be great. I believe there will likely be some missing dependency in my container which causes this, and if you have any idea off the top of your head from past experience as to which one it may be that would be fantastic.

Thank you,
Vincent

Bash run.sh issue.

Generated Python Script issue.

[vincepick]

After examining it more, I have also noticed that when the experiment is run multiple times consecutively, it begins to run more iterations before crashing. This is similar to some of the behavior me and Tudor noticed with simple.essence, but I don't think that we ever found a specific cause/fix for it.

After a few more consecutive calls of run.sh:

[ndangtt]

@vincepick:

The output seems to indicate that your target-runner is printing out the screen both the python command (python3 ...) and the score + time. irace needs the target-runner to only print out one line (the score + time). Perhaps you haven't commented out the echo $cmd line in the target-runner?

[vincepick]

I didn't realize that having target-runner output the command may interfere with the behavior of the framework overall. I tried running it again with the command echoing disabled and nothing changed.

I am happy to say though that I managed to fix the issue, and get it to run successfully again by continuing to experiment with the build process. Hopefully, progress after this point will be smoother, and I am planning to have the container done soon.

[ndangtt]

@vincepick: glad to hear that the issue is fixed, what change did you make to fix it?