Make sure correct conda channels are used

[alneberg]

A quick scan gave me these:

miarka-provision/roles/ngi_pipeline/tasks/main.yml

Line 12 in c29575c

shell: "conda create -n {{ NGI_venv_py2_name }} -c anaconda python={{ python2_version }}"

• Using -c anaconda is not good. Do we even need the python 2 environment still?

miarka-provision/roles/func_accounts/tasks/main.yml

Line 4 in c29575c

shell: conda install -n {{ NGI_venv_name }} supervisor

• Let's check if the supervisor can be installed from a different channel

miarka-provision/roles/anaconda/tasks/main.yml

Line 14 in c29575c

shell: "{{ anaconda_path }}/bin/conda update -n base -c defaults conda -y"

• Maybe possible to use different channel to update conda?

miarka-provision/roles/anaconda/defaults/main.yml

Line 1 in c29575c

anaconda_file: Anaconda3-2021.05-Linux-x86_64.sh

• Should we consider switching to miniconda? I'm not sure if it's necessary for licensing but it might be beneficial for speed of download anyway?

[Aratz]

Miniforge looks promising! I'm doing some tests now and might submit a PR later.

[Aratz]

After gathering some information, here are some additional remarks:

• according to Anaconda, these licensing policies have been in place since 2020, although by then it was unclear if the 200-employee threshold applied to academics too. This was clarified earlier this year. (This was an interesting read: https://www.theregister.com/2024/08/08/anaconda_puts_the_squeeze_on/)
• from the clarification issued by Anaconda (https://www.anaconda.com/blog/update-on-anacondas-terms-of-service-for-academia-and-research), it seems to be that we are allowed to use the installer as long as we don't pull packages from the default channel
• However, the terms of service seems to indicate the opposite, as the "Anaconda Distribution installer" and "Miniconda", are listed as "Offerings", which would fall under the following paragraph:

  2.1 Organizational Use. Your registration, download, use, installation, access, or enjoyment of all Anaconda Offerings on behalf of an organization that has two hundred (200) or more employees or contractors (“Organizational Use”) requires a paid license of Anaconda Business or Anaconda Enterprise.

• Furthermore, just creating python environments with Miniconda does include a bunch of packages from Anaconda's default channel, which as far as I understand is a big no-no for us. There might be a way to configure it not to do so, but at this point we might just be better off with another installer.
• As far as I understand, using an installer that is based on eg. the conda-forge channel from the start, such as miniforge, seems to be a safe option.
• However, as of today, the conda-forge channel is hosted by anaconda, so there is always a risk they discontinue that in the future
• Overall, conda seems to be a complete legal minefield, if it was at all possible I'd recommend staying away from it entirely

[alneberg]

Thank you @Aratz! Do you think it would be worth looking into completely different virtual environment solutions? I don't really find any clear evidence that anaconda hosts conda-forge? Conda-forge is available on the anaconda website but I think it's rather that they display all conda-recipes they can find?

[Aratz]

Do you think it would be worth looking into completely different virtual environment solutions?

Since we need to update the playbooks anyways, yes it might be worth checking out alternatives now. I know at SNP&SEQ we mostly use conda to be able to run a program with a specific python version, so in that respect pyenv may be a good alternative.

EDIT: I see now that we use conda to set up our nf-core pipelines, I need to look deeper bit it might be harder than I initially thought to get rid of this dependency.
It seems this is only to install the nf-core/tools software. But tbh I find it difficult to say what is actually dependent on conda and what just requires a regular python environment. Maybe the best is just to try on bimonthly and make sure we can still run everything we need...

I don't really find any clear evidence that anaconda hosts conda-forge? Conda-forge is available on the anaconda website but I think it's rather that they display all conda-recipes they can find?

This was mentioned in an interview with Anaconda's CEO a couple of years ago:

Not everybody is happy with the change to the terms of service. Some accuse Anaconda of using bait-and-switch techniques, of trying to monetize the hard work of other open source developers. Others have threatened to abandon Anaconda and use other Python packages, including Conda Forge, a GitHub-based community that distributes individual components in the Conda package manager.

That last one gets a chuckle out of Wang. “They don’t realize Conda Forge is hosted on our infrastructure,” he says. “I pay $80,000 to $100,000 [per month] to support the download volume of Conda Forge and the software infrastructure.”

https://www.datanami.com/2022/04/18/anacondas-commercial-fee-is-paying-off-ceo-says/