Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ray 1.4.1 #28

Merged
merged 15 commits into from
Aug 10, 2021
8 changes: 4 additions & 4 deletions .azure-pipelines/azure-pipelines-linux.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion .azure-pipelines/azure-pipelines-win.yml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 6 additions & 2 deletions .ci_support/linux_64_python3.6.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ c_compiler:
c_compiler_version:
- '9'
cdt_name:
- cos6
- cos7
channel_sources:
- conda-forge,defaults
channel_targets:
Expand All @@ -15,7 +15,11 @@ cxx_compiler:
cxx_compiler_version:
- '9'
docker_image:
- quay.io/condaforge/linux-anvil-comp7
- quay.io/condaforge/linux-anvil-cos7-x86_64
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
8 changes: 6 additions & 2 deletions .ci_support/linux_64_python3.7.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ c_compiler:
c_compiler_version:
- '9'
cdt_name:
- cos6
- cos7
channel_sources:
- conda-forge,defaults
channel_targets:
Expand All @@ -15,7 +15,11 @@ cxx_compiler:
cxx_compiler_version:
- '9'
docker_image:
- quay.io/condaforge/linux-anvil-comp7
- quay.io/condaforge/linux-anvil-cos7-x86_64
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
8 changes: 6 additions & 2 deletions .ci_support/linux_64_python3.8.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ c_compiler:
c_compiler_version:
- '9'
cdt_name:
- cos6
- cos7
channel_sources:
- conda-forge,defaults
channel_targets:
Expand All @@ -15,7 +15,11 @@ cxx_compiler:
cxx_compiler_version:
- '9'
docker_image:
- quay.io/condaforge/linux-anvil-comp7
- quay.io/condaforge/linux-anvil-cos7-x86_64
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
8 changes: 6 additions & 2 deletions .ci_support/linux_64_python3.9.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ c_compiler:
c_compiler_version:
- '9'
cdt_name:
- cos6
- cos7
channel_sources:
- conda-forge,defaults
channel_targets:
Expand All @@ -15,7 +15,11 @@ cxx_compiler:
cxx_compiler_version:
- '9'
docker_image:
- quay.io/condaforge/linux-anvil-comp7
- quay.io/condaforge/linux-anvil-cos7-x86_64
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
4 changes: 4 additions & 0 deletions .ci_support/win_64_python3.6.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ curl:
- '7'
cxx_compiler:
- vs2017
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
4 changes: 4 additions & 0 deletions .ci_support/win_64_python3.7.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ curl:
- '7'
cxx_compiler:
- vs2017
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
4 changes: 4 additions & 0 deletions .ci_support/win_64_python3.8.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ curl:
- '7'
cxx_compiler:
- vs2017
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
4 changes: 4 additions & 0 deletions .ci_support/win_64_python3.9.____cpython.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,10 @@ curl:
- '7'
cxx_compiler:
- vs2017
nodejs:
- '12'
- '14'
- '15'
pin_run_as_build:
curl:
max_pin: x
Expand Down
2 changes: 1 addition & 1 deletion .github/CODEOWNERS

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions README.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

8 changes: 8 additions & 0 deletions recipe/conda_build_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
cuda_compiler_version: # [linux64]
- None # [linux64]
docker_image: # [linux64]
- quay.io/condaforge/linux-anvil-cos7-x86_64 # [linux64]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so there is no way to stay on cos6?

Copy link
Member

@h-vetinari h-vetinari Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, we cannot, see this comment, specifically:

By adding the nodejs-dependency, we won't be able to avoid it here: https://github.com/conda-forge/nodejs-feedstock/blob/master/recipe/meta.yaml#L33

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But we can use older nodejs, e.g. one from 14.x version part, which is still in "active" phase according to their schedule and works on cos6 :)

Copy link
Member

@h-vetinari h-vetinari Aug 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, this comment didn't show up in the main thread for some reason.

I'm strongly against pinning to old versions for the sake of EOL distros.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you feel so strongly about cos6-support, we can add a branch where nodejs is pinned to 14.x, but master must move on. (though this would need some care with the build numbers; e.g. build numbers on master start from 100)

cudnn: # [linux64]
- undefined # [linux64]
cdt_name: # [linux64]
- cos7 # [linux64]
12 changes: 7 additions & 5 deletions recipe/meta.yaml
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
{% set version = "1.4.0" %}
{% set version = "1.4.1" %}

package:
name: ray-packages
version: {{ version }}

source:
url: https://github.com/ray-project/ray/archive/ray-{{ version }}.tar.gz
sha256: 1ad04845dad6824220ad7750d666e7186f6b9186fb99294aae7d0e35f8e75018
sha256: a5fcaeb405fc9e6ddf084d55d202f72c919e974dac736d0426a8b569f7022e4a
patches:
- patches/0001-Do-not-force-pickle5-in-sys.path.patch
- patches/0002-Fix-redis-build-for-non-default-compiler-drivers.patch
Expand All @@ -18,9 +18,9 @@ source:
- patches/0008-Add-workaround-for-os.path.isdir-on-Windows.patch
- patches/0009-Include-process.h-for-getpid-explicitly-on-Windows.patch
- patches/0010-Empty-install-requirements-installed-by-conda-build.patch
- patches/0011-permit-python-3.9.patch
- patches/0012-Ignore-warnings-in-event.cc-and-logging.cc.patch
- patches/0013-Disable-runfiles-on-windows.patch
- patches/0011-Ignore-warnings-in-event.cc-and-logging.cc.patch
- patches/0012-Disable-runfiles-on-windows.patch
- patches/0013-Add-bazel-linkopts-libs.patch


build:
Expand Down Expand Up @@ -71,6 +71,7 @@ outputs:
- {{ compiler('cxx') }}
- bazel <=3.4.1 # [not win]
- bazel <3.7 # [win]
- sysroot_linux-64 2.17 # [linux64]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we update the base to cos7 do we need to manually add this?..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's necessary so that this package gets the correct run-export (so that packages depending on ray fail to build if they don't have cos7 themselves), otherwise we might get crashes with the ABI.

- curl
- cython >=0.29
- make
Expand Down Expand Up @@ -305,3 +306,4 @@ extra:
- dHannasch
- h-vetinari
- vnlitvinov
- krfricke
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

welcome on board! 🙃

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, already mentioned that on the last PR - happy to have you on board!

6 changes: 3 additions & 3 deletions recipe/patches/0001-Do-not-force-pickle5-in-sys.path.patch
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 578717348bb90cd268d339e94b9302d349012929 Mon Sep 17 00:00:00 2001
From e1c5c21f5a9c254e98a8f10a467a181f9bec04bc Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <[email protected]>
Date: Thu, 5 Nov 2020 12:04:25 +0300
Subject: [PATCH 01/13] Do not force pickle5 in sys.path
Expand All @@ -9,7 +9,7 @@ Signed-off-by: Vasily Litvinov <[email protected]>
1 file changed, 6 deletions(-)

diff --git a/python/ray/__init__.py b/python/ray/__init__.py
index 4f18b9411..45753e5a1 100644
index 7c5eb9413..918000ce4 100644
--- a/python/ray/__init__.py
+++ b/python/ray/__init__.py
@@ -33,12 +33,6 @@ def _configure_system():
Expand All @@ -26,5 +26,5 @@ index 4f18b9411..45753e5a1 100644
# initialized.
thirdparty_files = os.path.join(
--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 8936e498f38abd5d12bf0e970f7ddcf276f453f9 Mon Sep 17 00:00:00 2001
From 2ad3ba43d6f33a9e18b86a5680e15c816b21f980 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <[email protected]>
Date: Thu, 5 Nov 2020 17:05:44 +0300
Subject: [PATCH 02/13] Fix redis build for non-default compiler drivers
Expand Down Expand Up @@ -29,5 +29,5 @@ index f428f7b36..9ec69d433 100644

# This library is for internal hiredis use, because hiredis assumes a
--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 311235bb948258691dbee9635e366fffc7f0ed3d Mon Sep 17 00:00:00 2001
From 2fa1e41b13b0caacabda3cac51db475350321f3e Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <[email protected]>
Date: Fri, 6 Nov 2020 00:55:05 +0300
Subject: [PATCH 03/13] Redis deps now build but do not link
Expand Down Expand Up @@ -142,5 +142,5 @@ index 000000000..36497c4b6
+2.11.0
+
--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 8a1a5c64bd41172ebc4fac39cc93baac56137cac Mon Sep 17 00:00:00 2001
From 4d5363d215b7373d207021032655282e41e9e143 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <[email protected]>
Date: Tue, 10 Nov 2020 23:26:35 +0300
Subject: [PATCH 04/13] Disable making non-core entry scripts
Expand All @@ -8,10 +8,10 @@ Subject: [PATCH 04/13] Disable making non-core entry scripts
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/python/setup.py b/python/setup.py
index 590020158..ea6925e25 100644
index 577373e70..cb79b08ca 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -444,10 +444,10 @@ setuptools.setup(
@@ -457,10 +457,10 @@ setuptools.setup(
entry_points={
"console_scripts": [
"ray=ray.scripts.scripts:main",
Expand All @@ -27,5 +27,5 @@ index 590020158..ea6925e25 100644
},
include_package_data=True,
--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From 1bb7ed73f7b894e84e5e638a4a9f793b99f8c229 Mon Sep 17 00:00:00 2001
From bfb149410ec41c4e1062a317d4f52374e5ace836 Mon Sep 17 00:00:00 2001
From: Vasily Litvinov <[email protected]>
Date: Wed, 18 Nov 2020 18:47:40 +0300
Subject: [PATCH 05/13] Contain bazel root and output dir right above $SRC_DIR
Expand All @@ -9,10 +9,10 @@ Signed-off-by: Vasily Litvinov <[email protected]>
1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/python/setup.py b/python/setup.py
index ea6925e25..e714d988d 100644
index cb79b08ca..2cfe6f609 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -285,12 +285,18 @@ def build(build_python, build_java):
@@ -286,12 +286,18 @@ def build(build_python, build_java):
logger.warning("Expected Bazel version {} but found {}".format(
".".join(map(str, SUPPORTED_BAZEL)), bazel_version_str))

Expand All @@ -33,5 +33,5 @@ index ea6925e25..e714d988d 100644


--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From d0ead439ac7e531f66e867eb19a06c90a6b48e91 Mon Sep 17 00:00:00 2001
From aef7576624fcbe7f63fdf0703f448c90127e953c Mon Sep 17 00:00:00 2001
From: Vasilij Litvinov <[email protected]>
Date: Fri, 20 Nov 2020 14:17:05 +0300
Subject: [PATCH 06/13] Do not crash if BAZEL_SH not set on Windows
Expand All @@ -9,10 +9,10 @@ Signed-off-by: Vasilij Litvinov <[email protected]>
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/python/setup.py b/python/setup.py
index e714d988d..ac937da29 100644
index 2cfe6f609..248f1cf85 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -237,7 +237,7 @@ def build(build_python, build_java):
@@ -238,7 +238,7 @@ def build(build_python, build_java):
SHELL = bazel_env.get("SHELL")
if SHELL:
bazel_env.setdefault("BAZEL_SH", os.path.normpath(SHELL))
Expand All @@ -22,5 +22,5 @@ index e714d988d..ac937da29 100644
wsl_bash = os.path.join(SYSTEMROOT, "System32", "bash.exe")
if (not BAZEL_SH) and SYSTEMROOT and os.path.isfile(wsl_bash):
--
2.30.2
2.26.2

Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
From e4339570f80880bea34420af7d2310ad6ba44892 Mon Sep 17 00:00:00 2001
From b9a0c4e81af7ca757bdcff2a057437b253261e41 Mon Sep 17 00:00:00 2001
From: Vasilij Litvinov <[email protected]>
Date: Fri, 20 Nov 2020 15:32:19 +0300
Subject: [PATCH 07/13] Convert symlinks to junctions on Windows before
Expand All @@ -11,10 +11,10 @@ Signed-off-by: Gregory Shimansky <[email protected]>
1 file changed, 43 insertions(+)

diff --git a/python/setup.py b/python/setup.py
index ac937da29..020acc922 100644
index 248f1cf85..b4f35220f 100644
--- a/python/setup.py
+++ b/python/setup.py
@@ -216,6 +216,48 @@ def download_pickle5(pickle5_dir):
@@ -217,6 +217,48 @@ def download_pickle5(pickle5_dir):
finally:
wzf.close()

Expand Down Expand Up @@ -63,7 +63,7 @@ index ac937da29..020acc922 100644

def build(build_python, build_java):
if tuple(sys.version_info[:2]) not in SUPPORTED_PYTHONS:
@@ -234,6 +276,7 @@ def build(build_python, build_java):
@@ -235,6 +277,7 @@ def build(build_python, build_java):
bazel_env = dict(os.environ, PYTHON3_BIN_PATH=sys.executable)

if is_native_windows_or_msys():
Expand All @@ -72,5 +72,5 @@ index ac937da29..020acc922 100644
if SHELL:
bazel_env.setdefault("BAZEL_SH", os.path.normpath(SHELL))
--
2.30.2
2.26.2

Loading