[Core] Support Intel GPU

[harborn]

Why are these changes needed?

This PR aim to support Intel GPU for Ray.
This PR make minimum changes to support the Intel GPU, and commonly, Intel GPU is called XPU

Upgrade list:

1. Define GPU is common accelerator type. CUDA is GPU, XPU is also GPU
2. Define an environment variable RAY_ACCELERATOR to specific accelerator type
3. Using the same scheduling policy of CUDA device

Usage compare:

CUDA	XPU
1. define the environment variable RAY_ACCELERATOR="CUDA", or not define this environment variable 2. ray init with cuda: ray.init(num_gpus=3) 3. use CUDA resource in remote function	1. define the environment variable RAY_ACCELERATOR="XPU" 2. ray init with xpu: ray.init(num_gpus=3) 3. use XPU resource in remote function

Related issue number

Checks

• I've signed off every commit(by using the -s flag, i.e., git commit -s) in this PR.
• I've run scripts/format.sh to lint the changes in this PR.
• I've included any doc changes needed for https://docs.ray.io/en/master/.
  • I've added any new APIs to the API Reference. For example, if I added a
    method in Tune, I've added it in doc/source/tune/api/ under the
    corresponding .rst file.
• I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
• Testing Strategy
  • Unit tests
  • Release tests
  • This PR is not tested :(

[harborn]

Can you help to remove additional reviewers? @jjyao

[harborn]

please switch to PR https://github.com/ray-project/ray/pull/38553

[harborn]

Thanks for the contribution and the answers to our questions. I have some high-level questions (see comments) to get out of the way; once those are answered I will go through and do a code-review pass.

Not sure how GPU related code got tested in the CI. Does it have a real GPU to run or just mock the test?

To say we have full XPU support we should have a real XPU test. But since this PR seems to just implement logical scheduling, for now we just need tests which mock out the XPU detection and validates that Ray can schedule XPUs. This PR should include XPU versions of the following tests:

• test_disable_cuda_devices https://sourcegraph.com/github.com/ray-project/ray@32212dc61b72cbb753123640ebf26fb07d26c712/-/blob/python/ray/tests/test_basic.py?L525
• test_gpu_ids https://sourcegraph.com/github.com/ray-project/ray@32212dc61b72cbb753123640ebf26fb07d26c712/-/blob/python/ray/tests/test_advanced_2.py?L17
• test_local_mode_gpus https://sourcegraph.com/github.com/ray-project/ray@32212dc61b72cbb753123640ebf26fb07d26c712/-/blob/python/ray/tests/test_advanced_6.py?L63

@xwu99 yes we are in process of creating a composite CI for xpu tests ( this is in development and collaboration with other orgs such as huggingface, pt lightning to name a few). This will only happen post the pr I raise,( and also this pr which gets merged) . I maintain the pt ecosystem( huggingface for xpus as mentioned) so ci and all would be post full support. I'll raise the pr next week.

Nice! Can I read more about this effort?

Also added 3 test on PR https://github.com/ray-project/ray/pull/38553