[cmake] [CUDA] ignore CUDA-specific source files in non-CUDA builds (fixes #6267)

[cmake] [CUDA] ignore CUDA-specific source files in non-CUDA builds (fixes #6267) #7780