Package compatibility issues

[HGanFan]

Hello, it seems that cudf-cu12 is not compatible with merlin-core and merlin-dataloader, even if they are the newest versions (23.8.0). Do you meet this issue before?

Traceback (most recent call last):
File "/scTab/train.py", line 12, in
from cellnet.estimators import EstimatorCellTypeClassifier
File "/scTab/cellnet/estimators.py", line 10, in
from cellnet.datamodules import MerlinDataModule
File "/scTab/cellnet/datamodules.py", line 7, in
import merlin.io
File "/python3.10/site-packages/merlin/io/init.py", line 18, in
from merlin.io import dataframe_iter, dataset, shuffle
File "/python3.10/site-packages/merlin/io/dataset.py", line 47, in
from merlin.io.dask import _ddf_to_dataset, _simple_shuffle
File "/python3.10/site-packages/merlin/io/dask.py", line 31, in
from merlin.io.writer_factory import _writer_cls_factory, writer_factory
File "/python3.10/site-packages/merlin/io/writer_factory.py", line 19, in
from merlin.io.parquet import CPUParquetWriter, GPUParquetWriter
File "/python3.10/site-packages/merlin/io/parquet.py", line 63, in
from merlin.io.fsspec_utils import _optimized_read_partition_remote, _optimized_read_remote
File "/python3.10/site-packages/merlin/io/fsspec_utils.py", line 34, in
from cudf.core.column import as_column, build_categorical_column
ImportError: cannot import name 'build_categorical_column' from 'cudf.core.column'

[felix0097]

Hi @HGanFan,

I think merlin 23.08 might only support cudf-cu11. I would recommend you to use the Merlin docker container, Merlin is already preinstalled there.

Otherwise, an option might be to try merlin 23.12 instead of 23.08. This should work as well, haven't tested it myself though.

[HGanFan]

thanks for response. but I cannot find the merlin 23.12, can you tell me the address?