Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DistGB]Restore the gb_convert_single_dgl_partition of the public function in partition.py #7803

Merged
merged 54 commits into from
Sep 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
54 commits
Select commit Hold shift + click to select a range
8f9e639
change a variable
Aug 14, 2024
91792fb
Merge branch 'dmlc:master' into master
CfromBU Aug 20, 2024
bfeb3b4
modify partition test case
Aug 20, 2024
bec2af3
change pr
Aug 20, 2024
241ccd8
change partition
Aug 20, 2024
21b592d
change test_partition.py and partiton.py
CfromBU Aug 21, 2024
4ef95d5
partition
CfromBU Aug 21, 2024
1074f85
change partition
CfromBU Aug 21, 2024
e03376d
change partition internal function
CfromBU Aug 21, 2024
090a430
change partition
CfromBU Aug 23, 2024
46afb4b
change partition
CfromBU Aug 23, 2024
6120a26
change code format
CfromBU Aug 23, 2024
c1dcbc0
change variable
CfromBU Aug 23, 2024
f046ec3
fix bug
CfromBU Aug 26, 2024
f279e3d
change format
CfromBU Aug 26, 2024
081c1ad
fix pr
CfromBU Aug 26, 2024
d1beec2
change test_partition
CfromBU Aug 26, 2024
5d3dc78
change format
CfromBU Aug 26, 2024
1c96380
change issues
CfromBU Aug 26, 2024
b83f2bf
new
CfromBU Aug 26, 2024
6d4c2e5
change format
CfromBU Aug 27, 2024
4330ce3
change partition
CfromBU Aug 28, 2024
070d370
[DistGB]change test_partition.py, renew the code.
Sep 1, 2024
ba77ab4
[distGB] change test_partition.py
Sep 2, 2024
f29cb5f
[distGB] change test_partition.py
Sep 2, 2024
da09a92
[distGB] change test_partition.py
Sep 2, 2024
a4111fb
resolve merge conflict
Sep 2, 2024
be69e1e
[distGB] modify partition.py
Sep 5, 2024
3ff236c
Merge branch 'master' into partition
Rhett-Ying Sep 6, 2024
ef6693a
change partition.py
Sep 6, 2024
66751fa
change partition.py
Sep 6, 2024
e80e353
partition.py
Sep 6, 2024
d7173b3
partition.py
Sep 6, 2024
b91ef14
chaneg doc
Sep 6, 2024
0819607
Merge branch 'master' into partition
Rhett-Ying Sep 9, 2024
b2907b4
change partition
Sep 9, 2024
24522e1
change partition.py
Sep 9, 2024
dcf3a39
change partition.py format
Sep 9, 2024
220a8b3
modify partition.py
Sep 10, 2024
45ddaa6
resolve conflic in partition
Sep 10, 2024
a2cfcad
reslove conflict in partition
Sep 10, 2024
200eb6c
change partition
Sep 10, 2024
bb32397
change partition
Sep 10, 2024
33784ad
change partition
Sep 10, 2024
2193ce4
add partition
Sep 10, 2024
a02a882
modify partition
Sep 10, 2024
99450be
change partition
Sep 11, 2024
0287b30
change partition
Sep 18, 2024
945565e
add gb_convert_single_dgl_partition to dgl.distributed.__init__.py
Sep 18, 2024
97d000b
change partition
Sep 23, 2024
27bb682
change partition
Sep 23, 2024
16a606c
change partition
Sep 23, 2024
be716ee
change format
Sep 23, 2024
53416c1
change gb_convert_single_dgl_partition args
Sep 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions python/dgl/distributed/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from .nn import *
from .partition import (
dgl_partition_to_graphbolt,
gb_convert_single_dgl_partition,
CfromBU marked this conversation as resolved.
Show resolved Hide resolved
load_partition,
load_partition_book,
load_partition_feats,
Expand Down
26 changes: 13 additions & 13 deletions python/dgl/distributed/partition.py
Original file line number Diff line number Diff line change
Expand Up @@ -687,7 +687,7 @@ def _partition_to_graphbolt(
part_config=part_config, part_id=part_i, part_metadata=part_metadata
)
graph = parts[part_i]
csc_graph = gb_convert_single_dgl_partition(
csc_graph = _convert_dgl_partition_to_gb(
ntypes=ntypes,
etypes=etypes,
gpb=gpb,
Expand Down Expand Up @@ -1702,7 +1702,7 @@ def _create_attributes_gb(
return node_attributes, edge_attributes, type_per_edge


def gb_convert_single_dgl_partition(
def _convert_dgl_partition_to_gb(
ntypes,
etypes,
gpb,
Expand Down Expand Up @@ -1820,30 +1820,30 @@ def gb_convert_single_dgl_partition(
return csc_graph


def _convert_partition_to_graphbolt(
part_config,
def gb_convert_single_dgl_partition(
part_id,
graph_formats=None,
store_eids=False,
store_inner_node=False,
store_inner_edge=False,
graph_formats,
part_config,
store_eids=True,
store_inner_node=True,
store_inner_edge=True,
):
"""
The pipeline converting signle partition to graphbolt.

Parameters
----------
part_config : str
The path of the partition config file.
part_id : int
The partition ID.
graph_formats : str or list[str], optional
graph_formats : str or list[str]
Save partitions in specified formats. It could be any combination of
`coo`, `csc`. As `csc` format is mandatory for `FusedCSCSamplingGraph`,
it is not necessary to specify this argument. It's mainly for
specifying `coo` format to save edge ID mapping and destination node
IDs. If not specified, whether to save `coo` format is determined by
the availability of the format in DGL partitions. Default: None.
part_config : str
The path of the partition config file.
store_eids : bool, optional
Whether to store edge IDs in the new graph. Default: True.
store_inner_node : bool, optional
Expand All @@ -1861,7 +1861,7 @@ def _convert_partition_to_graphbolt(
)
part = _load_part(part_config, part_id)
part_meta = copy.deepcopy(_load_part_config(part_config))
csc_graph = gb_convert_single_dgl_partition(
csc_graph = _convert_dgl_partition_to_gb(
graph=part,
ntypes=ntypes,
etypes=etypes,
Expand Down Expand Up @@ -1896,7 +1896,7 @@ def _convert_partition_to_graphbolt_wrapper(

# Iterate over partitions.
convert_with_format = partial(
_convert_partition_to_graphbolt,
gb_convert_single_dgl_partition,
part_config=part_config,
graph_formats=graph_formats,
store_eids=store_eids,
Expand Down
Loading