-
-
Notifications
You must be signed in to change notification settings - Fork 22
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
parallel implementation for all_pairs_bellman_ford_path #14
Changes from 3 commits
8e48408
614ada3
a79de33
f8f512f
6c5e0d4
2abd434
aaa36b7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
from .weighted import * |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,61 @@ | ||
from joblib import Parallel, delayed | ||
from networkx.algorithms.shortest_paths.weighted import ( | ||
single_source_bellman_ford_path | ||
) | ||
|
||
import nx_parallel as nxp | ||
|
||
__all__ = ["all_pairs_bellman_ford_path"] | ||
|
||
|
||
def all_pairs_bellman_ford_path(G, weight="weight"): | ||
"""Compute shortest paths between all nodes in a weighted graph. | ||
|
||
Parameters | ||
---------- | ||
G : NetworkX graph | ||
|
||
weight : string or function (default="weight") | ||
If this is a string, then edge weights will be accessed via the | ||
edge attribute with this key (that is, the weight of the edge | ||
joining `u` to `v` will be ``G.edges[u, v][weight]``). If no | ||
such edge attribute exists, the weight of the edge is assumed to | ||
be one. | ||
|
||
If this is a function, the weight of an edge is the value | ||
returned by the function. The function must accept exactly three | ||
positional arguments: the two endpoints of an edge and the | ||
dictionary of edge attributes for that edge. The function must | ||
return a number. | ||
|
||
Returns | ||
------- | ||
paths : iterator | ||
(source, dictionary) iterator with dictionary keyed by target and | ||
shortest path as the key value. | ||
|
||
Notes | ||
----- | ||
Edge weight attributes must be numerical. | ||
Distances are calculated as sums of weighted edges traversed. | ||
|
||
""" | ||
def _calculate_shortest_paths_subset(G, source, weight): | ||
return (source, single_source_bellman_ford_path(G, source, weight=weight)) | ||
|
||
if hasattr(G, "graph_object"): | ||
G = G.graph_object | ||
|
||
nodes = G.nodes | ||
|
||
total_cores = nxp.cpu_count() | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We should be moving this to the function signature and match something like what scikit-learn does ( |
||
|
||
paths = Parallel(n_jobs=total_cores, return_as="generator")( | ||
delayed(_calculate_shortest_paths_subset)( | ||
G, | ||
source, | ||
weight=weight | ||
) | ||
for source in nodes | ||
) | ||
return paths |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,4 +1,6 @@ | ||
import time | ||
import random | ||
import types | ||
|
||
import networkx as nx | ||
import pandas as pd | ||
|
@@ -11,23 +13,30 @@ | |
heatmapDF = pd.DataFrame() | ||
number_of_nodes_list = [10, 50, 100, 300, 500] | ||
pList = [1, 0.8, 0.6, 0.4, 0.2] | ||
currFun = nx.betweenness_centrality | ||
currFun = nx.all_pairs_bellman_ford_path | ||
for i in range(0, len(pList)): | ||
p = pList[i] | ||
for j in range(0, len(number_of_nodes_list)): | ||
num = number_of_nodes_list[j] | ||
|
||
# create original and parallel graphs | ||
G = nx.fast_gnp_random_graph(num, p, directed=False) | ||
|
||
# for weighted graphs | ||
for u, v in G.edges(): | ||
G[u][v]['weight'] = random.random() | ||
|
||
H = nx_parallel.ParallelGraph(G) | ||
|
||
# time both versions and update heatmapDF | ||
t1 = time.time() | ||
c = currFun(H) | ||
if type(c)==types.GeneratorType: d = dict(c) | ||
t2 = time.time() | ||
parallelTime = t2 - t1 | ||
t1 = time.time() | ||
c = currFun(G) | ||
if type(c)==types.GeneratorType: d = dict(c) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Is it worth checking that the results are the same? (outside the timing part) There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I checked |
||
t2 = time.time() | ||
stdTime = t2 - t1 | ||
timesFaster = stdTime / parallelTime | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think
G
andweight
are already loaded into the outer function's namespace, so they will be found when used within this helper function. So you can remove those two inputs and make this a function of onlysource
. That also shortens the later code that calls this function. Less time patching together function arguments, more time needed for variable lookups. But I think it could be faster overall. Can you tell?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yes, passing only
source
is a little faster, following are the speed-ups for the same :I have changed it in the recent commit