Potential Bug

[kribou]

Create a X matrix (500x200 000) without memory mapping using LocalEngine with 3 processes.

cv_svm_local = CV(Methods(*[SVC(kernel="linear"),
SVC(kernel="rbf")]),
n_folds=3)

 -> Pt1 : Beginning with 200000 features, memmap = False , 3 processes
 -> Pt2 : X and y created, building workflow
/home/jl237561/github/scikit-learn/sklearn/utils/extmath.py:129: UserWarning: Could not import BLAS, falling back to np.dot
  warnings.warn('Could not import BLAS, falling back to np.dot')
 -> Pt3 : Workflow built, running
Traceback (most recent call last):
  File "test_memmapping.py", line 108, in <module>
    func_memm_local(int(args[0]), int(args[1]), args[2], int(args[3]))
  File "test_memmapping.py", line 97, in func_memm_local
    cv_svm = local_engine.run(**Xy)
  File "/home/jl237561/github/pylearn-epac/epac/map_reduce/engine.py", line 143, in run
    for i in input_list)
  File "/home/jl237561/github/joblib/joblib/parallel.py", line 558, in __call__
    for function, args, kwargs in iterable:
  File "/home/jl237561/github/pylearn-epac/epac/map_reduce/engine.py", line 143, in <genexpr>
    for i in input_list)
  File "/home/jl237561/github/joblib/joblib/parallel.py", line 120, in delayed
    pickle.dumps(function)
SystemError: error return without exception set

[kribou]

It seems the error was coming from the dill library : as it overloads pickle, it may have caused some problems when joblib was trying to call pickle
An update of the dill libraby seems to have solved this bug