numpy ufuncs store poorly when serialized=False

[mmckerns]

In python 2.7 (and only python 2.7), I see TypeError for np.ptp:

>>> import klepto
>>> d = klepto.archives.dir_archive(cached=False, serialized=False)
>>> import numpy as np
>>> d['d'] = np.ptp
>>> x = np.array([1,2,3,4,5])
>>> d['d'](x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/dev/svn/pathos/klepto/tests/memo/K_d/__init__.py", line 136, in ptp
    return _wrapfunc(a, 'ptp', axis=axis, out=out)
TypeError: 'NoneType' object is not callable
>>> d['d']   
<function ptp at 0x10946b1b8>
>>> d['d'] = np.min
>>> d['d'](x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python2.7/site-packages/klepto-0.1.4.dev0-py2.7.egg/klepto/_archives.py", line 330, in __getitem__
    return self._lookup(key)
  File "/Users/mmckerns/lib/python2.7/site-packages/klepto-0.1.4.dev0-py2.7.egg/klepto/_archives.py", line 526, in _lookup
    raise KeyError(key)
KeyError: 'd'
>>> d['d'] = np.sqrt
>>> d['d'](x)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python2.7/site-packages/klepto-0.1.4.dev0-py2.7.egg/klepto/_archives.py", line 330, in __getitem__
    return self._lookup(key)
  File "/Users/mmckerns/lib/python2.7/site-packages/klepto-0.1.4.dev0-py2.7.egg/klepto/_archives.py", line 526, in _lookup
    raise KeyError(key)
KeyError: 'd'

However, the KeyError is demonstrated for python 3.6 (and likely other 3.x versions).

This corresponds to the test at 144L in test_readwrite.py.

[mmckerns]

Essentially, these are all failures of dill.source to detect dependencies of numpy ufuncs. In general this can be quite difficult, so I'd expect that this might not improve unless the number of dependencies can be shown to be a very small finite set.

Python 3.7.16 (default, Dec  7 2022, 05:04:27) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import klepto
>>> d = klepto.archives.dir_archive(cached=False, serialized=False)
>>> import numpy as np
>>> d['d'] = np.ptp
>>> d['d']
Traceback (most recent call last):
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 554, in _lookup
    exec(string, globals()) #FIXME: unsafe, potential name conflict
  File "<string>", line 1, in <module>
  File "/Users/mmckerns/dev/git/uqfoundation/dill/docs/memo/K_d/__init__.py", line 6, in <module>
    @array_function_dispatch(_ptp_dispatcher)
NameError: name 'array_function_dispatch' is not defined

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 382, in __getitem__
    return self._lookup(key)
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 558, in _lookup
    raise KeyError(key)
KeyError: 'd'
>>> d['d'] = np.min
>>> d['d']
Traceback (most recent call last):
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 554, in _lookup
    exec(string, globals()) #FIXME: unsafe, potential name conflict
  File "<string>", line 1, in <module>
  File "/Users/mmckerns/dev/git/uqfoundation/dill/docs/memo/K_d/__init__.py", line 112, in <module>
    from ._globals import (
ModuleNotFoundError: No module named 'K_d._globals'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 382, in __getitem__
    return self._lookup(key)
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 558, in _lookup
    raise KeyError(key)
KeyError: 'd'
>>> d['d'] = np.sqrt
>>> d['d']
Traceback (most recent call last):
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 554, in _lookup
    exec(string, globals()) #FIXME: unsafe, potential name conflict
  File "<string>", line 1, in <module>
  File "/Users/mmckerns/dev/git/uqfoundation/dill/docs/memo/K_d/__init__.py", line 2
    memo = <ufunc 'sqrt'>
           ^
SyntaxError: invalid syntax

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 382, in __getitem__
    return self._lookup(key)
  File "/Users/mmckerns/lib/python3.7/site-packages/klepto/_archives.py", line 558, in _lookup
    raise KeyError(key)
KeyError: 'd'

The good thing is, serialized numpy ufuncs work well.

>>> d = klepto.archives.dir_archive(cached=False, serialized=True)
>>> d['d'] = np.ptp
>>> d['d']
<function ptp at 0x10a8dd440>
>>> d['d'] = np.min
>>> d['d']
<function amin at 0x10a8dd8c0>
>>> d['d'] = np.sqrt
>>> d['d']
<ufunc 'sqrt'>
>>> d['d']([1,2,3])
array([1.        , 1.41421356, 1.73205081])

As this is essentially something that should get fixed in dill (i.e. not in klepto), this really isn't a klepto bug any more than not being able to store some unpicklable class object.