Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python 3 compatibility #11

Open
eastein opened this issue Oct 8, 2016 · 3 comments
Open

python 3 compatibility #11

eastein opened this issue Oct 8, 2016 · 3 comments

Comments

@eastein
Copy link
Owner

eastein commented Oct 8, 2016

Got started in a py3 labeled branch which I have since tagged (0.1.0) and released on PyPI. More work to do - under python 3.5.2, I get this when trying to start the bot (which really is just reading a msgpack persisted chain that was produced under python 2.7.12, using the same revision of the code) - the issue lies within andrey, not andrey_bot.

Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey_bot/run.py", line 186, in <module>
    ratio=float(options.ratio), word_replace=word_replace, word_filter=word_filter, read_only=options.read_only, chatty=options.chatty, chain_length=int(options.chain_length))
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey_bot/run.py", line 139, in __init__
    self.m = persist.PersistedMarkov.restore(filename, 2, 3)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/andrey/persist.py", line 53, in restore
    return cls.fromdict(next(msgpack.Unpacker(open(filename), encoding='utf-8')))
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 557, in next
    ret = self._fb_unpack(EX_CONSTRUCT, None)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 498, in _fb_unpack
    typ, n, obj = self._read_header(execute, write_bytes)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 346, in _read_header
    c = self._fb_read(1, write_bytes)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/site-packages/msgpack/fallback.py", line 324, in _fb_read
    tmp = self.file_like.read(to_read)
  File "/home/eastein/venvs/andrey-3/lib/python3.5/codecs.py", line 321, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x83 in position 0: invalid start byte
@eastein
Copy link
Owner Author

eastein commented Oct 8, 2016

I started doing some research into this and found that there have been compatiblity changes in msgpack-python at different points around encoding, which may be playing poorly with python 3's own encoding compatibility issues: https://pypi.python.org/pypi/msgpack-python - see section "Notes / string and binary type".

@eastein
Copy link
Owner Author

eastein commented Oct 8, 2016

I will want to use tox to automate testing while working on this issue. https://tox.readthedocs.io/en/latest/

eastein added a commit that referenced this issue Oct 9, 2016
…de (py3 default was breaking things) - for #11. Tests still fail though.
@estein-de
Copy link

On python 3.9.16, on macOS, using pyenv-virtualenv and having installed 0.1.0 off of PyPI, I get this when trying to train a chain:

$ echo "test" | python  -m andrey.train test.mpack 2 3
test.mpack 2 3
Traceback (most recent call last):
  File "/Users/EStein/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/Users/EStein/.pyenv/versions/3.9.16/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/andrey/train.py", line 25, in <module>
    m.save(fn)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/andrey/persist.py", line 31, in save
    msgpack.dump(self.todict(), fh)
  File "/Users/EStein/.pyenv/versions/andrey-3916/lib/python3.9/site-packages/msgpack/__init__.py", line 38, in pack
    stream.write(packer.pack(o))
TypeError: write() argument must be str, not bytes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants