Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for UnicodeDecodeError exception #30

Open
wagnerscastle opened this issue May 7, 2021 · 1 comment
Open

Fix for UnicodeDecodeError exception #30

wagnerscastle opened this issue May 7, 2021 · 1 comment

Comments

@wagnerscastle
Copy link

pp() will fail with malformed unicode strings that the default pprint library works fine with. At least on python3.

To fix this, replace
seg_list = [seg.decode('utf8') for seg in seg_list]
with
seg_list = [seg.decode('utf8','backslashreplace') for seg in seg_list]

To reproduce:

pp({'1':'string\xe9'})
{
  '1':
Traceback (most recent call last):
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe9 in position 6: invalid continuation byte

After patch:

pp({'1':'string\xe9'})
{
  '1': 'string\xe9',
}

Of note is that a dict.key is printed just fine. For example (without the patch):

pp({'\xe91\xe9':'string'})
{
  'é1é': 'string',
}
@Nevrai
Copy link

Nevrai commented Nov 23, 2023

I’m dealing with the UnicodeDecodeError as well:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x99 in position 43: invalid start byte.

It happens when trying to print a JSON stringified Python dict. I forked beeprint with the fix, and I can verify that it works.

Nevrai added a commit to Nevrai/beeprint that referenced this issue Nov 23, 2023

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature.
Nevrai added a commit to Nevrai/beeprint that referenced this issue Nov 23, 2023
Fix by wagnerscastle: panyanyany#30

Fork with the fix here: https://github.com/DeleteMetaInf/beeprint

Tested it, and it works. No longer causes a UnicodeDecodeError when trying to print a malformed Unicode string or a JSON stringified Python dict.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants