Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python-memcached should fail gracefully when the server does not exist. #92

Open
fratlee opened this issue Apr 7, 2016 · 0 comments
Open

Comments

@fratlee
Copy link

fratlee commented Apr 7, 2016

I recently ran into a situation where our build server did not have a running instance of memcached. I would have expected things to fail gracefully. (Clearly, there wouldn't be any caching happening. Each request would simply have to re-fetch any cached data.) But instead of failing gracefully, the caching code failed after the connect attempt to the memcached server failed.

It looks like once the server is marked dead there are some additional calls to cmemcache_hash that get made with strings as keys. To reproduce this, find a server without a memcached instance and try this:

python
Python 3.5.1 (default, Apr  6 2016, 16:48:42) 
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import memcache
>>> cache = memcache.Client(['127.0.0.1:11211'], debug=1)
>>> cache.set("working?",1,1)
MemCached: MemCache: inet:127.0.0.1:11211: connect: [Errno 111] Connection refused.  Marking dead.
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 700, in set
    return self._set("set", key, val, time, min_compress_len, noreply)
  File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 983, in _set
    server, key = self._get_server(key)
  File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 413, in _get_server
    serverhash = serverHashFunction(str(serverhash) + str(i))
  File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 65, in cmemcache_hash
    (((binascii.crc32(key) & 0xffffffff)
**TypeError: a bytes-like object is required, not 'str'**

So, I temporarily "fixed" the issue with something like this:

--- /<snipped>/lib/python3.5/site-packages/memcache.py  2016-04-07 17:23:41.428028602 -0500
+++ /<snipped>/lib/python3.5/site-packages/memcache.py_fixed    2016-04-07 17:23:13.606003863 -0500
@@ -61,6 +61,9 @@


 def cmemcache_hash(key):
+    if isinstance(key, six.text_type):
+        key = key.encode('utf-8')
+
     return (
         (((binascii.crc32(key) & 0xffffffff)
           >> 16) & 0x7fff) or 1)

I'm not sure if this is the right approach to correct the issue or not. Hopefully we can get back to a point where python-memcached can fail gracefully when the underlying service is absent.

Thanks,

Matthew Lee

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant