Some characters aren't imported correctly (sometimes)

[bernimel]

Sometimes, when you guess the information from an URL and there are accents in the text the importation fails and the system shows a "Incorrect string value: '\xEF\xBF\xBDn d...' for column 'text' at row 1" error if you submit the report. Anyway, the system creates the mention.

[nukeador]

Environment:


Request Method: POST
Request URL: https://www.mozilla-hispano.org/mozbuzz/mention/create

Django Version: 1.3.1
Python Version: 2.6.5
Installed Applications:
['django.contrib.auth',
 'django_browserid',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.admin',
 'django.contrib.markup',
 'south',
 'buzz']
Installed Middleware:
('django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware')


Traceback:
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/contrib/auth/decorators.py" in _wrapped_view
  23.                 return view_func(request, *args, **kwargs)
File "../mozbuzz/buzz/utils.py" in __inner__
  16.         result=view(request, *args, **kwargs)
File "/var/lib/mozbuzz/buzz/views.py" in mention
  50.             instance.save()
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py" in save
  460.         self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/base.py" in save_base
  553.                     result = manager._insert(values, return_id=update_pk, using=using)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/manager.py" in _insert
  195.         return insert_query(self.model, values, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/query.py" in insert_query
  1436.     return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py" in execute_sql
  791.         cursor = super(SQLInsertCompiler, self).execute_sql(None)
File "/usr/local/lib/python2.6/dist-packages/django/db/models/sql/compiler.py" in execute_sql
  735.         cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/util.py" in execute
  34.             return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.6/dist-packages/django/db/backends/mysql/base.py" in execute
  86.             return self.cursor.execute(query, args)
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py" in execute
  168.         if not self._defer_warnings: self._warning_check()
File "/usr/lib/pymodules/python2.6/MySQLdb/cursors.py" in _warning_check
  82.                     warn(w[-1], self.Warning, 3)

Exception Type: Warning at /mozbuzz/mention/create
Exception Value: Incorrect string value: '\xEF\xBF\xBD aq...' for column 'text' at row 1

[bbotella]

I'm not being able to reproduce the error... Have any of you a url which throws the error?

[bbotella]

@sebasmagri Any further info here?

[sebasmagri]

@bbotella I've not been able to test this. We sould probably test with several articles using unicode characters such as asian or arabian text and see the results.