Encoding::CompatibilityError: incompatible encoding regexp match

[orbanbotond]

Encoding::CompatibilityError: incompatible encoding regexp match (ASCII-8BIT regexp with UTF-8 string)
from /Users/boti/.rvm/gems/ruby-1.9.3-p327@search_server/gems/chardet2-1.0.1/lib/UniversalDetector.rb:134:in =~' from /Users/boti/.rvm/gems/ruby-1.9.3-p327@search_server/gems/chardet2-1.0.1/lib/UniversalDetector.rb:134:infeed'
from /Users/boti/.rvm/gems/ruby-1.9.3-p327@search_server/gems/chardet2-1.0.1/lib/UniversalDetector.rb:46:in `chardet'
from (irb):12

[saneshark]

same issue testing with:

UniversalDetector.chardet("∀,∈,≠,Ω,∑,∏,ɔ,⍴,€,ζ,π,ป่")

which should return utf8 as the encoding type.

[mremond]

Did you solve your issue ?

[orbanbotond]

Hi,

It is now a deprecated project. But despite that the issue is still there.
The lib didn't return me the proper encoding.

On 22 February 2014 12:54, Mickaël Rémond [email protected] wrote:

Did you solve your issue ?

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8#issuecomment-35799973
.

[mremond]

Thanks !
I guess I have to find an alternative way of detecting encoding then.

[orbanbotond]

Well no... I tried 3 other libs and then I decided to manually specify the
encoding...

On 22 February 2014 13:08, Mickaël Rémond [email protected] wrote:

Thanks !
I guess I have to find an alternative way of detecting encoding then.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8#issuecomment-35800192
.

[orbanbotond]

I think it was a hard case.

On 22 February 2014 13:09, Botond Orbán [email protected] wrote:

Well no... I tried 3 other libs and then I decided to manually specify the
encoding...

On 22 February 2014 13:08, Mickaël Rémond [email protected]:

Thanks !
I guess I have to find an alternative way of detecting encoding then.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8#issuecomment-35800192
.

[saneshark]

I just patched rchardet, an older library. Although I'm thinking one could just write the string to a temp file and use the system:

 encoding = `file --mime-encoding string.tmp | awk '{print $2}'`.strip.upcase
 string.force_encoding(encoding)