Query returns 'Call to a member function transliterate() on null"

[marygriffus]

I have been working through trying to install Nominatim on an Amazon Linux 2 instance. It is finally installing successfully, and I have imported an osm file and hooked everything up with nginx and php-fpm, but when I either run nominatim search or use the api search function, I get: {"error":{"code":0,"message":"Call to a member function transliterate() on null"}}.

I'm using ICU version 68.1, and I have the polyglot, PyICU, pycld2, and icu-tokenizer packages installed.

I saw that someone got the same error here: fisharebest/webtrees#3066, ran php -r 'print_r(transliterator_list_ids());' and it gave me the following:

    [0] => ASCII-Latin
    [1] => Accents-Any
    [2] => Amharic-Latin/BGN
    [3] => Any-Accents
    [4] => Any-Publishing
    [5] => Arabic-Latin
    [6] => Arabic-Latin/BGN
    [7] => Armenian-Latin
    [8] => Armenian-Latin/BGN
    [9] => Azerbaijani-Latin/BGN
    [10] => Belarusian-Latin/BGN
    [11] => Bengali-Devanagari
    [12] => Bengali-Gujarati
    [13] => Bengali-Gurmukhi
    [14] => Bengali-Kannada
    [15] => Bengali-Latin
    [16] => Bengali-Malayalam
    [17] => Bengali-Oriya
    [18] => Bengali-Tamil
    [19] => Bengali-Telugu
    [20] => Bopomofo-Latin
    [21] => Bulgarian-Latin/BGN
    [22] => Cyrillic-Latin
    [23] => Devanagari-Bengali
    [24] => Devanagari-Gujarati
    [25] => Devanagari-Gurmukhi
    [26] => Devanagari-Kannada
    [27] => Devanagari-Latin
    [28] => Devanagari-Malayalam
    [29] => Devanagari-Oriya
    [30] => Devanagari-Tamil
    [31] => Devanagari-Telugu
    [32] => Digit-Tone
    [33] => Fullwidth-Halfwidth
    [34] => Georgian-Latin
    [35] => Georgian-Latin/BGN
    [36] => Greek-Latin
    [37] => Greek-Latin/BGN
    [38] => Greek-Latin/UNGEGN
    [39] => Gujarati-Bengali
    [40] => Gujarati-Devanagari
    [41] => Gujarati-Gurmukhi
    [42] => Gujarati-Kannada
    [43] => Gujarati-Latin
    [44] => Gujarati-Malayalam
    [45] => Gujarati-Oriya
    [46] => Gujarati-Tamil
    [47] => Gujarati-Telugu
    [48] => Gurmukhi-Bengali
    [49] => Gurmukhi-Devanagari
    [50] => Gurmukhi-Gujarati
    [51] => Gurmukhi-Kannada
    [52] => Gurmukhi-Latin
    [53] => Gurmukhi-Malayalam
    [54] => Gurmukhi-Oriya
    [55] => Gurmukhi-Tamil
    [56] => Gurmukhi-Telugu
    [57] => Halfwidth-Fullwidth
    [58] => Han-Latin
    [59] => Han-Latin/Names
    [60] => Hangul-Latin
    [61] => Hans-Hant
    [62] => Hant-Hans
    [63] => Hebrew-Latin
    [64] => Hebrew-Latin/BGN
    [65] => Hiragana-Katakana
    [66] => Hiragana-Latin
    [67] => IPA-XSampa
    [68] => Jamo-Latin
    [69] => JapaneseKana-Latin/BGN
    [70] => Kannada-Bengali
    [71] => Kannada-Devanagari
    [72] => Kannada-Gujarati
    [73] => Kannada-Gurmukhi
    [74] => Kannada-Latin
    [75] => Kannada-Malayalam
    [76] => Kannada-Oriya
    [77] => Kannada-Tamil
    [78] => Kannada-Telugu
    [79] => Katakana-Hiragana
    [80] => Katakana-Latin
    [81] => Kazakh-Latin/BGN
    [82] => Kirghiz-Latin/BGN
    [83] => Korean-Latin/BGN
    [84] => Latin-ASCII
    [85] => Latin-Arabic
    [86] => Latin-Armenian
    [87] => Latin-Bengali
    [88] => Latin-Bopomofo
    [89] => Latin-Cyrillic
    [90] => Latin-Devanagari
    [91] => Latin-Georgian
    [92] => Latin-Greek
    [93] => Latin-Greek/UNGEGN
    [94] => Latin-Gujarati
    [95] => Latin-Gurmukhi
    [96] => Latin-Han
    [97] => Latin-Hangul
    [98] => Latin-Hebrew
    [99] => Latin-Hiragana
    [100] => Latin-Jamo
    [101] => Latin-Kannada
    [102] => Latin-Katakana
    [103] => Latin-Malayalam
    [104] => Latin-NumericPinyin
    [105] => Latin-Oriya
    [106] => Latin-Syriac
    [107] => Latin-Tamil
    [108] => Latin-Telugu
    [109] => Latin-Thaana
    [110] => Latin-Thai
    [111] => Macedonian-Latin/BGN
    [112] => Malayalam-Bengali
    [113] => Malayalam-Devanagari
    [114] => Malayalam-Gujarati
    [115] => Malayalam-Gurmukhi
    [116] => Malayalam-Kannada
    [117] => Malayalam-Latin
    [118] => Malayalam-Oriya
    [119] => Malayalam-Tamil
    [120] => Malayalam-Telugu
    [121] => Maldivian-Latin/BGN
    [122] => Mongolian-Latin/BGN
    [123] => NumericPinyin-Latin
    [124] => NumericPinyin-Pinyin
    [125] => Oriya-Bengali
    [126] => Oriya-Devanagari
    [127] => Oriya-Gujarati
    [128] => Oriya-Gurmukhi
    [129] => Oriya-Kannada
    [130] => Oriya-Latin
    [131] => Oriya-Malayalam
    [132] => Oriya-Tamil
    [133] => Oriya-Telugu
    [134] => Pashto-Latin/BGN
    [135] => Persian-Latin/BGN
    [136] => Pinyin-NumericPinyin
    [137] => Publishing-Any
    [138] => Russian-Latin/BGN
    [139] => Serbian-Latin/BGN
    [140] => Simplified-Traditional
    [141] => Syriac-Latin
    [142] => Tamil-Bengali
    [143] => Tamil-Devanagari
    [144] => Tamil-Gujarati
    [145] => Tamil-Gurmukhi
    [146] => Tamil-Kannada
    [147] => Tamil-Latin
    [148] => Tamil-Malayalam
    [149] => Tamil-Oriya
    [150] => Tamil-Telugu
    [151] => Telugu-Bengali
    [152] => Telugu-Devanagari
    [153] => Telugu-Gujarati
    [154] => Telugu-Gurmukhi
    [155] => Telugu-Kannada
    [156] => Telugu-Latin
    [157] => Telugu-Malayalam
    [158] => Telugu-Oriya
    [159] => Telugu-Tamil
    [160] => Thaana-Latin
    [161] => Thai-Latin
    [162] => Tone-Digit
    [163] => Traditional-Simplified
    [164] => Turkmen-Latin/BGN
    [165] => Ukrainian-Latin/BGN
    [166] => Uzbek-Latin/BGN
    [167] => XSampa-IPA
    [168] => cs-cs_FONIPA
    [169] => cs-ja
    [170] => cs-ko
    [171] => cs_FONIPA-ja
    [172] => cs_FONIPA-ko
    [173] => es-am
    [174] => es-es_FONIPA
    [175] => es-ja
    [176] => es-zh
    [177] => es_419-ja
    [178] => es_419-zh
    [179] => es_FONIPA-am
    [180] => es_FONIPA-es_419_FONIPA
    [181] => es_FONIPA-ja
    [182] => es_FONIPA-zh
    [183] => it-am
    [184] => it-ja
    [185] => ja_Latn-ko
    [186] => ja_Latn-ru
    [187] => pl-ja
    [188] => pl-pl_FONIPA
    [189] => pl_FONIPA-ja
    [190] => ro-ja
    [191] => ro-ro_FONIPA
    [192] => ro_FONIPA-ja
    [193] => ru-ja
    [194] => ru-zh
    [195] => sk-ja
    [196] => sk-sk_FONIPA
    [197] => sk_FONIPA-ja
    [198] => zh_Latn_PINYIN-ru
    [199] => Any-Null
    [200] => Any-Lower
    [201] => Any-Upper
    [202] => Any-Title
    [203] => Any-Name
    [204] => Name-Any
    [205] => Any-Remove
    [206] => Any-Hex/Unicode
    [207] => Any-Hex/Java
    [208] => Any-Hex/C
    [209] => Any-Hex/XML
    [210] => Any-Hex/XML10
    [211] => Any-Hex/Perl
    [212] => Any-Hex
    [213] => Hex-Any/Unicode
    [214] => Hex-Any/Java
    [215] => Hex-Any/C
    [216] => Hex-Any/XML
    [217] => Hex-Any/XML10
    [218] => Hex-Any/Perl
    [219] => Hex-Any
    [220] => Any-NFC
    [221] => Any-NFKC
    [222] => Any-NFD
    [223] => Any-NFKD
    [224] => Any-FCD
    [225] => Any-FCC
    [226] => Any-Latin
    [227] => Any-Telugu
    [228] => Any-Gurmukhi
    [229] => Any-Gujarati
    [230] => Any-Malayalam
    [231] => Any-Oriya
    [232] => Any-Devanagari
    [233] => Any-Kannada
    [234] => Any-Tamil
    [235] => Any-cs_FONIPA
    [236] => Any-ru
    [237] => Any-Bengali
    [238] => Any-Katakana
    [239] => Any-ro_FONIPA
    [240] => Any-zh
    [241] => Any-am
    [242] => Any-es_419_FONIPA
    [243] => Any-es_FONIPA
    [244] => Any-sk_FONIPA
    [245] => Any-Hant
    [246] => Any-Hans
    [247] => Any-Hiragana
    [248] => Any-Syriac
    [249] => Any-Greek
    [250] => Any-Greek/UNGEGN
    [251] => Any-Cyrillic
    [252] => Any-Hangul
    [253] => Any-Bopomofo
    [254] => Any-Arabic
    [255] => Any-Thai
    [256] => Any-Armenian
    [257] => Any-Thaana
    [258] => Any-Han
    [259] => Any-Georgian
    [260] => Any-Hebrew
    [261] => Any-pl_FONIPA

But I'm at a loss for what might be going wrong. Any help or ideas would be very appreciated!

[lonvia]

It looks like the PHP Transliterator class is not working as expected. What is the output for php -r 'var_dump(\Transliterator::createFromRules(":: lower ();"));' What PHP version do you use?

If the import was successful then the ICU version should be sufficient.

[marygriffus]

Output:

object(Transliterator)#1 (1) {
  ["id"]=>
  string(13) "RulesTransPHP"
}

And I'm using PHP v 7.4.33

[lonvia]

So the library is working.

Before anything else, please make sure that all of your system is using UTF-8, including php (check /etc/php/*/php.ini).

If encoding is fine, it probably won't like one of the rules. You will need to do a bit of debugging yourself here. Make a copy of tokenizer/tokenizer.php. In the copy, remove the line at the end of the file with require(..) and replace this with:

var_dump(\Transliterator::createFromRules(CONST_Term_Normalization_Rules));
var_dump(\Transliterator::createFromRules(CONST_Transliteration));

Run the file with php. One of the two lines should throw now. Trim the definition of the constant until the script runs fine.

[marygriffus]

Ok, php.ini already contained 'default_charset = "UTF-8"'. When I initially ran with the additional lines, I got NULL for each. I found that if I removed all rules involving ascii from each I'm now getting:

object(Transliterator)#1 (1) {
  ["id"]=>
  string(13) "RulesTransPHP"
}

for each, so.. will I be safe if I just remove those before running from the original? Or is there something I need to update elsewhere for ASCII usage?

Thank you so much for your help-- I'm sure you can tell, I am not a PHP user, so this would have taken a thousand times longer without you.

[lonvia]

I don't quite remember now which corner cases the ASCII rule is supposed to fix. I think you should be mostly fine without it.

It's still odd. The ASCII translators are clearly there in your list. Could you check if it fixes things when you write 'ASCII' like this in upper case?