Master fix node country decoding #145
Conversation
Signed-off-by: mario <[email protected]>
Signed-off-by: mario <[email protected]>
Signed-off-by: mario <[email protected]>
Signed-off-by: mario <[email protected]>
src/mappings/nodes.ts
Outdated
| newNode.country = "" | ||
| } | ||
| if (newNode.city?.toString().startsWith('0x')) { | ||
| if (newNode.city?.includes('\x00')) { |
There was a problem hiding this comment.
The issue here is that one of the bytes we try to parse is the null character, which BTW a valid character in UTF-8. It is just PostgreSQL doesn't support storing NULL (\0x00) characters in text fields.
So would it better to patch this for the rest of text fields ? it mean more work but at least this won't come up again tomorrow when we get another null character in different field?
Also, would it be sufficient to only replace the null char instead of truncate the whole value ?
There was a problem hiding this comment.
sure, will add this to all text fields.
i don't really like replacing the null character as the processor is expected to parse the events and write them to the sql db without manipulation, so i would prefer that this happens on the chain's side if it was accepted at all.
There was a problem hiding this comment.
The input validation is a something need to happen on tfchain side but it would take sometime. I expect that upgrading the processor would be quicker than upgrading the runtime. otherwise, why we are patching this here at first place.
Beside, chain store bytes, indexer have no issue with encoding this, the value can be stored on different db with no issue or using postgersql bytea filed. so it is more about how the processor store this info and a limitation in postgersql. so we can indeed patch it here. make sense ?
Signed-off-by: mario <[email protected]>
Related issues