Update failed! - unexpected character

[haaino]

Did you read the instructions?

• I have read the README, including the known-limitations section.

The problem

This error originated from a custom integration.

Logger: custom_components.knmi
Source: custom_components/knmi/coordinator.py:48
integration: KNMI (documentation, issues)
First occurred: 17:20:21 (8 occurrences)
Last logged: 18:35:21

Update failed! - unexpected character: line 1 column 316 (char 315)
Update failed! - unexpected character: line 1 column 313 (char 312)
Update failed! - unexpected character: line 1 column 315 (char 314)

Integration version

2.0.0

Home Assistant version

2024.3.0

Python version

3.12

Home Assistant log

No response

Diagnostics

No response

Additional information

No response

[golles]

Can you provide more info like raw log and diagnostics, please?
Have you tried receiving the JSON from the server manually? Do you know if it was valid?

This issue doesn't contain a lot of information I can work with.

[haaino]

home-assistant_knmi_2024-03-11T10-50-05.292Z.log
Attached the debuglogging. Hope this helps.

The corresponding logfile is:

---

This error originated from a custom integration.

Logger: custom_components.knmi
Source: custom_components/knmi/coordinator.py:48
integration: KNMI (documentation, issues)
First occurred: 10:22:19 (12 occurrences)
Last logged: 11:35:07

Update failed! - Timeout error fetching information
Update failed! - unexpected character: line 1 column 291 (char 290)
Update failed! - unexpected character: line 1 column 295 (char 294)

---- and ---

This error originated from a custom integration.

Logger: custom_components.knmi
Source: helpers/update_coordinator.py:350
integration: KNMI (documentation, issues)
First occurred: 11:20:07 (1 occurrences)
Last logged: 11:20:07

Error fetching knmi data:

---

[golles]

The Timeout error isn't great but as long as it isn't happening often it's no problem as the server wasn't able to respond within 10 seconds. I've seen this issue a few times as well and when it was happening more often a month ago, increasing the timeout (120s) wasn't helping as it would still timeout (and wait a lot longer).

unexpected character error is a little worrying, I haven't seen it myself. If I suggest a code change, to log the json, would you be able to apply that change locally? If it than happens again we have the json string and we can have a look what is wrong.

[haaino]

Sure.

If you give me instructions on what to do. Mind you, I'm not that technical.

[gerritjanf]

I run into the same issue. I am more technical. If you can provide instructions to help me catch the JSON i am willing to help.

[golles]

Thanks, in https://github.com/golles/ha-knmi/blob/main/custom_components/knmi/api.py#L65-L72 extend the 2 excepts with a third:

        except Exception as exception:
            import logging
            logging.getLogger(__package__).error("Response text: %s", response_text)
            raise exception

Make sure the indentation is the same as the other 2 excepts

This should log the entire json string to the log file.
Please provide these logs from the logfile and not from the frontend, thanks!

[koendiender]

I have the same problem and added the except. Here is my log

It looks like the API returns a "_" instead of a number for "luchtd" and "ldmmhg"

home-assistant_2024-03-14T18-28-42.984Z.log

[gerritjanf]

The JSON string:

2024-03-14 21:23:29.845 ERROR (MainThread) [custom_components.knmi] Response text: { "liveweer": [{"plaats": "Emmeloord", "timestamp": 1710447484, "time": "14-03-2024 21:18:04", "temp": 11.9, "gtemp": 8.7, "samenv": "Zwaar bewolkt", "lv": 83, "windr": "ZZO", "windrgr": 168.6, "windms": 4.59, "windbft": 3, "windknp": 8.9, "windkmh": 16.5, "luchtd": _, "ldmmhg": _, "dauwp": 9.1, "zicht": 34400, "gr": 0, "verw": "Vanavond kans op een bui. Morgen enkele buien, plaatselijk met onweer", "sup": "06:51", "sunder": "18:43", "image": "wolkennacht", "alarm": 0, "lkop": "Er zijn geen waarschuwingen", "ltekst": "Er zijn momenteel geen waarschuwingen van kracht.", "wrschklr": "groen", "wrsch_g": "-", "wrsch_gts": 0, "wrsch_gc": "-"}], "wk_verw":[{"dag":"14-03-2024","image":"halfbewolkt","max_temp":11,"min_temp":11,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":192,"windr":"Z","neersl_perc_dag":0,"zond_perc_dag":43},{"dag":"15-03-2024","image":"halfbewolkt","max_temp":12,"min_temp":10,"windbft":4,"windkmh":25,"windknp":14,"windms":7,"windrgr":222,"windr":"ZW","neersl_perc_dag":40,"zond_perc_dag":28},{"dag":"16-03-2024","image":"halfbewolkt","max_temp":9,"min_temp":5,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":254,"windr":"ZW","neersl_perc_dag":30,"zond_perc_dag":51},{"dag":"17-03-2024","image":"bewolkt","max_temp":10,"min_temp":5,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":233,"windr":"ZW","neersl_perc_dag":0,"zond_perc_dag":0},{"dag":"18-03-2024","image":"buien","max_temp":10,"min_temp":9,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":218,"windr":"ZW","neersl_perc_dag":40,"zond_perc_dag":4}],"uur_verw":[{"uur":"14-03-2024 22:00","timestamp":1710450000,"image":"helderenacht","temp":12,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":196,"windr":"Z","neersl":0,"gr":0},{"uur":"14-03-2024 23:00","timestamp":1710453600,"image":"nachtbewolkt","temp":12,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":201,"windr":"Z","neersl":0,"gr":0},{"uur":"15-03-2024 00:00","timestamp":1710457200,"image":"bewolkt","temp":12,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":199,"windr":"Z","neersl":0,"gr":0},{"uur":"15-03-2024 01:00","timestamp":1710460800,"image":"bewolkt","temp":12,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":198,"windr":"Z","neersl":0,"gr":0},{"uur":"15-03-2024 02:00","timestamp":1710464400,"image":"bewolkt","temp":12,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":213,"windr":"ZW","neersl":0,"gr":0},{"uur":"15-03-2024 03:00","timestamp":1710468000,"image":"regen","temp":12,"windbft":4,"windkmh":25,"windknp":14,"windms":7,"windrgr":208,"windr":"Z","neersl":0.3,"gr":0},{"uur":"15-03-2024 04:00","timestamp":1710471600,"image":"bewolkt","temp":12,"windbft":4,"windkmh":25,"windknp":14,"windms":7,"windrgr":213,"windr":"ZW","neersl":0,"gr":0},{"uur":"15-03-2024 05:00","timestamp":1710475200,"image":"bewolkt","temp":12,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":226,"windr":"ZW","neersl":0,"gr":0},{"uur":"15-03-2024 06:00","timestamp":1710478800,"image":"bewolkt","temp":11,"windbft":4,"windkmh":25,"windknp":14,"windms":7,"windrgr":216,"windr":"ZW","neersl":0,"gr":0},{"uur":"15-03-2024 07:00","timestamp":1710482400,"image":"regen","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":221,"windr":"ZW","neersl":0.1,"gr":0},{"uur":"15-03-2024 08:00","timestamp":1710486000,"image":"regen","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":223,"windr":"ZW","neersl":0.1,"gr":8},{"uur":"15-03-2024 09:00","timestamp":1710489600,"image":"bewolkt","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":225,"windr":"ZW","neersl":0,"gr":17},{"uur":"15-03-2024 10:00","timestamp":1710493200,"image":"halfbewolkt","temp":11,"windbft":4,"windkmh":25,"windknp":14,"windms":7,"windrgr":218,"windr":"ZW","neersl":0,"gr":141},{"uur":"15-03-2024 11:00","timestamp":1710496800,"image":"bewolkt","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":226,"windr":"ZW","neersl":0,"gr":233},{"uur":"15-03-2024 12:00","timestamp":1710500400,"image":"bewolkt","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":228,"windr":"ZW","neersl":0,"gr":127},{"uur":"15-03-2024 13:00","timestamp":1710504000,"image":"regen","temp":11,"windbft":5,"windkmh":32,"windknp":17,"windms":9,"windrgr":234,"windr":"ZW","neersl":0.1,"gr":144},{"uur":"15-03-2024 14:00","timestamp":1710507600,"image":"bewolkt","temp":11,"windbft":5,"windkmh":32,"windknp":17,"windms":9,"windrgr":227,"windr":"ZW","neersl":0,"gr":155},{"uur":"15-03-2024 15:00","timestamp":1710511200,"image":"regen","temp":11,"windbft":5,"windkmh":36,"windknp":19,"windms":10,"windrgr":229,"windr":"ZW","neersl":0.1,"gr":130},{"uur":"15-03-2024 16:00","timestamp":1710514800,"image":"bewolkt","temp":11,"windbft":5,"windkmh":36,"windknp":19,"windms":10,"windrgr":229,"windr":"ZW","neersl":0,"gr":172},{"uur":"15-03-2024 17:00","timestamp":1710518400,"image":"bewolkt","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":233,"windr":"ZW","neersl":0,"gr":169},{"uur":"15-03-2024 18:00","timestamp":1710522000,"image":"halfbewolkt","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":230,"windr":"ZW","neersl":0,"gr":55},{"uur":"15-03-2024 19:00","timestamp":1710525600,"image":"helderenacht","temp":11,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":229,"windr":"ZW","neersl":0,"gr":19},{"uur":"15-03-2024 20:00","timestamp":1710529200,"image":"helderenacht","temp":10,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":222,"windr":"ZW","neersl":0,"gr":0},{"uur":"15-03-2024 21:00","timestamp":1710532800,"image":"bewolkt","temp":10,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":221,"windr":"ZW","neersl":0,"gr":0}],"api":[{"bron":"Bron: Weerdata KNMI/NOAA via Weerlive.nl","max_verz":300,"rest_verz":22}]}
2024-03-14 21:23:29.845 ERROR (MainThread) [custom_components.knmi] Update failed! - unexpected character: line 1 column 268 (char 267)
2024-03-14 21:23:29.845 ERROR (MainThread) [custom_components.knmi] Error fetching knmi data: 

For me also the _ at 'luchtd' and 'ldmmhg' stands out and is at character 268.

[golles]

This is indeed an invalid JSON, "-" is still valid, but indeed a few values are missing the quotations.
I'll contact the supplier about this.

Thanks both for helping out, this still hasn't happened on my location so I wouldn't be able to reproduce this!

[gerritjanf]

No problem. Let me know if there is anything to test or further investigate.

[gerritjanf]

I notice the issue is intermitting on my side. A screenshot from my logbook

[golles]

If the JSON is consistently invalid, this is expected

[koendiender]

Just out of curiosity. Have you heard anything back from the supplier? The issue has not been fixed yet.

[golles]

They confirmed the issue and said they are going to look into it. I'm also investigating a workaround on this side

[koendiender]

Great! Thanks for the update

[golles]

This should now be fixed in the API.
Let me know if you still see this issue.

[koendiender]

It does not seem to be fixed in the API. Still get the same error

2024-03-27 16:57:28.914 ERROR (MainThread) [custom_components.knmi] Response text: { "liveweer": [{"plaats": "Zwolle", "timestamp": 1711554484, "time": "27-03-2024 16:48:04", "temp": 9.9, "gtemp": 8, "samenv": "Onbewolkt", "lv": 77, "windr": "ZZO", "windrgr": 150.7, "windms": 3.75, "windbft": 3, "windknp": 7.3, "windkmh": 13.5, "luchtd": _, "ldmmhg": _, "dauwp": 6.2, "zicht": 96900, "gr": 64, "verw": "Af en toe zon. In het oosten eerst regen", "sup": "06:19", "sunder": "19:04", "image": "zonnig", "alarm": 0, "lkop": "Er zijn geen waarschuwingen", "ltekst": "Er zijn momenteel geen waarschuwingen van kracht.", "wrschklr": "groen", "wrsch_g": "-", "wrsch_gts": 0, "wrsch_gc": "-"}], "wk_verw":[{"dag":"27-03-2024","image":"halfbewolkt","max_temp":11,"min_temp":7,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":184,"windr":"Z","neersl_perc_dag":0,"zond_perc_dag":24},{"dag":"28-03-2024","image":"halfbewolkt","max_temp":10,"min_temp":6,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":187,"windr":"Z","neersl_perc_dag":30,"zond_perc_dag":33},{"dag":"29-03-2024","image":"buien","max_temp":9,"min_temp":7,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":183,"windr":"Z","neersl_perc_dag":20,"zond_perc_dag":13},{"dag":"30-03-2024","image":"buien","max_temp":17,"min_temp":7,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":177,"windr":"Z","neersl_perc_dag":50,"zond_perc_dag":18},{"dag":"31-03-2024","image":"halfbewolkt","max_temp":14,"min_temp":5,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":166,"windr":"ZO","neersl_perc_dag":0,"zond_perc_dag":43}],"uur_verw":[{"uur":"27-03-2024 17:00","timestamp":1711555200,"image":"bewolkt","temp":9,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":179,"windr":"Z","neersl":0,"gr":152},{"uur":"27-03-2024 18:00","timestamp":1711558800,"image":"bewolkt","temp":9,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":163,"windr":"ZO","neersl":0,"gr":78},{"uur":"27-03-2024 19:00","timestamp":1711562400,"image":"bewolkt","temp":8,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":143,"windr":"ZO","neersl":0,"gr":33},{"uur":"27-03-2024 20:00","timestamp":1711566000,"image":"bewolkt","temp":7,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":144,"windr":"ZO","neersl":0,"gr":0},{"uur":"27-03-2024 21:00","timestamp":1711569600,"image":"bewolkt","temp":7,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":140,"windr":"ZO","neersl":0,"gr":0},{"uur":"27-03-2024 22:00","timestamp":1711573200,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":137,"windr":"ZO","neersl":0,"gr":0},{"uur":"27-03-2024 23:00","timestamp":1711576800,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":149,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 00:00","timestamp":1711580400,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":150,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 01:00","timestamp":1711584000,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":151,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 02:00","timestamp":1711587600,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":135,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 03:00","timestamp":1711591200,"image":"bewolkt","temp":6,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":125,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 04:00","timestamp":1711594800,"image":"bewolkt","temp":5,"windbft":2,"windkmh":10,"windknp":6,"windms":3,"windrgr":123,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 05:00","timestamp":1711598400,"image":"bewolkt","temp":5,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":135,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 06:00","timestamp":1711602000,"image":"bewolkt","temp":5,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":132,"windr":"ZO","neersl":0,"gr":0},{"uur":"28-03-2024 07:00","timestamp":1711605600,"image":"halfbewolkt","temp":5,"windbft":3,"windkmh":14,"windknp":8,"windms":4,"windrgr":115,"windr":"ZO","neersl":0,"gr":17},{"uur":"28-03-2024 08:00","timestamp":1711609200,"image":"regen","temp":5,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":144,"windr":"ZO","neersl":0.1,"gr":58},{"uur":"28-03-2024 09:00","timestamp":1711612800,"image":"regen","temp":5,"windbft":3,"windkmh":18,"windknp":10,"windms":5,"windrgr":151,"windr":"ZO","neersl":0.4,"gr":33},{"uur":"28-03-2024 10:00","timestamp":1711616400,"image":"bewolkt","temp":6,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":194,"windr":"Z","neersl":0,"gr":183},{"uur":"28-03-2024 11:00","timestamp":1711620000,"image":"regen","temp":6,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":223,"windr":"ZW","neersl":0.7,"gr":47},{"uur":"28-03-2024 12:00","timestamp":1711623600,"image":"regen","temp":6,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":216,"windr":"ZW","neersl":0.7,"gr":58},{"uur":"28-03-2024 13:00","timestamp":1711627200,"image":"bewolkt","temp":7,"windbft":4,"windkmh":21,"windknp":12,"windms":6,"windrgr":215,"windr":"ZW","neersl":0,"gr":122},{"uur":"28-03-2024 14:00","timestamp":1711630800,"image":"bewolkt","temp":8,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":223,"windr":"ZW","neersl":0,"gr":341},{"uur":"28-03-2024 15:00","timestamp":1711634400,"image":"bewolkt","temp":9,"windbft":5,"windkmh":28,"windknp":16,"windms":8,"windrgr":220,"windr":"ZW","neersl":0,"gr":424},{"uur":"28-03-2024 16:00","timestamp":1711638000,"image":"halfbewolkt","temp":10,"windbft":5,"windkmh":32,"windknp":17,"windms":9,"windrgr":210,"windr":"Z","neersl":0,"gr":366}],"api":[{"bron":"Bron: Weerdata KNMI/NOAA via Weerlive.nl","max_verz":300,"rest_verz":98}]}
2024-03-27 16:57:28.915 ERROR (MainThread) [custom_components.knmi] Update failed! - unexpected character: line 1 column 258 (char 257)

[golles]

That is odd, reopened

[Ascathon]

Guess you asked again after them telling saying it is fixed? Seems its quite often both air pressure values.
Nothing this integration should fixed, though I appreciate keeping this issue open for some tracking.

[golles]

I've got a PR ready to fix this, if anyone can test that and provide feedback, that would be great!
#141