StateChanged and Syncthing API

[enoryw]

Disclaimer: noob, no knowledge in programming. Apologies if not completely relevant to Syncthingtray itself, could not find answers elsewhere as Syncthing devs expect a level of competence I don't have wrt to using the API.

---

I was wondering how efficient is it for syncthingtray to constantly poll/listen for Syncthing activity and if this is suitable to run on a laptop configured for power saving (e.g. runs a tiling window manager, few GUI applications, etc.). Is such an application any different than typical desktop applications that perhaps do the same thing internal (e.g. constantly polling/listening for events)l?

Also, was looking into the Syncthing API and was curious how to actually use this and if Syncthingtray does the same internally. Basically, from reading the event API, to constantly listen for new activity, one would need to keep track of the most recent ID and use the event API to look for the next ID (most recent ID + 1) in a polling fashion? E.g. in bash:

• curl ... -X GET "https://127.0.0.1:8384/rest/events?events=StateChanged&since=0&limit=1" gets the latest recent ID and exits
• We want keep listening for the next event, so keep a count of the recent ID from the above command and run the same command over and over with the most recent ID incremented by 1?
• Run this infinite while loop (with sleep 1 to limit excessive CPU cycles?)

Is the above the correct way to use the API as intended? I don't see how this is different than using the REST API where you're still polling (repeatedly running the curl command) for event changes. I would think with the event API, I can see new events coming in without the request exiting. I can only assume this is not possible and you can only get one response per request and the event API (as opposed to the REST API) just means each individual response does not get exit immediately (since it has a default timeout of 60 seconds). So "polling" in programming terms only refer to the request itself whether it exits immediately or listens for the next response, but "polling" for each request (i.e. running the curl command repeatedly with the most recent ID incremented by one to listen for the next event) in an infinite sleep 1 loop is inevitable? This is what Syncthing and Syncthingtray does internally and it's as efficient as it gets right?

I don't necessarily need SyncthingTray to run in the background for the entirety of system use. For example, I only want to know whether a sync was made so that the system can sleep/shutdown or on startup before I begin working on the freshly synced files. Syncthingtray provides the --wait-for-idle but wouldn't it be more useful to provide an --until to only exit when the next state change to idle is detected? This option seems to assume the device is already in a non-idle state before it is run (with --at-least being an attempt to avoid this issue) or more likely it is used to "wait --at-least time, if after that device is idle, it is most likely up to sync". However, it seems an --until to only exit when the next state change to idle is more appropriate so you're not defining an arbitrary time that is prone to error if the value is too small (e.g. Syncthing requires more time time fo the next sync). This is basically what I've described earlier with constantly listening to the next event, except I should filter all the State(Changed events for "to": "idle", exiting the command when an event has this value, right?

Much appreciated and feel free to be as pedantic as possible, I"m just here to learn.

[Martchus]

could not find answers elsewhere as Syncthing devs expect a level of competence I don't have wrt to using the API.

The developers of Syncthing are in my experience very patient people. Don't be afraid to post in official forums. This makes much more sense for generic questions.

e.g. runs a tiling window manager, few GUI applications, etc.

It is indeed a well known fact that "tiling" windows instead of "stacking" them saves around one Watt per window.

I was wondering how efficient is it for syncthingtray to constantly poll/listen for Syncthing activity

From the GUI/client-perspective I'd say it is quite efficient. The Syncthing API uses "long polling" (as you've already noticed) so the GUI only needs to do one request per minute during idle periods. The long polling interval can maybe be tweaked via some configuration of Syncthing itself. Maybe this is also possible to tweak from the GUI/client-side but Syncthing Tray so far doesn't offer this.

We want keep listening for the next event, so keep a count of the recent ID from the above command and run the same command over and over with the most recent ID incremented by 1? … Is the above the correct way to use the API as intended?

I suppose that's how it works. I'm not sure about the increment (out of my head). To verify your understanding, you can simply open the official Syncthing web-based GUI and a web browser and option the network analysis tools. The requests that show up in the logs tell you exactly how the API is supposed to be used (without having to rely on your correct understanding of the wording of the documentation).

This is what Syncthing and Syncthingtray does internally and it's as efficient as it gets right?

I'm not exactly sure what you mean with your previous questions. In general the API usage of Syncthing Tray is as efficient as it gets.
You could check with tools like powertop whether this is efficient enough for you.

Syncthingtray provides the --wait-for-idle but wouldn't it be more useful to provide an --until to only exit when the next state change to idle is detected?

The idea is to trigger e.g. a rescan and then wait. In this case the initial state would be scanning or synchronizing or idle and in the case of idle it is assumed that scanning+synchronizing happened so fast that is was not noticed. Waiting for a transition to another state and going back to idle would lead to getting stuck in this case.

with --at-least being an attempt to avoid this issue

The --at-least flag is for the use case of waiting until all downloads from a remote have been concluded. Obviously files may change continuously on the remote but Syncthing only scans in intervals (even with inotify enabled it backs off for efficiency purposes). That means a folder may look idle although it actually isn't idle as it is still changing and the remote Syncthing still has to keep-up with those changes. I added the --at-least flag is to be able to be patient with the remote Syncthing instance in this case.

[Martchus]

Looks like the polling interval can be customized via the timeout parameter. Maybe I should expose a way to customize it because a higher timeout would be more efficient (at the cost of not being able to distinguish a stuck connection from just nothing happening for a longer time).

[rieje]

I wrote a simple shell script which seems to fit my purposes well and maybe you will find useful or adapt to something more general for syncthingctl.

Run the script without arguments and it will continuously output the folder ID along with whether its a "LocalChangeDetected" or "RemoteChangeDetected" if a full sync is detected. To check for full syncs, it looks at events and for every *ChangeDetected events, it will look for its folder and find corresponding StateChanged events with value "to: idle".

With the --exit-on-initial-timeout argument, it implements what I was looking for--it will exit when a full sync is detected or in 60 seconds, whichever comes first (setting this to 15 seconds is probably more reasonable, especially for my intended use case which is to run on startup and before shutdown).

Earlier I thought processing each event with a curl request updated with event ID incremented by 1 was intended, but it's only necessary to track the last event reported by Syncthing and update the curl request with that which is of course more efficient.

[Martchus]

Ok, I guess this could be useful for some users. Not sure what exactly is missing in syncthingctl compared to that script.

Note that I only added syncthingctl because at this point Syncthing had no official CLI itself. I added mainly features that were very easy to implement as I could reuse the internal library that I had already developed for the GUI. I did generally not add features that would had required extending the internal library. Today Syncthing itself provides a CLI. So it would make most sense to extend that - especially if the internal libraries of Syncthing Tray needed to be extended for that feature anyways.