Add API for triggering a speedtest.

[ChristophKronberger]

📃 Description

Added support for triggering speedtests using an API post request.
The client sends a POST request to /api/speedtest, and the API responds with a uuid that maps to a speedtest task.
The client can get the status of the task with a get request to /api/speedtest/trackingid/{id}.
The server responds with the current status of the speedtest, if not finished, or with the result of the speedtest.

Closes: #1047

🪵 Changelog

➕ Added

• Database table "job_trackings" to track the current status of the speedtest
• Model for JobTracking
• JsonResource for table results
• API route /api/speedtest/trackingid/{id}, supports GET
• API route /api/speedtest, supports POST
• MeasurementController Class
• ClearJobTrackingCommand, deletes tracking for failed speedtests and for completed speedtests older than x days. I plan to add this to the system schedule. @alexjustesen what do you think is a reasonable amount of time? i would say 180.

✏️ Changed

• Added 'tracked' and 'tracking_key' properties to ExecSpeedtest, added updteJobStatus to keep track of speedtest status, but only for API triggered speedtests.
• routes/api.php

Removed 🗑️

• GetLatestController -> moved functionality 1:1 to MeasurementController

[ChristophKronberger]

Im also thinking about securing the API with an Key, but i saw that the old enpoint was unsecured, so i didnt secure the new as well. What do you think about?

[alexjustesen]

Couple of thoughts so bare with me as I respond while trying to feed my daughter. I also noticed there are a couple "new" features in this PR, my preference is to have these in separate PR's as they make reviewing code and testing features easier. My first couple of comments come from that thought.

Feature feedback

1. Looks like we both had a thought about being able to track pending speedtests. I started that work here in my json dq PR and intend on finishing that "Feature" separately. For now I'd say let's remove that feature, I'll be adding it right to the results table. Source: https://github.com/alexjustesen/speedtest-tracker/pull/812/files#diff-2193f66a6992436e02719a3f5862bd99113ca6253473de0bde6ccd9b047e83fa
2. For pruning model records, Laravel provides a Prunable method we can attach to models. If you spike out that feature into a different PR and use the "Framework" way I'll get that pushed through quickly. From a process perspective I'd add a new setting to the UI under "General Settings" so a user can change that whenever they need to but disabled by default. [Feature] Automatically prune results #1144 will introduce pruning so you can remove it.

API feedback

1. The "old" API endpoint is used by Homepage and provides compatibility for those still running the other speedtest application. Their widget is currently setup to hit that route without authentication so we'll need to keep that in place.
2. I agree that we should have authentication (token based) in place for any API calls going forward. If you take the two features out of the branch from above I'll get us going an API working branch where we can merge in a large API feature after [Bug] Fix incorrectly formatted json data #812 is done.

[ChristophKronberger]

To your point about "many new features", I agree that the PR contains more than the API, but we need some way for the API caller to track the triggered speedtest, because we cannot just wait for the results to be returned, because with slow internet connections we expect timeouts. The caller then has no way to get information about the triggered test.

Personally, I would use a separate table for tracking speedtests, because I think it better meets the 2NF, and of course the result table should only contain information about the result of the speedtest.

API Feedback

1. Yes, the 'old' endpoint for the client has not changed, I just created a generic controller for speedtest results.

Can you please summarize what should be left of this feature?

[alexjustesen]

Getting the quick one out of the way, the existing API endpoint should be left as-is. Homepage expects a response at that URI with the given payload already.

For tracking the status of a requested speedtest walk me through the benefits of using 2NF and keeping history over not. I'm trying to keep an eye to simplicity (K.I.S.S.) so if we're going to add data and complexity I want to make sure it's going to be useful.

Edit: for anyone else who reads this and wonders what "2NF" is this stands for normal forms regarding the data structure inside of the database.

[ChristophKronberger]

I did not touch the existing API endpoint. It still returns the same resource in the same path. I just changed the location of the code, from a separate controller to one that also handles other requests.

A separate table allows you to quickly search the status of a speedtest.
If we add a column index to the tracking key, it will be even faster. Only when the tracking data says that the speedtest is finished, we have to query this large database only once.
Because of the frequent changes in the status of a speedtest, a column index on the result table is not practical.
But performance should not be the key decision, because if you run a speedtest every 15 minutes and store the speedtest for half a year, you will only get ~20,000 entries, which is not that much.

Keeping the job tracking information separate from the results data helps maintain a clear separation of concerns. This separation can make the schema easier to understand and maintain, especially as the system grows in complexity.

A separate table allows you to keep a history of all job attempts, statuses, and changes over time, which can be invaluable for debugging, auditing, and analysis.

A separate job_tracking table is easier to extend in the future with additional tracking-related attributes (e.g. started_at, finished_at, error_message) without cluttering the results table.

[alexjustesen]

Just doing a little house keeping, can you update your branch from integrate-an-api to pull in recent changes to resolve the conflicts?

I also need to get you a priorities list which is why I changed the base branch so we have a "working" branch.

[ChristophKronberger]

Did the merge, but now we have the status duplicated.
Did you decide how to proceed with that?

[alexjustesen]

I haven't, I've been focused hardcore on bringing stability back to v0.16.x and squashing bugs.

Give me a couple of days to get through client work so I have the mental capacity to give this the time it needs.

[alexjustesen]

@ChristophKronberger #1264 has some pretty fundamental changes into how the process works for running manual and scheduled speedtests.

My goal is to make the process standardized and more flexible when triggering manual or scheduled speedtests. This also includes when I go to implement multiple speedtest services (i.e. iPerf). My hope is this makes hooking into the process a whole lot easier.

Once this is wrapped up we should be able to put a nail in how we're going to track the status of a speedtest run and hook the API into this process to easily trigger a speedtest.

[alexjustesen]

@ChristophKronberger can you resolve conflicts? I want to get this merged into the api branch so I can leverage the work you've done and expand on it for the upcoming API.

[ChristophKronberger]

@alexjustesen
Sure, i will work on this next week, since im realy busy this week.