Coordinate conversion implemented

[gabrielburnworth]

Conversion from pixel locations to machine coordinates has been implemented.

Usage:

detect_plants("soil_image.jpg", blur=15, morph=6, iterations=4,
             calibration_img="pixel_to_coordinate/p2c_test_calibration.jpg",
             known_plants=[[800, 200, 100], [1130, 600, 120]])

Example output shown below used a rotated calibration image.

Example image output:

Green = known plant
Blue = saved plant
Red = plant marked for removal

Example text output:

Processing image: soil_image.jpg
14 plants detected in image.
Detected object machine coordinates ( X Y ) with R = radius:
    (  1224   115 ) R = 28
    (   677   126 ) R = 33
    (   826   147 ) R = 28
    (  1077   220 ) R = 21
    (   870   238 ) R = 23
    (  1132   280 ) R = 24
    (  1178   317 ) R = 5
    (   950   438 ) R = 27
    (  1291   473 ) R = 31
    (   777   507 ) R = 31
    (   420   552 ) R = 32
    (  1140   602 ) R = 53
    (   303   644 ) R = 23
    (   309   843 ) R = 32

2 known plants inputted.
Plants at the following machine coordinates ( X Y ) with R = radius are to be saved:
    (   800   200 ) R = 100
    (  1130   600 ) R = 120

11 plants marked for removal.
Plants at the following machine coordinates ( X Y ) with R = radius are to be removed:
    (  1224   115 ) R = 49
    (   677   126 ) R = 57
    (  1077   220 ) R = 35
    (  1132   280 ) R = 41
    (  1178   317 ) R = 8
    (   950   438 ) R = 47
    (  1291   473 ) R = 54
    (   777   507 ) R = 53
    (   420   552 ) R = 55
    (   303   644 ) R = 39
    (   309   843 ) R = 55

3 detected plants are known or have escaped removal.
Plants at the following machine coordinates ( X Y ) with R = radius have been saved:
    (   826   147 ) R = 47
    (   870   238 ) R = 39
    (  1140   602 ) R = 92

[roryaronson]

Awesome! This program could query the SQL database on the Pi for the known plant coordinates, and also write the weed coordinates it finds to the db. Then the Pi controller can use those coordinates when executing a "remove weeds" sequence.

I think the easiest way to implement this would be to have a "Run Script" block on the web app/controller. This way this (and other) scripts/programs could be easily called from the web app and executed just like any other block. The scripts could be in any language and do whatever. Eventually scripts could be given/denied permissions to interact with the web API if they needed it. Or with the Pi's database. Ideally there would be a way for the script to emit a status that the controller would listen for. For example: "Working...", "Error: 123", "Done. Result = XYZ", which could then be added to the main FarmBot logs and sent back to the web app too. Each script would have to have an execution time limit so that if it misbehaves or hangs, the pi controller could move on. Maybe 30 seconds as a default.

With this framework the web app/FarmBot is opened up to all sorts of functionality, especially from third parties. Rather than having to add new functionality to the pi controller directly, one can simply add whatever scripts they want without having to have changes made to the more generic controller codebase. There could be a "script store"! Popular scripts could be implemented "natively" in the pi controller over time.

cc: @RickCarlino

[RickCarlino]

Looks great, @gabrielburnworth !! 🎆

We have a couple of options for this, and all of them revolve around the concept of adding "pluggable modules" to the RPI.

OPTION I: ENV + STDOUT

If you've done web app development in the 90's or early 00's, you might remember CGI scripts. CGI scripts offered a way to add custom behavior to web servers without modifying the web server. This pattern worked really well for creating light dynamic behavior. It might work well for what we're trying to do right now.

The premise was:

1. You write a script in any language, usually with a shebang line at the top of the file so that the host program knows how to run the script.
2. The host passes all "inputs" in via ENV vars. Eg, if you needed to know Farmbot's current x/y you could call os.environ['FARMBOT_X'] from Python.
3. The script does some sort of calculation, printing the results to STDOUT.
4. The script terminates.

Advantages

• Really easy to implement on my end.
• Easy to write plugins with almost 0 config needed.
• Can support every programming language out there.

Disadvantages

• Only useful one-off calculations.
• Might not be as simple for long running scripts, or scripts that take more than one step to complete.
• Script needs to restart every time it runs, causing performance issues (this is why people stopped writing CGI scripts for the web).

Option II: Unix / TCP Sockets.

Another approach would be to expose a TCP or Unix Domain socket as a means of reading / writing information to the bot.

The socket would provide a command line interface to the bot, similar to the way one would connect to a SQL database, Memcached, Redis etc.

1. Open a socket.
2. Talk to the bot the same way you would talk to a SQL database or Redis server, etc.
3. Terminate socket connection when finished.

Advantages

• We get a command line interface to the bot for free (could be nice for debugging / SSHing in).
• More flexible (long running processes are NBD).
• More performant (scripts stay open instead of re-initializing every time).

Disadvantages

• Might require configuration, which is evil.
• Modules will be more difficult to write because they don't follow a request / response pattern.
• Will take more time to implement than solution I.

Option III: Something Else?

I'm just spit balling ideas at this point. Let me know if you had a different idea.

Ideal outcomes:

• Language agnostic.
• Minimal configuration.
• Extensible.
• Avoid re-inventing the wheel if possible (IPC is mostly a solved problem).

CC: @roryaronson

[roryaronson]

What about doing both solution 1 and 2 (in the long run)? We could start with 1 because its fast and easy and we don't really need to worry about performance or long-running scripts right now. As you said, it would be easier for people to get their scripts working with it too. Later we can implement 2 as a second option for scripts that would benefit from it.