Expose Board via a HTTP Server

[Vido]

We just had an idea to expose the Board via a HTTP Server.
This would take a step further into the Internet of Things concept.

We have some o this on Yun. It has a HTTP server an a REST-like API.
Pingo could provide this server for all supported Boards.

[s-celles]

Great idea !

[ramalho]

Yes, great idea, @Vido! We can start by emulating exactly the RESTful API of the Yun, which will also be available in the Arduino Tre. Also, we can implement that as a POC using just BaseHttpServer but I believe we should move to integrate Trollius (the Tulip/asyncio backport to Python 2.7). Trollius is not only a solid foundation for the HTTP API but also for the asynchronous handling of inputs. People are using Node.js for IoT exactly for this reason: effective support for event-oriented programming.

[s-celles]

I also like asynchronous handling of inputs. Maybe Autobahn could help
http://autobahn.ws/python/websocket/programming.html
see also Crossbar http://crossbar.io/

some interesting links about WAMP, Autobahn and Crossbar (in french sorry):
http://sametmax.com/un-petit-gout-de-meteor-js-en-python/
http://sametmax.com/le-potentiel-de-wamp-autobahn-et-crossbar-io/
http://sametmax.com/crossbar-le-futur-des-applications-web-python/

caution: some links inside sametmax website might be "NSFW".

[s-celles]

Same idea could also apply to an other open source project: sigrok http://sigrok.org/
http://sigrok.org/bugzilla/show_bug.cgi?id=554

[s-celles]

For RESTful only API, this project can be interesting https://flask-restful.readthedocs.org

[Vido]

Related: #60
Using asyncio (or something similar) would provide tools to deal with interruptions.

[s-celles]

Maybe a first step (for REST API) could be to have most of Pingo objects JSON serializable.

import pingo
import json
board = pingo.detect.MyBoard()
json.dumps(board)

raises `<pingo.ghost.ghost.GhostBoard object at 0x1065213d0> is not JSON serializable``

same for other objects such as pins

led_pin = board.pins[13]
json.dumps(led_pin)

raises TypeError: <DigitalPin @13> is not JSON serializable

Here is some code with Python Flask_restful https://flask-restful.readthedocs.org/ and flask_restful_swagger https://github.com/rantav/flask-restful-swagger
API doc is autogenerated using Swagger
see for example: http://127.0.0.1:5000/api/spec.html

with server-restful.py

#!/usr/bin/env python
# -*- coding: utf-8 -*-

import pingo
from flask import Flask, Response, jsonify
from flask.ext import restful
from flask.ext.restful import Resource
from flask_restful_swagger import swagger
import click
import logging
import logging.config
import traceback
import json


"""
Auto generated API docs by flask-restful-swagger

http://127.0.0.1:5000/api/spec
returns JSON spec

http://127.0.0.1:5000/api/spec.html
displays HTML API doc

http://127.0.0.1:5000/api/v1/boards/%7Bboard%7D.help.json
"""

def bool_to_int(b):
    if b:
        return(1)
    else:
        return(0)

def json_error_response(debug, msg='', status=500, success=False):
    if msg=='':
        if debug:
            msg = traceback.format_exc()
        else:
            msg = "Internal server error"
    d = {"success": bool_to_int(success), "error": msg}
    dat_json = json.dumps(d)
    resp = Response(response=dat_json,
        status=status, \
        mimetype="application/json")
    return(resp)

def json_normal_response(dat, status=200, success=True):
    d = {"success": bool_to_int(success), "return": dat}
    dat_json = json.dumps(d)
    resp = Response(response=dat_json,
        status=status, \
        mimetype="application/json")
    return(resp)

class PingoBoards(Resource):
    @swagger.operation(
        notes='Returns boards',
        responseClass=dict
    )
    def get(self):
        return(json_normal_response(ws.app.board_instances))

class PingoBoard(Resource):
    @swagger.operation(
        notes='Returns a given board from {board}',
        responseClass=dict,
        parameters = [
            {
                "name": "board_instance",
                "paramType": "path",
                "dataType": "int",
                "description": "pins of a given board"
            }
        ]
    )
    def get(self, board):
        logging.info("get board from board %r" % board)
        status = 200

        try:
            if board in ws.app.board_instances.keys():
                data = ws.app.board_instances[board]
                data = board #data.pin_states # ToFix: pingo.ghost.ghost.GhostBoard object at 0x1065213d0> is not JSON serializable
                logging.info(data)
                logging.info(type(data))
                return(json_normal_response(data))
            else:
                if ws.api.app.debug:
                    msg = "Requested board is %r but current board should be in %r" % (board, ws.app.board_instances.keys())
                else:
                    msg = "Invalid board"
                return(json_error_response(ws.app.debug, msg))

        except:
            return(json_error_response(ws.app.debug))

        #logging.info(d)
        logging.info(dat_json)

class PingoPins(Resource):
    def get(self, board):
        pass

class PingoPin(Resource):
    def get(self, board, pin):
        pass

class PingoPinMode(Resource):
    def get(self, board, pin):
        pass

    def put(self, board, pin, mode):
        pass

class PingoPinState(Resource):
    def get(self, board, pin):
        pass

    def put(self, board, pin, state):
        pass

class PingoWebservice(object):
    def __init__(self, debug):
        self.app = Flask(__name__)
        self.api = swagger.docs(restful.Api(self.app), apiVersion='1.0')

        self.app.config.update(
            DEBUG=debug,
            JSONIFY_PRETTYPRINT_REGULAR=debug
        )
        self.app.board_instances = {}
        self.api.add_resource(PingoBoards, '/api/v1/boards/')
        self.api.add_resource(PingoBoard, '/api/v1/boards/<string:board>')
        #self.api.add_resource(PingoPins, '/api/v1/boards/<string:board>/pins/')
        #self.api.add_resource(PingoPin, '/api/v1/boards/<string:board>/pins/{pin_number}')
        #self.api.add_resource(PingoPinMode, '/api/v1/boards/<string:board>/pins/{pin_number}/mode') # get and set (put) but maybe we should only use get ?
        ##self.api.add_resource(PingoPinModeSet, '/api/v1/boards/<string:board>/pins/{pin_number}/mode/{mode}') # set mode using a get request ?
        #self.api.add_resource(PingoPinState, '/api/v1/boards/<string:board>/pins/{pin_number}/state') # get and set (put) but maybe we should only use get ?
        ##self.api.add_resource(PingoPinStateSet, '/api/v1/boards/<string:board>/pins/{pin_number}/state/{state}') # set mode using a get request ?

    def add_board(self, key, board):
        self.app.board_instances[key] = board
        logging.info(self.app.board_instances)

    def run(self, host):
        self.app.run(host=host)

@click.command()
@click.option('--host', default='127.0.0.1', \
    help="host ('127.0.0.1' or '0.0.0.0' to accept all ip)")
@click.option('--debug/--no-debug', default=False, help="debug mode")
def main(debug, host):
    global ws
    ws = PingoWebservice(debug)
    ws.add_board('default', pingo.detect.MyBoard())
    ws.run(host=host)

if __name__ == '__main__':
    logging.config.fileConfig("logging.conf")    
    logger = logging.getLogger("simpleExample")
    main()

with logging.conf

[loggers]
keys=root,simpleExample

[handlers]
keys=consoleHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_simpleExample]
level=DEBUG
handlers=consoleHandler
qualname=simpleExample
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

Run server using:

$ python server-restful.py --debug

Here is what a client code with requests could looks like

import requests
base_url = 'http://127.0.0.1:5000/api/v1'
endpoint = '/boards/default/pins/13'
url = base_url + endpoint
response = requests.get(url)

[ramalho]

@Vido, I do not believe have an entire board serializable is a prerequisite
for implementing a REST API. This complicates things a lot without having a
strong use case to justify it, as far as I can see. Unfortunately I do not
have time today to go deeper into this discussion...

On Mon, Feb 16, 2015 at 6:28 PM, scls19fr [email protected] wrote:

Maybe a first step (for REST API) could be to have most of Pingo objects
JSON serializable.

import pingo
import json
board = pingo.detect.MyBoard()
json.dumps(board)

raises <pingo.ghost.ghost.GhostBoard object at 0x1065213d0> is not JSON
serializable`

same for other objects such as pins

led_pin = board.pins[13]
json.dumps(led_pin)

raises TypeError: <DigitalPin @13> is not JSON serializable

Here is some code with Python Flask_restful and flask_restful_swagger
API doc is autogenerated using Swagger

#!/usr/bin/env python

-- coding: utf-8 --

import pingo
from flask import Flask, Response, jsonify
from flask.ext import restful
from flask.ext.restful import Resource
from flask_restful_swagger import swagger
import click
import logging
import logging.config
import traceback
import json

"""
Auto generated API docs by flask-restful-swagger
http://127.0.0.1:5000/api/spec
returns JSON spec
http://127.0.0.1:5000/api/spec.html
displays HTML API doc
http://127.0.0.1:5000/api/v1/boards/%7Bboard%7D.help.json
"""

def bool_to_int(b):
if b:
return(1)
else:
return(0)

def json_error_response(debug, msg='', status=500, success=False):
if msg=='':
if debug:
msg = traceback.format_exc()
else:
msg = "Internal server error"
d = {"success": bool_to_int(success), "error": msg}
dat_json = json.dumps(d)
resp = Response(response=dat_json,
status=status,
mimetype="application/json")
return(resp)

def json_normal_response(dat, status=200, success=True):
d = {"success": bool_to_int(success), "return": dat}
dat_json = json.dumps(d)
resp = Response(response=dat_json,
status=status,
mimetype="application/json")
return(resp)

class PingoBoards(Resource):
def get(self):
return(json_normal_response(ws.app.board_instances))

class PingoBoard(Resource):
@swagger.operation(
notes='Returns board pins from {board_instance}',
responseClass=dict,
parameters = [
{
"name": "board_instance",
"paramType": "path",
"dataType": "int",
"description": "pins of a given board"
}
]
)
def get(self, board):
logging.info("get board from board %r" % board)
status = 200

    try:
        if board in ws.app.board_instances.keys():
            data = ws.app.board_instances[board]
            data = board #data.pin_states
            logging.info(data)
            logging.info(type(data))
            return(json_normal_response(data))
        else:
            if ws.api.app.debug:
                msg = "Requested board is %r but current board should be in %r" % (board, ws.app.board_instances.keys())
            else:
                msg = "Invalid board"
            return(json_error_response(ws.app.debug, msg))

    except:
        return(json_error_response(ws.app.debug))

    #logging.info(d)
    logging.info(dat_json)

class PingoPin(Resource):
def get(self, board, pin):
pass

class PingoPinMode(Resource):
def get(self, board, pin, mode):
pass

#def put(self, board, pin, mode):

class PingoWebservice(object):
def init(self, debug):
self.app = Flask(name)
self.api = swagger.docs(restful.Api(self.app), apiVersion='1.0')

    self.app.config.update(
        DEBUG=debug,
        JSONIFY_PRETTYPRINT_REGULAR=debug
    )
    self.app.board_instances = {}
    self.api.add_resource(PingoBoards, '/api/v1/boards/')
    self.api.add_resource(PingoBoard, '/api/v1/boards/<string:board>')
    #self.api.add_resource(PingoPins, '/api/v1/boards/<string:board>/pins/')
    #self.api.add_resource(PingoPin, '/api/v1/boards/<string:board>/pins/{pin_number}')
    #self.api.add_resource(PingoPinMode, '/api/v1/boards/<string:board>/pins/{pin_number}/mode') # get and set (put)
    #self.api.add_resource(PingoPinState, '/api/v1/boards/<string:board>/pins/{pin_number}/state') # get and set (put)

def add_board(self, key, board):
    self.app.board_instances[key] = board
    logging.info(self.app.board_instances)

def run(self, host):
    self.app.run(host=host)

@click.command()
@click.option('--host', default='127.0.0.1',
help="host ('127.0.0.1' or '0.0.0.0' to accept all ip)")
@click.option('--debug/--no-debug', default=False, help="debug mode")
def main(debug, host):
global ws
ws = PingoWebservice(debug)
ws.add_board('default', pingo.detect.MyBoard())
ws.run(host=host)

if name == 'main':
logging.config.fileConfig("logging.conf")
logger = logging.getLogger("simpleExample")
main()

with logging.conf

[loggers]
keys=root,simpleExample

[handlers]
keys=consoleHandler

[formatters]
keys=simpleFormatter

[logger_root]
level=DEBUG
handlers=consoleHandler

[logger_simpleExample]
level=DEBUG
handlers=consoleHandler
qualname=simpleExample
propagate=0

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=simpleFormatter
args=(sys.stdout,)

[formatter_simpleFormatter]
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
datefmt=

—
Reply to this email directly or view it on GitHub
https://github.com/garoa/pingo/issues/68#issuecomment-74567817.

Luciano Ramalho
Twitter: @ramalhoorg

Professor em: http://python.pro.br
Twitter: @pythonprobr

[s-celles]

For async API this could help

"There is a complete example for 2 way comms to GPIO on the Pi : https://github.com/crossbario/crossbarexamples/tree/master/device/pi/gpio
No docs though the code is pretty straight."

from Tobias Oberstein (WAMP / Autobahn / Crossbar dev)

[lamenezes]

I started writing the REST API for accessing the board via HTTP. The code is here: http://git.io/vZ9PU

There is still a lot to do, but it is working for basic I/O. I tested it on my Raspberry Pi and lit a led and read from a button via HTTP.

[Vido]

@lamenezes ,

There is a debate wheather we should use Flask or Bottle.

When I wrote the mockup YúnBridge, I used Flask:
https://github.com/pingo-io/pingo-py/blob/master/scripts/yunserver.py
But @ramalho suggested Bottle, because we can ship it within Pingo's package.

Flask and Bottle are very similar. They are kind of the same thing.
But Bottle's single-file approach can lead into a more batteries-included design.

[lamenezes]

I didn't know about the difference about Flask and Bottle, but from what you said and a little research I think it is a good idea using Bottle. Shipping bottle within pingo will allow simpler installation and usage compared to flask.

[s-celles]

Hi @lamenezes ,

I noticed your PR #92

These 2 links:

• http://bottlepy.org/docs/dev/async.html
• https://github.com/zeekay/bottle-websocket

could help to stream data which is a great feature for Internet Of Things.

Kind regards

[lamenezes]

Great ideia, @scls19fr.

It really is a great feature for IoT. But what async framework is the best for pingo? Tornado? Twisted? gevent?

Look at this @Vido, WebSocket solves that problem we were discussing at garoa. Now we may be able to use PWM (and other things) over the HTTP which sounds very cool to me.

[Vido]

In fact,

It's possible to have PWM, even with REST. Because It's a hardware/lib
feature. Not a Pingo implemente feature.

If you try to switch on and off a pin with WebSocket, I believe (guesswork)
the maximum frequency would be < 100Hz. A DMA implemented PWM frequency can
easily operate at 44kHz

Thanks.

On Fri, Sep 25, 2015 at 8:33 AM, Luiz [email protected] wrote:

It really is a great feature for IoT. But what async framework is the best
for pingo? Tornado? Twisted? gevent?

WebSocket solves that problem we were discussing at garoa. Now we may be
able to use PWM over the HTTP which sounds very cool to me.

—
Reply to this email directly or view it on GitHub
#68 (comment).