Separate client, server, lib.

The repository is now split into 3 parts: grpc, grpc_client and
grpc_lib. This way the dependencies can be handled better.

.gitignore

@@ -1,4 +1,6 @@
 _build
+.erlang.mk
+grpc.d
 .eunit
 deps
 *.o

Makefile

@@ -5,7 +5,11 @@ PROJECT_VERSION = 0.1.0
 # Whitespace to be used when creating files from templates.
 SP = 4
 
-DEPS = gpb cowboy chatterbox
+DEPS = cowboy grpc_lib
 dep_cowboy = git https://github.com/willemdj/cowboy
-dep_chatterbox = git https://github.com/willemdj/chatterbox
+dep_grpc_lib = git https://github.com/Bluehouse-Technology/grpc_lib
+
+TEST_DEPS = http2_client
+dep_http2_client = git https://github.com/Bluehouse-Technology/http2_client
+
 include erlang.mk

README.md

@@ -1,6 +1,8 @@
 # gRPC
 
-An implementation of gRPC in Erlang.
+An implementation of a gRPC server in Erlang. 
+
+An implementation of the client side is alo available: [grpc_client](https://github.com/Bluehouse-Technology/grpc_client).
 
 ## Build
 gRPC uses [erlang.mk](https://erlang.mk/) as build tool. On Unix systems it can be built
@@ -11,7 +13,7 @@ make
 ```
 
 `make doc` can be used to generate documentation for the individual
-modules from edoc comments in thos modules.
+modules from edoc comments in those modules.
 
 See the [erlang.mk documentation](https://erlang.mk/guide/installation.html#_on_windows) for
 an explanation on how the tool can be used in a Windows environment.
@@ -20,19 +22,17 @@ an explanation on how the tool can be used in a Windows environment.
 `make ct` can be used to run a number of tests. 
 
 In the test directory there is an explanation how the software can be
-tested against the go grpc implementation.
+tested against the go gRPC implementation.
 
 ## Dependencies
 
-- [gpb](https://github.com/tomas-abrahamsson/gpb) is used to encode and
-  decode the protobuf messages.
-
 - [cowboy](https://github.com/willemdj/cowboy) is used for the server.
   This is a fork with some changes to the support for HTTP/2.
 
-- [chatterbox](https://github.com/willemdj/chatterbox) is used for the
-  client. This is also a fork with a couple of modifications related to missing
-  HTTP/2 features.
+- [gpb](https://github.com/tomas-abrahamsson/gpb) is used to encode and
+  decode the protobuf messages. This is a 'build' dependency: gpb is
+  required to create some modules from the .proto files, but the modules
+  are self contained and don't have a runtime depedency on gpb.
 
 ## gRPC functionality
 

doc/tutorial.md

@@ -1,8 +1,14 @@
 <p class="lead">This tutorial provides a basic Erlang programmer's introduction to working with gRPC.</p>
 
-(This is a version for Erlang of the tutorial that is available for several
-other languages on grpc.io, see for example [the C
-version](http://www.grpc.io/docs/tutorials/basic/c.html))
+> _This is an Erlang version of the tutorial that is available for several
+> other languages on grpc.io, see for example
+> [the C version](http://www.grpc.io/docs/tutorials/basic/c.html).
+>
+> The tutorial covers both the client and the server side of the Erlang gRPC
+> implementation. The implementations are in different repositories:
+>
+> - Client: [grpc_client](https://github.com/Bluehouse-Technology/grpc_client)
+> - Server: [grpc](https://github.com/Bluehouse-Technology/grpc)_
 
 By walking through this example you'll learn how to:
 
@@ -158,15 +164,11 @@ Running this command generates the following files in your current directory:
 
 -  `route_guide.erl`, the code to encode and decode the protobuf messages
 -  `route_guide_server.erl`, which contains skeleton functions for a server
--  `route_guide_client.erl`, which contains stubs to be used on the server
-   side. 
 
 These contain:
 
 - All the protocol buffer code to populate, serialize, and retrieve our request
   and response message types
-- a set of functions (or *stubs*) for clients to call to invoke the methods
-  defined in the `RouteGuide` service.
 - a set of ("empty") functions (or *skeleton functions*) in which you have
   to provide your implementation of the server side of the methods 
   defined in the `RouteGuide` service.
@@ -189,7 +191,7 @@ There are two parts to making our `RouteGuide` service do its job:
   service responses.
 
 You can find our example `RouteGuide` server in
-[examples/route_guide/server/route_guide_server.erl](/examples/route_guide/server/route_guide_server.erl).
+[examples/route_guide/server/route_guide_server_1.erl](/examples/route_guide/server/route_guide_server_1.erl).
 Let's take a closer look at how it works.
 
 ### Implementing RouteGuide
@@ -206,7 +208,7 @@ this fits well for *simple RPC* requests, but perhaps less so for
 standard Erlang concepts it is not difficult to implement an *a-synchonous*
 version of the service, but we won't consider that in this tutorial.
 
-`route_guide_server.erl` contains a set of type specifications for the
+`route_guide_server_1.erl` contains a set of type specifications for the
 messages (they will be translated to and from Erlang maps), and functions
 for all our service methods. Let's look at the simplest
 type first, `GetFeature`, which just gets a `Point` from the client and returns
@@ -371,22 +373,30 @@ the `route_guide_server` module and the generated encoder/decoder module
 `route_guide` are on the code path.
 
 ```erlang
-grpc:start_server(grpc, route_guide_server, [{port, 10000}]).
+5> {ok, Server} = grpc:start_server(grpc, tcp, 10000, route_guide_server_1).
 ```
 
 The first argument is the name of the server (strictly speaking: the Cowboy
-listener). It can be any term. The second argument specifies the name of
-the handler module. In this case we also need to specify the port, because
-the `RouteGuide` service does not run on the default port. There are more
-options, for example to enable compression or authentication, see *TODO*
-for the details.
+listener). It can be any term. The second argument specifies the transport
+layer. It can be `tcp` or `ssl`. The third argument specifies the port and
+the fourth argument is the handler module. As an optional fifth argument
+some options can be provided, for example to enable compression or
+authentication, details about that are provided in separate sections below.
 
 <a name="client"></a>
 
 ## Creating the client
 
 In this section, we'll look at creating an Erlang client for our `RouteGuide`
-service. 
+service.  Note that the Erlang client is in a different repository:
+[grpc_client](https://github.com/Bluehouse-Technology/grpc_client).
+
+As for the server, a client module can be generated by compile the .proto
+file: 
+
+```erlang
+1> grpc_client:compile("route_guide.proto").
+```
 
 ### Creating a connection
 
@@ -405,7 +415,7 @@ for the connection, such as TLS related options. In
 our case we'll use the simple form without TLS or other options.
 
 ```erlang
-1> {ok, Connection} = grpc_client:connect(http, "localhost", 10000).
+1> {ok, Connection} = grpc_client:connect(tcp, "localhost", 10000).
 ```
 
 Now we can use the connection to create a "stream". The stream is specific
@@ -430,7 +440,7 @@ server sends back a response (on the wire you would also see some
 headers being exchanged). Since this is such simple pattern, the whole
 exchange of messages can be handled by one function call:
 `grpc_client:unary/6`. We don't even
-have to bother about establishing a stream, this will all be handled by
+have to bother about establishing (and stopping) a stream, this will all be handled by
 `unary`.
 
 ```erlang
@@ -486,7 +496,7 @@ In all cases we start by initiating a connection, and on that connection a strea
 Let's start one for the 'RouteChat' RPC, which is a bidirectional streaming RPC.
 
 ```erlang
-{ok, Connection} = grpc_client:connect(http, "localhost", 10000),
+{ok, Connection} = grpc_client:connect(tcp, "localhost", 10000),
 {ok, Stream} = grpc_client:new_stream(Connection, 'RouteGuide', 'RouteChat', route_guide).
 ```
 
@@ -499,6 +509,7 @@ P1 = #{latitude => 1, longitude => 2},
 P2 = #{latitude => 3, longitude => 5},
 grpc_client:send(Stream, #{location => P1, message => "something about P1"}),
 grpc_client:send(Stream, #{location => P2, message => "something about P2"}).
+
 ```
 
 Now we can see whether we have already received and responses:
@@ -508,8 +519,8 @@ grpc_client:get(Stream).
 ```
 
 This returns `empty` - we did not yet receive anything. This is not
-surprising, because the server should only send a message if it receives
-information about a point that it nows something about.
+surprising, because the server will only send a message if it receives
+information about a point that it knows something about.
 
 We can also check using the blocking `rcv` function. Let's specify a
 timeout to avoid getting stuck.
@@ -524,7 +535,7 @@ When we send some additional information for P1, we should get a response
 message with the information that we sent earlier.
 
 ```erlang
-grpc_client:send(Stream, #{location => P1, message => "more about P1"}),
+grpc_client:send(Stream, #{location => P1, message => "more about P1"}).
 ```
 
 Now try to get a message:
@@ -536,7 +547,7 @@ grpc_client:get(Stream).
 Now we get: 
 
 ```erlang
-{headers,[{<<":status">>,<<"200">>}]}`. 
+{headers,#{<<":status">>,<<"200">>}}`. 
 ```
 
 Calling it again gives: 
@@ -549,7 +560,8 @@ Calling it again gives:
 So we do not only get the messages this way, but also the headers.
 
 Bi-directional streaming RPCs such as `RouteChat` have no specific end,
-they can go on forever and there is no particular way to stop them. This is
+they can go on forever and there is no particular way to stop them, apart
+from stopping the stream (using `grpc_client:stop_stream(Stream).`) This is
 different for the other RPC types: there the client has to signal that it
 has sent its last message, and the server may send a trailer which may
 contain additional metadata. 
@@ -583,7 +595,7 @@ eof
 ```
 
 You will notice a few things:
--  `sent_last/2` is used to indicate that this message is the last one. For
+-  `send_last/2` is used to indicate that this message is the last one. For
    the `ListFeatures` we now expect to receive a number of response
    messages.
 -  As long as no anwer has been received, `get/1` returns `empty`. (Using
@@ -594,7 +606,7 @@ You will notice a few things:
    this example they do not contain anything special, but they may contain
    metadata that may be relevant.
 -  The server sends headers before the first message and after the last
-   message. Thes headers are returned by `get/1` and `rcv/1` in the same
+   message. These headers are returned by `get/1` and `rcv/1` in the same
    way, that is, separately and in the right order.
 -  After the last set of headers has been received the stream is considered 
    closed, and both `get/1` and `rcv/1` return `eof`.
@@ -616,21 +628,22 @@ domain. Note that you must ensure that the required modules
 (`route_guide_server` and also `route_guide`, the decoding module) are on
 the erlang code path, and the location of the certificate files must also
 be correct. The example certificates can be found in
-examples/route_guide/server/certificates.
+test/certificates.
 
 ```erlang
-grpc:start_server(grpc, route_guide_server, 
-    [{port, 10000},
-     {tls_options, [{certfile, "certificates/localhost.crt"},
-                    {keyfile, "certificates/localhost.key"},
-                    {cacertfile, "certificates/My_Root_CA.crt"}]}]).
+grpc:start_server(grpc, ssl, 10000, route_guide_server_1, 
+    [{transport_options, [{certfile, "certificates/localhost.crt"},
+                          {keyfile, "certificates/localhost.key"},
+                          {cacertfile, "certificates/My_Root_CA.crt"}]}]).
+
 ```
 
 On the client side we must now also use TLS:
 
 ```erlang
-1> {ok, Connection} = grpc_client:connect(tls, "localhost", 10000).
-2> {ok, S} = grpc_client:new_stream(Connection, 'RouteGuide', 'ListFeatures', route_guide).
+1> application:ensure_all_started(ssl).
+2> {ok, Connection} = grpc_client:connect(ssl, "localhost", 10000, []).
+3> {ok, S} = grpc_client:new_stream(Connection, 'RouteGuide', 'ListFeatures', route_guide).
 3> grpc_client:send_last(S, #{hi => P1, lo => P2}).
 ok
 4> grpc_client:get(S).
@@ -640,29 +653,44 @@ ok
 ...
 ```
 
+To ensure that the client checks the identity of the serve, provide a few
+extra options to enforce the exchange of certificates and to tell the
+client to perform the check:
+
+```erlang
+6> SslOptions = [{cacertfile, "certificates/My_Root_CA.crt"},{fail_if_no_peer_cert, true}].
+7> Options = [{verify_server_identity, true}, {transport_options, SslOptions}].
+8> {ok, C} = grpc_client:connect(ssl, "localhost", 10000, Options).
+```
+
+This will check the subject of the certificate against the host name
+("localhost", in this case). If you know that the certifcate will have
+another subject, the option 'server_host_override' can be used to provide
+that name.
+
 To tell the server that it must authenticate the client, provide an extra
 option 'client_cert_dir' which will indicate where the required
-certificates are, and we must pass a few extra tls_options to ensure that
-the auhtentication is enforced:
+certificates are, and we must pass a few extra transport_options to ensure that
+the authentication is enforced. So the server will be started like this:
 
 ```erlang
-grpc:start_server(grpc, route_guide_server, 
-    [{port, 10000},
-     {client_cert_dir, "certificates"},
-     {tls_options, [{certfile, "certificates/localhost.crt"},
-                    {keyfile, "certificates/localhost.key"},
-                    {cacertfile, "certificates/My_Root_CA.crt"},
-                    {fail_if_no_peer_cert, true},
-                    {verify, verify_peer}]}]).
+grpc:start_server(grpc, ssl, 10000, route_guide_server_1, 
+    [{client_cert_dir, "certificates"},
+     {transport_options, [{certfile, "certificates/localhost.crt"},
+                          {keyfile, "certificates/localhost.key"},
+                          {cacertfile, "certificates/My_Root_CA.crt"},
+                          {fail_if_no_peer_cert, true},
+                          {verify, verify_peer}]}]).
 ```
 
 The client must now pass its certificate:
 
 ```erlang
-1> TlsOptions = [{certfile, "certificates/127.0.0.1.crt"},
+1> SslOptions = [{certfile, "certificates/127.0.0.1.crt"},
                  {keyfile, "certificates/127.0.0.1.key"},
                  {cacertfile, "certificates/My_Root_CA.crt"}].
-2> {ok, Connection} = grpc_client:connect(tls, "localhost", 10000, TlsOptions).
+2> {ok, Connection} = grpc_client:connect(ssl, "localhost", 10000,
+[{transport_options, SslOptions}]).
 3> {ok, S} = grpc_client:new_stream(Connection, 'RouteGuide', 'ListFeatures', route_guide).
 4> grpc_client:send_last(S, #{hi => P1, lo => P2}).
 ...
@@ -680,9 +708,8 @@ To add metadata to a request (from the client to the server), specify the
 key-value pairs when creating the stream:
 
 ```erlang
-Metadata = #{"password" => "secret"},
-{ok, Stream} = grpc_client:new_stream(Connection, 'RouteGuide',
-'RouteChat', route_guide, Metadata).
+Metadata = #{<<"password">> => <<"secret">>},
+{ok, Stream} = grpc_client:new_stream(Connection, 'RouteGuide', 'RouteChat', route_guide, [{metadata, Metadata}]).
 ```
 
 On the server side, the metadata that was sent by the client can be
@@ -734,7 +761,7 @@ messages that it will receive (also the "standard" HTTP/2 and grpc headers such
 ":status" and "grpc-status" are available in this way).
 
 ```erlang
-1> {ok, C} = grpc_client:connect(http, "localhost", 10000).
+1> {ok, C} = grpc_client:connect(tcp, "localhost", 10000).
 ...
 2> {ok, Stream} = grpc_client:new_stream(C, 'RouteGuide', 'ListFeatures', route_guide),
 ...