Merge pull request #136 from instana/instrument_db_sql

Add instrumentation for database/sql

README.md

@@ -7,11 +7,36 @@ The Instana Go sensor consists of two parts:
 
 * metrics sensor
 * [OpenTracing](http://opentracing.io) tracer
+* AutoProfile™ continuous profiler
 
 [![Build Status](https://travis-ci.org/instana/go-sensor.svg?branch=master)](https://travis-ci.org/instana/go-sensor)
-[![GoDoc](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue)](https://pkg.go.dev/github.com/instana/go-sensor)
+[![GoDoc](https://img.shields.io/static/v1?label=godoc&message=reference&color=blue)][pkg.go.dev]
 [![OpenTracing Badge](https://img.shields.io/badge/OpenTracing-enabled-blue.svg)](http://opentracing.io)
 
+## Table of Contents
+
+* [Installation](#installation)
+  * [Running on AWS Fargate](#running-on-aws-fargate)
+  * [Using Instana to gather process metrics only](#using-instana-to-gather-process-metrics-only)
+* [Common Operations](#common-operations)
+  * [Setting the sensor log output](#setting-the-sensor-log-output)
+  * [Trace Context Propagation](#trace-context-propagation)
+  * [HTTP servers and clients](#http-servers-and-clients)
+    * [Instrumenting HTTP request handling](#instrumenting-http-request-handling)
+    * [Instrumenting HTTP request execution](#instrumenting-http-request-execution)
+  * [Database Calls](#database-calls)
+    * [Instrumenting sql\.Open()](#instrumenting-sqlopen)
+    * [Instrumenting sql\.OpenDB()](#instrumenting-sqlopendb)
+  * [GRPC servers and clients](#grpc-servers-and-clients)
+  * [Kafka producers and consumers](#kafka-producers-and-consumers)
+* [OpenTracing](#opentracing)
+* [W3C Trace Context](#w3c-trace-context)
+* [Events API](#events-api)
+* [AutoProfile™](#autoprofile)
+  * [Activation from within the application code](#activation-from-within-the-application-code)
+  * [Activation without code changes](#activation-without-code-changes)
+* [Examples](#examples)
+
 ## Installation
 
 To add Instana Go sensor to your service run:
@@ -25,11 +50,22 @@ To activate background metrics collection, add following line at the beginning o
 ```go
 func main() {
 	instana.InitSensor(instana.DefaultOptions())
-    
-    // ...
+
+	// ...
 }
 ```
 
+The `instana.InitSensor()` function takes an `*instana.Options` struct with the following optional fields:
+
+* **Service** - global service name that will be used to identify the program in the Instana backend
+* **AgentHost**, **AgentPort** - default to `localhost:42699`, set the coordinates of the Instana proxy agent
+* **LogLevel** - one of `Error`, `Warn`, `Info` or `Debug`
+* **EnableAutoProfile** - enables automatic continuous process profiling when `true`
+* **MaxBufferedSpans** - the maximum number of spans to buffer
+* **ForceTransmissionStartingAt** - the number of spans to collect before flushing the buffer to the agent
+* **MaxBufferedProfiles** - the maximum number of profiles to buffer
+* **IncludeProfilerFrames** - whether to include profiler calls into the profile or not
+
 Once initialized, the sensor performs a host agent lookup using following list of addresses (in order of priority):
 
 1. The value of `INSTANA_AGENT_HOST` env variable
@@ -42,6 +78,14 @@ Once a host agent found listening on port `42699` (or the port specified in `INS
 
 To use Instana Go sensor for monitoring a service running on AWS Fargate make sure that you have `INSTANA_ENDPOINT_URL` and `INSTANA_AGENT_KEY` env variables set in your task definition. Note that the `INSTANA_AGENT_HOST` and `INSTANA_AGENT_PORT` env variables will be ignored in this case. Please refer to [Instana documentation](https://www.instana.com/docs/ecosystem/aws-fargate/#configure-your-task-definition) for detailed explanation on how to do this.
 
+### Using Instana to gather process metrics only
+
+To use sensor without tracing ability, import the `instana` package and add the following line at the beginning of your `main()` function:
+
+```go
+instana.InitSensor(opt)
+```
+
 ## Common Operations
 
 The Instana Go sensor offers a set of quick features to support tracing of the most common operations like handling HTTP requests and executing HTTP requests.
@@ -54,7 +98,7 @@ var sensor = instana.NewSensor("my-service")
 
 A full example can be found under the examples folder in [example/webserver/instana/http.go](./example/webserver/instana/http.go).
 
-### Log Output
+### Setting the sensor log output
 
 The Go sensor uses a leveled logger to log internal errors and diagnostic information. The default `logger.Logger` uses `log.Logger`
 configured with `log.Lstdflags` as a backend and writes messages to `os.Stderr`. By default this logger only prints out the `ERROR` level
@@ -133,7 +177,9 @@ func MyFunc(ctx context.Context) {
 }
 ```
 
-### HTTP Server Handlers
+### HTTP servers and clients
+
+#### Instrumenting HTTP request handling
 
 With support to wrap a `http.HandlerFunc`, Instana quickly adds the possibility to trace requests and collect child spans, executed in the context of the request span.
 
@@ -165,7 +211,7 @@ h := http.FileServer(http.Dir("./"))
 http.HandleFunc("/files", instana.TracingHandlerFunc(sensor, "index", h.ServeHTTP))
 ```
 
-### Executing HTTP Requests
+#### Instrumenting HTTP request execution
 
 Requesting data or information from other, often external systems, is commonly implemented through HTTP requests. To make sure traces contain all spans, especially over all the different systems, certain span information have to be injected into the HTTP request headers before sending it out. Instana's Go sensor provides support to automate this process as much as possible.
 
@@ -183,30 +229,56 @@ resp, err := client.Do(req.WithContext(ctx))
 
 The provided `parentSpan` is the incoming request from the request handler (see above) and provides the necessary tracing and span information to create a child span and inject it into the request.
 
-### GRPC servers and clients
+### Database Calls
 
-[`github.com/instana/go-sensor/instrumentation/instagrpc`](./instrumentation/instagrpc) provides both unary and stream interceptors to instrument GRPC servers and clients that use `google.golang.org/grpc`.
+The Go sensor provides `instana.InstrumentSQLDriver()` and `instana.WrapSQLConnector()` (since Go v1.10+) to instrument SQL database calls made with `database/sql`. The tracer will then automatically capture the `Query` and `Exec` calls, gather information about the query, such as statement, execution time, etc. and forward them to be displayed as a part of the trace.
 
-### Kafka producers and consumers
+#### Instrumenting `sql.Open()`
 
-[`github.com/instana/go-sensor/instrumentation/instasarama`](./instrumentation/instasarama) provides both unary and stream interceptors to instrument Kafka producers and consumers built on top of `github.com/Shopify/sarama`.
+To instrument a database driver, register it using `instana.InstrumentSQLDriver()` first and replace the call to `sql.Open()` with `instana.SQLOpen()`. Here is an example on how to do this for `github.com/lib/pq` PostgreSQL driver:
+
+```go
+// Create a new instana.Sensor instance
+sensor := instana.NewSensor("my-daatabase-app")
+
+// Instrument the driver
+instana.InstrumentSQLDriver(sensor, "postgres", &pq.Driver{})
+
+// Create an instance of *sql.DB to use for database queries
+db, err := instana.SQLOpen("postgres", "postgres://...")
+```
+
+You can find the complete example in the [Examples section][godoc] of package documentation on [pkg.go.dev][pkg.go.dev].
+
+The instrumented driver is registered with the name `<original_name>_with_instana`, e.g. in the example above the name would be `postgres_with_instana`.
 
-## Sensor
+#### Instrumenting `sql.OpenDB()`
 
-To use sensor only without tracing ability, import the `instana` package and run
+Starting from Go v1.10 `database/sql` provides a new way to initialize `*sql.DB` that does not require the use of global driver registry. If the database driver library provides a type that satisfies the `database/sql/driver.Connector` interface, it can be used to create a database connection.
+
+To instrument a `driver.Connector` instance, wrap it using `instana.WrapSQLConnector()`. Here is an example on how this can be done for `github.com/go-sql-driver/mysql/` MySQL driver:
 
 ```go
-instana.InitSensor(opt)
+// Create a new instana.Sensor instance
+sensor := instana.NewSensor("my-daatabase-app")
+
+// Initialize a new connector
+connector, err := mysql.NewConnector(cfg)
+// ...
+
+// Wrap the connector before passing it to sql.OpenDB()
+db, err := sql.OpenDB(instana.WrapSQLConnector(sensor, "mysql://...", connector))
 ```
 
-in your main function. The init function takes an `Options` object with the following optional fields:
+You can find the complete example in the [Examples section][godoc] of package documentation on [pkg.go.dev][pkg.go.dev].
 
-* **Service** - global service name that will be used to identify the program in the Instana backend
-* **AgentHost**, **AgentPort** - default to `localhost:42699`, set the coordinates of the Instana proxy agent
-* **LogLevel** - one of `Error`, `Warn`, `Info` or `Debug`
-* **EnableAutoProfile** - enables automatic continuous process profiling when `true`
+### GRPC servers and clients
 
-Once initialized, the sensor will try to connect to the given Instana agent and in case of connection success will send metrics and snapshot information through the agent to the backend.
+[`github.com/instana/go-sensor/instrumentation/instagrpc`](./instrumentation/instagrpc) provides both unary and stream interceptors to instrument GRPC servers and clients that use `google.golang.org/grpc`.
+
+### Kafka producers and consumers
+
+[`github.com/instana/go-sensor/instrumentation/instasarama`](./instrumentation/instasarama) provides both unary and stream interceptors to instrument Kafka producers and consumers built on top of `github.com/Shopify/sarama`.
 
 ## OpenTracing
 
@@ -276,5 +348,6 @@ Following examples are included in the `example` folder:
 For more examples please consult the [godoc][godoc].
 
 [godoc]: https://pkg.go.dev/github.com/instana/go-sensor/?tab=doc#pkg-examples
+[pkg.go.dev]: https://pkg.go.dev/github.com/instana/go-sensor
 [instana.TracingHandlerFunc]: https://pkg.go.dev/github.com/instana/go-sensor/?tab=doc#TracingHandlerFunc
 [instana.RoundTripper]: https://pkg.go.dev/github.com/instana/go-sensor/?tab=doc#RoundTripper

example_instrumentation_go1.10_test.go

@@ -0,0 +1,37 @@
+// +build go1.10
+
+package instana_test
+
+import (
+	"context"
+	"database/sql"
+
+	instana "github.com/instana/go-sensor"
+)
+
+// This example demonstrates how to instrument an *sql.DB instance created with sql.OpenDB() and driver.Connector
+func ExampleWrapSQLConnector() {
+	// Here we initialize a new instance of instana.Sensor, however it is STRONGLY recommended
+	// to use a single instance throughout your application
+	sensor := instana.NewSensor("my-http-client")
+
+	// Instrument the connector. Normally this would be a type provided by the driver library.
+	// Here we use a test mock to avoid bringing external dependencies.
+	//
+	// Note that instana.WrapSQLConnector() requires the connection string to send it later
+	// along with database spans.
+	connector := instana.WrapSQLConnector(sensor, "driver connection string", &sqlConnector{})
+
+	// Use wrapped connector to initialize the database client. Note that
+	db := sql.OpenDB(connector)
+
+	// Inject parent span into the context
+	span := sensor.Tracer().StartSpan("entry")
+	ctx := instana.ContextWithSpan(context.Background(), span)
+
+	// Query the database, passing the context containing the active span
+	db.QueryContext(ctx, "SELECT * FROM users;")
+
+	// SQL queries that are not expected to return a result set are also supported
+	db.ExecContext(ctx, "UPDATE users SET last_seen_at = NOW();")
+}

example_instrumentation_test.go

@@ -32,10 +32,34 @@ func ExampleRoundTripper() {
 		Transport: instana.RoundTripper(sensor, nil),
 	}
 
-	// Inject parent span into therequest context
+	// Inject parent span into the request context
 	ctx := instana.ContextWithSpan(context.Background(), span)
 	req, _ := http.NewRequest("GET", "https://www.instana.com", nil)
 
 	// Execute request as usual
 	client.Do(req.WithContext(ctx))
 }
+
+// This example demonstrates how to instrument an *sql.DB instance created with sql.Open()
+func ExampleSQLOpen() {
+	// Here we initialize a new instance of instana.Sensor, however it is STRONGLY recommended
+	// to use a single instance throughout your application
+	sensor := instana.NewSensor("my-http-client")
+
+	// Instrument the driver. Normally this would be a type provided by the driver library, e.g.
+	// pq.Driver{} or mysql.Driver{}, but here we use a test mock to avoid bringing external dependencies
+	instana.InstrumentSQLDriver(sensor, "your_db_driver", sqlDriver{})
+
+	// Replace sql.Open() with instana.SQLOpen()
+	db, _ := instana.SQLOpen("your_db_driver", "driver connection string")
+
+	// Inject parent span into the context
+	span := sensor.Tracer().StartSpan("entry")
+	ctx := instana.ContextWithSpan(context.Background(), span)
+
+	// Query the database, passing the context containing the active span
+	db.QueryContext(ctx, "SELECT * FROM users;")
+
+	// SQL queries that are not expected to return a result set are also supported
+	db.ExecContext(ctx, "UPDATE users SET last_seen_at = NOW();")
+}