Skip to content
This repository has been archived by the owner on Jun 14, 2024. It is now read-only.

Home

Jump to bottom
Osvaldo Pinali Doederlein edited this page Aug 2, 2014 · 18 revisions

OpenRTB Core

This quick tutorial guides you into programming with the OpenRTB Core library.

The OpenRTB specification

This library implements the protocol / data model from the latest OpenRTB Specification, so reading that should be your first step; it's also a good introduction to concepts of real-time bidding.

OpenRTB is only specified in terms of JSON messages, so an implementation has to provide bindings for specific programming languages so you can create and manipulate its message objects in a convenient way, i.e. not needing to deal with the raw JSON representation. We do that by first translating the specification to a Protocol Buffer descriptor, which can be used to generate model classes. This library will only build a Java model out-of-the-box, but you can easily use the same Protobuf descriptor to generate the model code for many other languages. (Other features though, are only available for Java or other JVM-based languages.)

The RTB Model

Protobuf is often used to create very efficient, evolvable binary protocols, but in this case we're only interested in its ability to produce a very rich model. Here's a snippet that uses the Java model:

BidRequest request = BidRequest.newBuilder()
    .setId("1")
    .addImp(Impression.newBuilder()
        .setId("1")
        .setBidfloor(1.5)
        .setVideo(Video.newBuilder()
            .setLinearity(Linearity.LINEAR)
            .setMinduration(100)
            .setMaxduration(200)
            .setProtocol(Protocol.VAST_3_0)
            .setW(640)
            .setH(480)))
    .build();

Some qualities of this generated model:

  • Fully static-typed (including extensions, commented later)
  • All message types (e.g. BidRequest) are immutable, so they are paired by fluent Builders
  • All message types, getters and setters carry documentation extracted from the OpenRTB specification
  • Present/absent state for all properties, including those of primitive type, with hasXxx() methods
  • Other conveniences, check the Protobuf API Reference

Extensions

OpenRTB is a best-effort at a common RTB protocol, but no SSP or DSP platform uses only "pure OpenRTB". RTB protocols are varied and fast-changing, so everyone has at least a few fields or objects that are not yet available in the OpenRTB spec. That's why OpenRTB allows platforms to add extensions, in the form of arbitrary objects/fields inside "ext" properties of most objects, like this example:

"geo": { "country": "USA", "city": "New New York", "zip": "102879",
         "ext": { "sa-planet": "Mars" }
}

The snippet above is part of a BidRequest message, in its JSON representation. Unfortunately for the SpaceAds SSP, OpenRTB's Geo object doesn't support a planet field, necessary for geotargeting users or devices from our Martian colonies. This hypothetical SSP from the future would need an extension for this information, at least until a new OpenRTB release supports it. (Field name prefixes are common practice, and highly recommended, so messages might combine extension fields from multiple parties with less risk of conflict; that's why the example uses sa-planet, "sa-" standing for SpaceAds.)

The library supports OpenRTB extension conveniently and safely, via Protobuf Extensions:

BidRequest request = BidRequest.newBuilder()
    // ... add Impression, etc.
    .addDevice(Device.newBuilder()
        .setModel("Nexus 85")
        // ... other standard Device fields
        .setGeo(Geo.newBuilder()
            .setCountry("USA")
            .setCity("New New York")
            .zetZip("102879")
            .setExtension(SpaceAdsExt.geo, SpaceAdsExt.Geo.newBuilder()
                .setPlanet("Mars").build())))
        .build();

Here's how extensions work: the SpaceAds SSP provides a separate extension library, also Protobuf-generated, which only defines the extensions it contributes to specific OpenRTB extension points. In the generated Java class SpaceAdsExt, Each of these extensions will have a key like geo and a message type like Geo. In the core OpenRTB object like BidRequest.Geo, which doesn't know anything about SpaceAds, you have to use a generic method setExtension() to add extension data. Notice however that this setter is static-typed: it will only accept matching (key, value) pairs, and both the key and value have to be declared as compatible with the BidRequest.Geo.ext extension point. This mechanism combines modularity (allowing extensions to be provided by third-party libraries without any change to the core OpenRTB library) with type-safety (any incorrect use of extensions will not compile).

JSON Support

OpenRTB messages will typically be received and sent in their JSON format. Protobuf's generated code doesn't support JSON serialization; there are third-party libraries that do that, but they don't help us because support for extensions creates serious problems for any automatic JSON serializer (the content of the ext fields is not typed in the container object, and it may be a combination of fields contributed by several independent extensions). The library solves this problem by providing a custom JSON serializer, based on the popular Jackson library. (Your programs only need the Jackson dependency if they use the JSON serializer component.) Sample usage:

OpenRtbJsonFactory openrtbJson = OpenRtbJsonFactory.create()
    .setJsonFactory(new JsonFactory())
    .register(new SpaceAdsGeoReader(),
              "BidRequest.device.geo", "BidRequest.user.geo")
    .register(new SpaceAdsGeoWriter(), SpaceAdsExt.Geo.class,
              "BidRequest.device.geo", "BidRequest.user.geo")
    .create();
// How to serialize
ByteString jsonRequest = openrtbJson.newWriter().writeBidRequest(request);
// How to desserialize
BidResponse response = openrtbJson.newReader().readBidResponse(jsonResponse);

You start by creating an OpenRtbJsonFactory, which can be optionally configured with a JsonFactory and with serialization helpers for extensions provided by third parties. In the example, the SpaceAds SSP provides a library that contains both the protobuf-generated SpaceAdsExt (the model for its extensions) and classes like SpaceAdsGeoReader and SpaceAdsGeoWriter for their serialization. A real-world scenario may need a large number of these register() calls, but the third-party library will typically provide an utility method that returns a fully-configured OpenRtbJsonFactory.

Notice that the serializer's writeBidRequest() / writeBidResponse() and readBidRequest() / readBidResponse() all use the ByteString class (from Protobuf's library) as their preferred representation for JSON text. This is because that class is more efficient for this task than any standard alternative (like StringBuilder or ByteArrayOutputStream). You can convert ByteString to/from String, and there are also overloads of the read/write methods that can take a standard InputStream or OutputStream (these may be more efficient if you're serializing directly from/to the content of a HttpServletRequest / HttpServletResponse, for example).

Macro Processing

A common usage scenario for the RTB model is that of a bidder or DSP that receives a BidRequest and replies a BidResponse. The response is simple in number of objects/fields, but it may contain fields which content is complex (notably Bid.adm = ad markup) or derived from information from the corresponding request or even from other response fields. This problem is well-fit for macros, so this library offers a SnippetProcessor API that performs post-processing of a BidResponse.Builder (notice the Builder here, since we need the response's mutable representation for postprocessing). An example will make this clear:

BidResponse.Builder responseBuilder = BidResponse.newBuilder()
    .addSeatbid(SeatBid.newBuilder().addBid(Bid.newBuilder()
        .setAdid("ad-1234567")
        .setImpid(imp.getId())
        .setPrice(1.2)
        .setAdm("<a href='%{http://click.com}&adid=${AUCTION_AD_ID}}%'>"
                + "<img src='http://cdn.com/my-ad.jpg'/>"
                + "<img src='http://mybidder.com/impression-pixel'>"
              + "</a>")))));
Snippetprocessor proc = new OpenRtbSnippetProcessor();
proc.process(request, response);
BidResponse response = responseBuilder.build();

We're creating a BidResponse corresponding to a specific BidRequest request, containing a Bid corresponding to a specific Impression imp from that request. The HTML markup for this bid contains a parameter adid in the click-through URL, which value will be the same as the Bid.adid. You could just put the same ad-1234567 inside the markup, but these values are typically not constants; they are often retrieved from some campaign database. Now you don't want to create a complex HTML string with a spaghetti of concatenations, do you? (This example is simple, but real ads can have bigger HTML snippets with several dynamic parameters.) The solution is using the ${AUCTION_AD_ID} macro, which will be expanded to the content of Bid.adid by the process() call.

Notice that you actually use an OpenRtbSnippetProcessor, which supports macros defined by the OpenRTB specification. Third-party libraries can provide further subclasses to support extended macros. There's only one non-OpenRTB feature here, implemented in SnippetProcessor because it's too useful: the syntax %{...}% will URL-escape its contents, which is often necessary in ad markup, see the first URL in the example above.

Other Utilities

This library also offers some small conveniences in the util package:

  • OpenRtbValidator: inspects a pair of request/response and finds many potential errors in the response.
  • OpenRtbUtils: methods to lookup, filter and update elements of requests and responses.

Finally, the model package contains only an abstract interface that can be used by third-party libraries that provide converters between the OpenRTB model and some proprietary RTB model.

Next Steps

The openrtb library is a central building block for RTB systems, but building any of that (a bidder, DSP, or other OpenRTB-related application) is typically bigger puzzle that benefits from additional pieces.

  • The openrtb-doubleclick library is a companion open source project that provides extensive support for the DoubleClick Ad Exchange platform. This includes mapping between DoubleClick's protocol and OpenRTB (so you can write OpenRTB-portable code for this exchange), and other DoubleClick-specific conveniences such as encryption / decryption.
  • Open Bidder is a complete toolkit for building scalable, full-featured RTB bidders, optimized for the Google Cloud Platform and DoubleClick but also extensible for other platforms and exchanges.
Clone this wiki locally