Functional Optimization Reinforcement Learning for Real-Time Bidding

This is the research project of optimal real-time bidding in computational advertising field.

Abstract

Real-time bidding is the new paradigm of programmatic advertising. An advertiser wants to make the intelligent choice of utilizing a Demand-Side Platform to improve the performance of their ad campaigns. Existing approaches are struggling to provide a satisfactory solution for bidding optimization due to stochastic bidding behavior. In this paper, we proposed a multi-agent reinforcement learning architecture for RTB with functional optimization. We designed four agents bidding environment: three Lagrange-multiplier based functional optimization agents and one baseline agent (without any atttribute of functional optimization) First, numerous attributes have been assigned to each agent, including biased or unbiased win probability, Lagrange multiplier, and click-through rate. In order to evaluate the proposed RTB strategy’s performance, we demonstrate the results on ten sequential simulated auction campaigns. The results show that agents with functional actions and rewards had the most significant average winning rate and winning surplus, given biased and unbiased winning information respectively. The experimental evaluations show that our approach significantly improve the campaign’s efficacy and profitability.

Index Terms—Real-Time Bidding Optimization, Multi-Agents Deep Reinforcement Learning, Functional Optimization.

Question

Any question please contact [email protected] and [email protected]