A Reinforcement Learning Approach in Multi-Phase Second-Price Auction Design

Understand

We study reserve price optimization in multi-phase second price auctions, where the seller's prior actions affect the bidders' later valuations through a Markov Decision Process (MDP).

• Compared to the bandit setting in existing works, the setting in ours involves three challenges.
• First, from the seller's perspective, we need to efficiently explore the environment in the presence of potentially untruthful bidders who aim to manipulate the seller's policy.
• Second, we want to minimize the seller's revenue regret when the market noise distribution is unknown.