Handling Five Billion Ad Requests Daily at Low Latency Using Amazon EMR with Siprocal

by | Oct 3, 2023 | Connected TV, Technology | 0 comments

Siprocal migrated its high-throughput, low-latency AdTech workloads to AWS, creating an infrastructure that scaled to handle five billion requests daily while maintaining 99.999 percent availability.


99.999%5 billion30 msThousandsZero
service availability transactions processed per day at peakaverage latencyof engineer hours saveddowntime and no data loss in migration

Overview

Siprocal needed to overhaul its connected television (CTV) advertising infrastructure, which ran using on-premises hardware that imposed limits on critical infrastructure that handled billions of API requests per day. Engineers were spending excessive time and resources on support for the company’s supply-side platform, an automated advertising technology that publishers use to sell space for digital advertisements.

Siprocal set a challenging goal of migrating to the cloud with no downtime or data loss. The company selected Amazon Web Services (AWS) and enlisted support from TO THE NEW, an AWS Partner that provides digital transformation and product engineering services to enterprises. Together, the teams redesigned Siprocal’s infrastructure and completed a successful migration in 8 months. Using AWS, Siprocal now operates with five nines of availability, scales automatically to accommodate surges in traffic, and has freed engineers to add new technology and work on feature enhancements.

Siprocal Team photo

Opportunity | Using Amazon EMR to Build a Dynamic Infrastructure for Siprocal

Siprocal offers gaming, mobile distribution, and engagement solutions for CTV, a catchall term for content that streams through digital devices. The company has capabilities as a supply-side platform and as an ad server. In Siprocal’s model, advertisements appear within streamed content as the culmination of a complex process called programmatic advertising: advertisers bid in real-time competition for publishers’ available ad spaces within a consumer’s streaming content. A millisecond can mean the difference between winning and losing a bid.  

Siprocal had been running its demanding CTV workloads through open-source software, using an Apache Hadoop cluster in a traditional data center. Some engineers were spending
their whole 40-hour workweek on maintenance. The company sometimes had to turn off systems because engineers couldn’t track down necessary parts, and in some cases, repairs were cost prohibitive. “We started to spend as much time on maintenance and upkeep to keep things running as we did on enhancements and improvements,” says Mike Ryan, DevOps engineer at Siprocal. “We had to do something. It made a whole lot of sense to migrate to the cloud and use managed services.”

Siprocal had lofty goals for its migration. There could be no service interruption or downtime for its CTV offerings and no loss of data. “We record everything for every single transaction in our Apache Hadoop cluster,” says Fan Wu, Siprocal’s chief technology officer. “The data is very important for our business to gain insights.” Impressed by the quality of support offered by the AWS account team and by TO THE NEW, Siprocal decided to migrate the Apache Hadoop cluster to Amazon EMR, which runs big data applications and petabyte-scale
data analytics faster and at less than half the cost of on-premises solutions. “We knew that we could dynamically scale our Amazon EMR cluster up and down, which would help us optimize the process,” Wu says.  

“Our efficiency, productivity, and stability on AWS are not even comparable to how they were in the data center.”

Fan Wu, CTO for CTV & Streaming at Siprocal

Solution | Migrating with Zero Downtime to a Modernized Infrastructure That Handles Five Billion Requests Daily

In September 2021, Siprocal’s relatively small 10-person engineering team began the migration alongside TO THE NEW, which has been an AWS Partner since 2013. The team created a parallel setup on AWS and replicated the components of the Apache Hadoop cluster, which consisted of 40 nodes running 32 processors each. Engineers slowly migrated 1 PB of data that had been stored on disk within the Apache Hadoop cluster to Amazon Simple Storage Service (Amazon S3), an object storage service that offers industry-leading scalability, data availability, security, and performance. Siprocal completed the migration with no downtime or loss of data. “We went from a very old infrastructure to AWS in 8 months,” says Wu. “This is a big achievement.”

Siprocal used the migration as an opportunity to improve the efficiency of its data pipeline. It now directs incoming ad requests using Application Load Balancer, which provides advanced request routing targeted at delivery of modern application architectures. For example, Siprocal can route traffic to a user interface, where clients and internal teams manage ad campaigns. Additionally, an event service receives impressions from publishers and connects them to various databases for cached data, metadata, an event message bus, and the ad-pacing service that manages the rate at which ad campaigns roll out.

Business-critical applications run in Amazon Elastic Container Service (Amazon ECS), a fully managed container orchestration service that simplifies deployment, management, and scaling of containerized applications. Containers run using AWS Fargate, a serverless, pay-as-you-go compute engine that lets organizations focus on building applications without managing servers. “Previously, we didn’t take in more business because we just couldn’t support it,” says Steven Cothren, director of software engineering at Siprocal. “Now we can scale to meet the business demand.” Before the migration Siprocal could handle 1.5 billion ad requests per day, but now it can process up to 5 billion ad requests per day during peak times, and the average response time is 300 ms. “The speed at which we respond to requests directly affects our revenue,” Cothren says. “A component of determining which ad gets displayed is who responds quickest. Our performance using AWS helps with that. It also avoids outages. If we have an outage, we make no money.”

Freed from storage limitations for logs, Siprocal improves observability using Amazon CloudWatch, which collects and visualizes near-real-time logs, metrics, and event data in automated dashboards to streamline infrastructure and application maintenance. Engineers see the number of requests, application errors, response times, and other metrics. “Even if there’s not a problem yet, we can see if we’re trending toward a problem,” says Cothren. “We have a true view into all our applications.”

Architecture Diagram

Siprocal_architectural diagram

Outcome | Migrating Additional Critical Workloads to AWS

Siprocal hopes to use the AdTech migration as a model, working alongside AWS to migrate its on-premises gaming workloads to the cloud. “Under pressure, we did a great job finishing our migration within the timeline with no business impact,” says Wu. “This has been a great experience. Our efficiency, productivity, and stability on AWS are not even comparable to how they were in the data center.”

Read it at AWS

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

Related Articles

Talladega Bytes 

Talladega Bytes 

How do you illustrate Siprocal’s innovative hyper-targeting with AI? Think NASCAR …  Pro tip for anyone who buys or sells advertisements online:  “Watch a NASCAR race,” says AI (artificial intelligence) guru Tracy Malingo.  Targeting consumers on platforms like...

Record News fecha acordo com a Siprocal para uso de tecnologia em CTV

Record News fecha acordo com a Siprocal para uso de tecnologia em CTV

Segundo dados da Comscore, 66 milhões de brasileiros consomem conteúdo em TV Conectada O canal de notícias Record News fechou um acordo com a Siprocal, plataforma de tecnologia para mídia que ajuda publishers, operadoras e anunciantes líderes nas Américas a crescerem...