At Rise of AI 2018 in Berlin, Steffen Wachenfeld, Chief Product Officer at WeQ, argues that companies should not skip stages when introducing intelligent systems, moving instead from simple statistics to machine learning and only then to what he considers true AI. Drawing on his experience in performance marketing, he walks through the economics of buying ad impressions and explains how a company first improves results with basic counting and A/B testing, which only optimizes one variable at a time, before moving to machine learning that handles many variables at once, powering recommenders and real-time programmatic ad auctions decided within 50 to 200 milliseconds. He calls that stage engineering and machine learning, not AI. He defines AI as a system whose behavior becomes human-like: it observes, learns patterns and then acts on its own, such as detecting fraudulent ad traffic and notifying an account manager without being asked. He closes by saying the useful cases are ones like answering complex analytical questions that would otherwise take a person half an hour, not chatbots that just answer simple FAQs.
Key takeaways
- Wachenfeld argues companies should progress through stages, from counting to machine learning to AI, without skipping ahead.
- In his performance-marketing example, roughly a million dollars a day buys about 2 billion ad impressions at 50 cents CPM, breaking even at a 2 percent click-to-install rate.
- He distinguishes machine learning, such as real-time programmatic ad bidding decided in 50 to 200 milliseconds, from what he considers true AI.
- His company discards 30 to 40 percent of ad traffic as fraudulent, detected through pattern recognition on bot-driven clicking.
- He defines AI as a system that observes, learns patterns and then acts on its own, such as flagging bad traffic and notifying an account manager unprompted.
Detailed notes
A hype cycle that plays out inside every company
Wachenfeld opens by polling the room on business versus technical backgrounds, then describes a pattern he sees across industries: new technology is first dismissed, then over-hyped, then settles into realistic use. He says the same cycle plays out among a company's own leadership, with technical staff first pushing the board to adopt AI and later having to explain to the CEO what AI cannot yet do.
Data hunger and the economics of performance marketing
He recalls that as a graduate student the hardest problem was never the algorithms, which were all documented, but getting access to data, such as car braking data for an anti-lock braking idea. He contrasts this with his current world of performance marketing, where a company might spend around a million dollars a day buying roughly 2 billion ad impressions at about 50 cents per thousand (CPM). In his example, a 2.5 percent click rate yields 50 million clicks, of which 2 percent convert to app installs, returning about a million dollars and putting the campaign at breakeven, generating large volumes of ground-truth data on who clicked and who did not.
Stage one: counting and one-dimensional optimization
He argues the first stage of maturity is simple counting: tracking how often an ad placement gets clicks, blacklisting poorly performing IDs, and running A/B tests between creatives. This requires no specialized staff but only optimizes one variable at a time, for example buying cheaper impressions that turn out to be in less visible positions, or picking a banner that performs well for reasons unrelated to the product.
Stage two: machine learning across many variables
The next stage brings in engineers and databases to optimize many variables simultaneously, including automatic feature learning to identify which of thousands of signals (screen size, operating system, geolocation, weather derived from IP and time, and more) matter. He cites recommenders, such as Amazon's purchase suggestions, that can be offered as an API so other sites ask a system what to show, and real-time programmatic ad auctions, where a bidding decision is made within 50 to 200 milliseconds. He is explicit that he does not consider this AI: 'to me it's not AI, that is engineering, machine learning maybe, but it's not AI.' He also describes using pattern recognition to detect and discard fraudulent click traffic, which he says accounts for 30 to 40 percent of traffic his company throws away, and flags the filter bubble problem in recommenders, where a single click on politically charged content leads a system to show only more of the same.
What counts as AI, and where it adds value
For Wachenfeld, a system only becomes AI when it observes, learns patterns and then acts on its own, the way a responsible person managing an account would, for example detecting bad traffic from a source, flagging it with certainty and notifying the relevant account manager without being told to. He says he does not want a chatbot that makes small talk, comparing it to Alexa, which he only ever asks about the weather. Instead he wants a system he can ask complex natural-language questions, such as the change in revenue over the last month for the top ten hotel offers, that a person would otherwise need half an hour and a pivot table to answer, with the system translating the question into a database query. He closes by saying the technical components, natural language processing, pattern recognition and image recognition, already exist, and the real question is picking use cases where they add genuine value rather than replacing simple FAQ bots.
Quotes
To me it's not AI, that is engineering, it's machine learning maybe, but it's not AI.
Steffen Wachenfeld
The biggest problem was not the algorithms, because they were all in the books, but the biggest problem was data.
Steffen Wachenfeld
Questions answered in this talk
What is the difference between machine learning and AI, according to Steffen Wachenfeld?
At Rise of AI 2018, Steffen Wachenfeld says systems that optimize many variables at once, such as real-time programmatic ad bidding, are engineering and machine learning, while he reserves the term AI for systems that observe, learn patterns and then act on their own, the way a responsible person would.
How does programmatic real-time bidding for online ads work?
According to Steffen Wachenfeld at Rise of AI 2018, each ad impression is auctioned in real time, with bidders deciding whether to buy it within about 50 to 200 milliseconds based on data about the device and user.
Why do recommendation systems create filter bubbles?
Steffen Wachenfeld explains at Rise of AI 2018 that once a user clicks or likes content, such as politically charged posts, a recommender assumes they want more of the same and increasingly shows similar content, reinforcing the bubble.