Pioneering tech economist Susan Athey joins federal antitrust team

Susan Athey, whose pioneering work in economics has crisscrossed academia and industry, has joined the U.S. Department of Justice as chief economist of the antitrust division.

The new role for Athey, the Economics Professor of Technology at Stanford Graduate School of Business (GSB) and a senior fellow at the Stanford Institute for Economic Policy Research (SIEPR), is the latest in a wide-ranging career that has garnered both accolades and respect.

It’s a great time for Athey to try her hand at government work, says Jonathan Levin, the Philip H. Knight Professor and Dean at the GSB. “At a moment when technology is ascendant and promoting competition is essential, I can’t think of anyone I’d rather have at the DOJ than Susan.”

Athey is known for making technical advances in economic theory and industrial organization, conducting original research that combines machine learning with econometric modeling, and pioneering the new field of tech economics. She is frequently tapped for her insights to inform policymakers and organizations, and she has helped catalyze efforts to apply economic research and technological advances to make an impact. At Stanford, she helped found and foster the Stanford Institute for Human-Centered Artificial Intelligence and launched the Golub Capital Social Impact Lab. Before joining Stanford, she served as chief economist at Microsoft.

“Susan is a force of nature,” Levin says. “She moves from machine learning to business strategy to technology policy to social impact, producing deep ideas at every turn.”

Now, in her new job as the nation’s top antitrust economist, Athey will be addressing the challenges of the digital economy from the top down — as opposed to the bottoms-up approach of bringing change through the work, for instance, of the Golub Capital Social Impact Lab.

“Government laws and policies affect everything from how competition works to what mergers go through, to what investments people make,” Athey says.

Athey, a founder and associate director at Stanford HAI, says she hopes to continue many of the institute’s efforts to help governments adapt to an era of rapidly changing technology, particularly around the use of data in industry and in government.

“Because technology such as artificial intelligence moves so quickly, it’s hard for the government to keep up,” she says. “We have to figure out how all branches of government are going to be prepared to guide us through a different age.”

Athey's multidimensional work

Introducing AI to economics

Athey’s propensity for immersing herself in diverse fields of study goes back to her undergraduate days at Duke University, where she graduated in 1991 with a triple-major in economics, mathematics, and computer science.

She went on to become a professor of economics and business at MIT, Harvard University, and then at Stanford starting in 2013. While at Harvard in 2007, Athey won the John Bates Clark Medal — the first woman to win the prestigious award — for her contributions to multiple subfields in economics, including industrial organization, microeconomic theory, and econometrics.

But it was during a leave from academia to serve as Microsoft’s chief economist from 2008 to 2013 that Athey made a surprising connection between her passion for economics and the tools of AI and machine learning.

Athey already knew that digitization and tech platforms were going to play a significant role in the economy, and that search engines were poised to have outsized importance. She also knew that the research community was just beginning to tackle questions about how to design digital markets and what healthy competition looked like in those markets, and she was excited to help develop that research.

At Microsoft, Athey discovered something she wasn’t expecting: the potential for machine learning to address economic problems. The creators of the Bing search engine were conducting experiments in a manner that economists only dreamed about. They were simultaneously running thousands of randomized A/B tests — asking large numbers of “what if” questions to better understand such things as which search results should rise to the top and how to run auctions for setting advertising prices on a search page. By comparison, she says, economists would typically run one experiment in a year.

“Microsoft was using an artificial intelligence system composed of hundreds of algorithms that were all working together to make a search results page,” she says. “That was something new.”

Machine learning and causal effects

In the field of economics until then, data mining and machine learning had been pejorative terms for a less advanced form of statistics. “They were seen as a mechanical exercise to separate cats from dogs,” she says. But at Microsoft, Athey saw an opportunity to combine the computational advances from predictive machine learning with statistical theory so that researchers could better understand causal effects not only in business applications like the search engine, but also in social science and economics. It was an epiphany that launched her in a new research direction and helped define her as one of the early tech economists.

Coming out of her experience at Microsoft, Athey realized that the insights from predictive algorithms could be harnessed in new ways by combining them with recent developments in econometrics and statistics. For example, machine learning algorithms could be tailored to answer cause-and-effect questions in economics, such as, what will happen if we change the minimum wage? Expand immigration policy? Raise prices? Allow two firms to merge?

“Predictive machine learning can’t solve these questions alone, but it can help,” she says.

For example, Athey has used machine learning to look at the impact on consumers of personalized pricing, a form of price discrimination that involves charging different prices to consumers according to their willingness to pay. Traditional economics methods would give aggregate solutions to that problem, she says. They would study perhaps one product category at a time, considering demand for, say, different brands of yogurt or towels. By applying machine learning methods to consumers’ historical purchase data, Athey’s research team can estimate personalized consumer preferences across multiple products at the same time.

In turn, building these predictive models of consumer choice allows researchers to ask even bigger questions about such things as what happens to prices if you apply a tariff, or if generics arrive on the market. “As an input to answering these questions, we want to understand how consumers make choices,” Athey says. And machine learning delivers that input in a way that allows researchers to do this work more efficiently, at a larger scale, and in a more personalized way. “If you’re assuming everybody’s the same, that gives different answers than if you assume that people have different preferences,” she says.

A tech economics pioneer

Athey’s tenure at Microsoft defined her as one of the first people to be considered a “tech economist.” It’s a field she has since helped establish as an independent discipline by convening early conferences in the field and mentoring numerous students along that career path.

“Now tech economists hold an annual conference that draws about 800 participants,” she says. “And we have a specialized job market because being a tech economist is a distinct profession that people can pursue.”

Athey has also written about what it means to be a tech economist. “It’s partly a career, but it’s also a combination of different fields of study,” she says. Tech economists study the impact of digitization on the economy, which entails thinking about market design, privacy, data security, fairness, competition policy, and more, she says. “They also help create and analyze business models and competitive strategy, and they connect the models to data to guide decisions.”

During her deep dive into machine learning and AI at Microsoft, Athey also witnessed firsthand the challenges created by these technologies — ethical and legal issues, First Amendment problems, fairness and bias, privacy and copyright, and the prevalence of unforeseen consequences as people manipulated or gamed the system in response to market shifts or new rules.

Because of these observations, Athey developed a desire to impact the ways that machine learning and AI would play out in the world. When she returned to academia full time, her initial steps in that direction included helping to plan the launch of HAI. “Stanford HAI was really created to address these problems,” she says. “We want to make AI beneficial for humans, and we want to avoid all of these unintended consequences.”

Athey also wanted to translate the successful uses of AI from the for-profit sector into the social impact sector, which led her to launch the Golub Capital Social Impact Lab. “We’re bringing the tech toolkit to social impact applications,” she says. The lab’s work has included case studies of digital education technology to enhance students’ learning; the development and evaluation of digital tablet applications that guide nurses through counseling patients; and the creation of methods to prioritize candidates for clinical trials of COVID-19 medications.

Athey will remain a professor at Stanford GSB and a senior fellow at SIEPR. At Stanford HAI, she will step down from her associate director role but continue to be an affiliated faculty member.

The original version of this article was published by Stanford HAI.