» read the full case study

All AI companies face the dilemma of how to guard against overconfidence and hallucinations. Sometimes a LLM can provide an extremely accurate answer with high confidence, while other times it’s a tenuous approximation. So should your AI disclose its confidence level? Is displaying uncertainty and nuance a good UX decision?

Here, I conducted a randomized experiment in a national sample that tested the effect of adding certainty indicators (1-10 ratings of confidence) onto AI responses to factual questions. This builds off of my Ph.D. dissertation about how people respond to uncertainty when information-seeking.

» read the full case study

Our client was launching a AI-powered tool to improve the contextual fit of digital ads (i.e., to make sure they show up in relevant, appropriate contexts), but needed feedback from advertising execs to understand their needs and pain points. I designed a multi-stage research plan in which I first conducted dozens of interviews with these Fortune 500 executives to discover how to tailor this tool for them and their teams. After distilling the main themes, I used these insights to lead a comprehensive overhaul of the design of their brand profile calibration process (a self-guided system where advertisers describe brand values and how they want their ads to show up).

By working closely with the product team, sales, and C-suite, I was able to create an enduring rebuild that continues to define their product strategy.

» read the full case study

The typical election forecast uses a mix of polling and “fundamentals” (data about the current political landscape). But I decided this could be even more accurate by accounting for something that nobody else was looking at: information-seeking trends by internet users. The idea was that web traffic on a candidate’s Wikipedia page was a strong behavioral indicator of public interest that wasn’t being well-captured by the polling data (which only shows self-reported intentions, not actual action). By adding in this behavioral data, I was able to significantly improve upon the best election forecasts for dozens of elections. And the model had the largest advantage at times furthest out from election day when traditional polling models are at their weakest.

Overall, this showed how scraping behavioral data about online information-seeking can give us key insights into how people will take action in the future (voting, purchasing, etc.).

In this project my team crafted a multimedia comms campaign to resonate with a target audience’s values and motivations. We pre-tested videos (survey experiments, dial testing, etc.) and then after final editing and refinement they were launched as digital ads in 2 congressional districts in Missouri and Georgia. We randomly assigned half the zip codes to a treatment condition (got ads) and half to a control condition (no ads), and then used a representative survey to assess differences between the treatment and control.

After running for 30 days and gathering millions of views, this field experiment showed a large persuasive effect of the campaign specifically in our target demographic.

In this project, I lead a comprehensive custom study of consumer behavior and price tolerance for a large energy company so they could calibrate their communications and financial strategies as they expanded into a new market.

Through a representative survey of energy consumers in their target market, our findings showed exactly how much consumers were willing to pay, and how different types of energy (onshore wind, offshore wind, solar, etc.) would impact public support for new builds. These data points helped us build communication campaign strategies that were then deployed during the rollout after the successful bid.

»read the full case study

Here I led an international market research initiative and strategic comms consulting for National Geographic. The results showed how to tailor sustainability comms to the unique attitudes and behaviors of each of 12 countries (USA, Mexico, Brazil, UAE, Kenya, Indonesisa, etc).

Using predictive modeling I was able to uncover the psychological mechanisms that are most strongly linked to key behavior outcomes, and also how this underlying structure of influences differs widely across international markets. These insights fed into the comms strategies that National Geographic would then deploy in their international outreach efforts.