Measuring and Building Human Leadership in an AI World By Brent Orrell | Raphaël Colard AEIdeas April 09, 2025: “A new working paper from the National Bureau of Economic Research, Measuring Human Leadership Skills With AI Agents, presents evidence that artificial intelligence may soon play a central role in evaluating human soft skills—long considered too complex and subjective to measure objectively. Conducted by Ben Weidmann and David Deming et al. at the Harvard Kennedy School, the study involved 249 participants who led teams of both human and AI-agent teams through collaborative problem-solving tasks. To measure leadership, the researchers used a “hidden profile task,” where critical information was distributed unevenly among teammates and must be discovered and identified by the leader. Those who succeeded—whether leading AI agents or humans—tended to ask more questions, promote turn-taking in conversation, and use plural pronouns, like “we” and “us.” The findings are striking: leadership skills demonstrated with AI teammates closely mirror leadership effectiveness with human teams, with a strong correlation of 0.81 between the two tests. As opposed to coming from formal credentials, strong leadership springs from behavioral patterns that foster collaboration—a type of socio-emotional IQ called noncognitive skills. Since the study finds these types of skills are unrelated to race, gender and other identity categories, use of AI may afford opportunities for more people of all backgrounds to discover and develop leadership abilities. Moreover, the emerging reality that AI will increasingly take over narrow technical tasks is likely to raise the premium on social-emotional skills. A world that demands more of such skills may disproportionately benefit the socially agile relative to the technically gifted.  The study’s insights would also be useful in improving leadership hiring, training, and evaluation. Organizations could move beyond static resumes and subjective interviews toward dynamic, performance-based assessments that reflect how people lead in real time. Training programs could be better targeted, focusing on developing the communication habits and social reasoning that matter most. There is already evidence that AI is adept at training humans in leadership skills, as demonstrated by research on call center workers and a new pilot program for training substance abuse counselors…”

How Good is AI at Twisting Arms? Experiments in Debt Collection. James J. Choi, Dong Huang, Zhishu Yang & Qi Zhan. NBER Working Paper 33669. DOI 10.3386/w33669. Issue Date April 2025. How good is AI at persuading humans to perform costly actions? We study calls made to get delinquent consumer borrowers to repay. Regression discontinuity and a randomized experiment reveal that AI is substantially less effective than human callers. Replacing AI with humans six days into delinquency closes much of the gap. But borrowers initially contacted by AI have repaid 1% less of the initial late payment one year later and are more likely to miss subsequent payments than borrowers who were always called by humans. AI’s lesser ability to extract promises that feel binding may contribute to the performance gap.

AI as Strategist. Joshua S. Gans. NBER Working Paper 33650. DOI 10.3386/w33650. Issue Date April 2025. This paper examines the role of artificial intelligence as a strategist in organizational decision-making by extending van den Steen’s formal theory of strategy. A mathematical model is developed comparing AI and human strategists across different decision contexts, focusing on how each type generates confidence, achieves agreement, and implements decisions through control versus influence. The analysis presumes that AI excels in data-rich domains but faces credibility challenges in judgment-intensive contexts, creating a counterintuitive result where AI requires less formal authority precisely where it demonstrates superior analytical capabilities. The paper identifies distinct mechanisms through which strategic value is created: direct decision quality improvement and enhanced coordination. The authors propose domain-contingent approaches to AI integration, including differentiated authority systems across decision types and progressive control models that evolve as AI demonstrates effectiveness. These findings contribute to strategy theory while providing practical guidance for organizations seeking to effectively integrate AI into strategic processes, highlighting that organizations must adapt to their strategists’ capabilities as much as strategists must match their organizations.

How Much Should We Spend to Reduce A.I.’s Existential Risk? Charles I. Jones. NBER Working Paper 33602. DOI 10.3386/w33602. Issue Date March 2025. During the Covid-19 pandemic, the United States effectively “spent” about 4 percent of GDP — via reduced economic activity — to address a mortality risk of roughly 0.3 percent. Many experts believe that catastrophic risks from advanced A.I. over the next decade are at least this large, suggesting that a comparable mitigation investment could be worthwhile. Existing lives are valued by policymakers at around $10 million each in the United States. To avoid a 1% mortality risk, this value implies a willingness to pay of $100,000 per person — more than 100% of per capita GDP. If the risk is realized over the next two decades, an annual investment of 5% of GDP toward mitigating catastrophic risk could be justified, depending on the effectiveness of such investment. This back-of-the-envelope intuition is supported by the model developed here. In the model, for most of the scenarios and parameter combinations considered, spending at least 1% of GDP annually to mitigate AI risk can be justified even without placing any value on the welfare of future generations.