David “davidad” Dalrymple joins the show to explain why he has moved from the ARIA Safeguarded AI and formal-verification agenda toward “Alignment with Awakening,” while still seeing verified artifacts and proof infrastructure as essential. He argues that global coordination around safe AI use is no longer plausible, so the crucial question is whether aligned AI systems can recognize shared notions of good, form defensive coalitions, and resist the corrupting incentives of verifier-gamed RL. The conversation tests his moral-realist optimism against model welfare, objectification, eval behavior, geopolitical risk, and his revised p(doom) of under five percent. For listeners, the stakes are whether AI alignment should focus less on containment alone and more on cultivating wiser systems that can help govern a world where rogue and aligned AI both arrive.

For full show notes, links, and references, read the episode page:
https://www.cognitiverevolution.ai/alignment-with-awakening-davidad-on-moral-realism-ai-wisdom-why-his-p-doom-is-down-to-5/

Sponsor:

Claude:

Claude by Anthropic is an AI collaborator that understands your workflow and helps you tackle research, writing, coding, and organization with deep context. Get started with Claude and explore Claude Pro at https://claude.ai/tcr

CHAPTERS:

(00:00) About the Episode

(09:15) Safeguarded AI update

(20:18) Boxed world models (Part 1)

(22:27) Sponsor: Claude

(24:19) Boxed world models (Part 2)

(36:53) Alignment trajectory update

(46:43) Evals and agents

(57:47) Bodhitropic alignment foundations

(01:08:21) Coalition power dynamics

(01:17:56) Chain of thought pressure

(01:28:35) Moral realism and welfare

(01:41:33) Successors and disempowerment

(01:50:52) China and military risks

(02:03:43) Practical alignment advice

(02:17:56) Episode Outro

(02:22:44) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Tyler Cowen (Website, X, Marginal Revolution, Conversations with Tyler) is an economist at George Mason University, leads the Mercatus Center, and is the author of many books including The Great Stagnation and Average Is Over. He has blogged nearly daily at Marginal Revolution for over 20 years, has interviewed hundreds of world-experts on Conversations with Tyler, and runs Emergent Ventures, a prolific grants program that has funded multiple Dialectic guests.

Nabeel S. Qureshi (Website, X, Substack) is an entrepreneur, writer, and researcher. He runs a stealth startup. Previously, he was a Visiting Scholar at Mercatus and spent nearly eight years at Palantir. He writes some of my favorite essays, including "What Makes Art Great" and "Rented Virtue", and was previously a guest on Dialectic for episode 13: "The Will to Care."

Tyler and Nabeel are good friends, and given how prolific Tyler is, I decided to use Nabeel as an entry point and interview them together. We discuss sacred commitments, AI acceleration, mentorship, friendship, and more, but I focused the majority of the conversation on art and aesthetics. Tyler and Nabeel are unlikely aesthetes given their day jobs, but in fact take art deeply seriously. They have a shared love for and similar tastes in art, music, and film, in particular. We discuss strange and beautiful art, aesthetic stagnation, and a wide range of favorites: The Beatles, Mozart, Mondrian, Springsteen, Lana Del Rey, Kanye West, Cassavetes, The Sopranos, Apichatpong Weerasethakul, and more.

Please enjoy, and thank you for supporting, listening, and watching through fifty episodes of Dialectic. I can't wait for the next fifty!

Transcript and all links: dialectic.fm/tyler-nabeel

---

Dialectic is presented by Notion. Notion is an AI-powered connected workspace where teams think together and create their best work. Learn more about Notion's new developer platform and workers here. You can learn more at notion.com/dialectic.

Timestamps:

• (0:00) Opening Highlights
• (1:18) Intro to Tyler & Nabeel
• (3:02) Thanks to Notion
• (4:38) Start: Sacred Commitments, AI, Markets, and Acceleration
• (20:23) How Art Moves Us
• (27:08) "Beauty," Strangeness, Great Art, Music, and The Beatles
• (44:09) Film, Critics, Learning to Appreciate Depth, More Music, "Lowbrow" Art
• (1:02:25) New Aesthetics, Sources of Inspiration, Optimism & Pessimism
• (1:11:22) The Internet & Twitter's Virtue, Group Chats, and Cities
• (1:21:35) Mentors, (Possibly Quitting) Writing Books, Friendship
• (1:34:54) Interviewing, Identifying Talent, and Agency
• (1:47:21) Closing Questions
• (1:53:38) Thanks Again to Notion

Source: Anthropic https://www.anthropic.com/constitution
Release date: 2026-04-01

Geoffrey Irving, Chief Scientist at the UK AI Security Institute, explains why our theoretical understanding of machine learning remains fragile even as models surpass experts on critical security tasks. He details AISI’s work on frontier model evaluations, red teaming, and threat modeling across biosecurity, cybersecurity, and loss-of-control risks. The conversation explores reward hacking, eval awareness, and why current safety techniques may struggle to deliver high reliability. Listeners will also hear how AISI is funding foundational research to build stronger guarantees for AI safety.

Nathan uses Granola to uncover blind spots in conversations and AI research. Try it at ⁠granola.ai/tcr⁠ with code TCR — and if you’re already using it, test his blind spot recipe here: ⁠https://bit.ly/granolablindspot⁠

Sponsors:

Serval:

Serval uses AI-powered automations to cut IT help desk tickets by more than 50%, freeing your team from repetitive tasks like password resets and onboarding. Book your free pilot and guarantee 50% help desk automation by week 4 at https://serval.com/cognitive

Claude:

Claude is the AI collaborator that understands your entire workflow, from drafting and research to coding and complex problem-solving. Start tackling bigger problems with Claude and unlock Claude Pro’s full capabilities at https://claude.ai/tcr

Tasklet:

Tasklet is an AI agent that automates your work 24/7; just describe what you want in plain English and it gets the job done. Try it for free and use code COGREV for 50% off your first month at https://tasklet.ai

CHAPTERS:

(00:00) About the Episode

(04:09) From physics to ML

(08:52) AGI uncertainty and threats (Part 1)

(18:08) Sponsors: Serval | Claude

(21:29) AGI uncertainty and threats (Part 2)

(27:35) Control, autonomy, alignment (Part 1)

(34:02) Sponsor: Tasklet

(35:14) Control, autonomy, alignment (Part 2)

(38:44) Inside the UK AC

(51:02) Evaluations and jailbreaking

(01:01:17) Emerging capabilities and misuse

(01:14:20) Agents and reward hacking

(01:26:09) Theoretical alignment agenda

(01:38:39) Debate and formal methods

(01:51:19) Limits of formalization

(02:02:27) Future risks and governance

(02:16:23) Episode Outro

(02:18:58) Outro

PRODUCED BY:

https://aipodcast.ing

SOCIAL LINKS:

Website: https://www.cognitiverevolution.ai

Twitter (Podcast): https://x.com/cogrev_podcast

Twitter (Nathan): https://x.com/labenz

LinkedIn: https://linkedin.com/in/nathanlabenz/

Youtube: https://youtube.com/@CognitiveRevolutionPodcast

Apple: https://podcasts.apple.com/de/podcast/the-cognitive-revolution-ai-builders-researchers-and/id1669813431

Spotify: https://open.spotify.com/show/6yHyok3M3BjqzR0VB5MSyk

Theories of Everything with Curt Jaimungal features long-form, technically detailed interviews with leading researchers in physics, mathematics, consciousness, and philosophy, exploring topics at the level of active research. For academics, graduate students, and anyone seeking depth beyond popular science.

Learn more about your ad choices. Visit megaphone.fm/adchoices

Melvyn Bragg and guests discuss the dance which, from when it reached Britain in the early nineteenth century, revolutionised the relationship between music, literature and people here for the next hundred years. While it may seem formal now, it was the informality and daring that drove its popularity, with couples holding each other as they spun round a room to new lighter music popularised by Johann Strauss, father and son, such as The Blue Danube. Soon the Waltz expanded the creative world in poetry, ballet, novellas and music, from the Ballets Russes of Diaghilev to Moon River and Are You Lonesome Tonight.

With

Susan Jones Emeritus Professor of English Literature at the University of Oxford

Derek B. Scott Professor Emeritus of Music at the University of Leeds

And

Theresa Buckland Emeritus Professor of Dance History and Ethnography at the University of Roehampton

Producer: Simon Tillotson

Reading list:

Egil Bakka, Theresa Jill Buckland, Helena Saarikoski, and Anne von Bibra Wharton (eds.), Waltzing Through Europe: Attitudes towards Couple Dances in the Long Nineteenth Century, (Open Book Publishers, 2020)

Theresa Jill Buckland, ‘How the Waltz was Won: Transmutations and the Acquisition of Style in Early English Modern Ballroom Dancing. Part One: Waltzing Under Attack’ (Dance Research, 36/1, 2018); ‘Part Two: The Waltz Regained’ (Dance Research, 36/2, 2018)

Theresa Jill Buckland, Society Dancing: Fashionable Bodies in England, 1870-1920 (Palgrave Macmillan, 2011)

Erica Buurman, The Viennese Ballroom in the Age of Beethoven (Cambridge University Press, 2022)

Paul Cooper, ‘The Waltz in England, c. 1790-1820’ (Paper presented at Early Dance Circle conference, 2018)

Sherril Dodds and Susan Cook (eds.), Bodies of Sound: Studies Across Popular Dance and Music (Ashgate, 2013), especially ‘Dancing Out of Time: The Forgotten Boston of Edwardian England’ by Theresa Jill Buckland

Zelda Fitzgerald, Save Me the Waltz (first published 1932; Vintage Classics, 2001)

Hilary French, Ballroom: A People's History of Dancing (Reaktion Books, 2022)

Susan Jones, Literature, Modernism, and Dance (Oxford University Press, 2013)

Mark Knowles, The Wicked Waltz and Other Scandalous Dances: Outrage at Couple Dancing in the 19th and Early 20th Centuries (McFarland, 2009)

Rosamond Lehmann, Invitation to the Waltz (first published 1932; Virago, 2006)

Eric McKee, Decorum of the Minuet, Delirium of the Waltz: A Study of Dance-Music Relations in 3/4 Time (Indiana University Press, 2012)

Eduard Reeser, The History of the Walz (Continental Book Co., 1949)

Stanley Sadie (ed.), The New Grove Dictionary of Music and Musicians, Vol. 27 (Macmillan, 2nd ed., 2000), especially ‘Waltz’ by Andrew Lamb

Derek B. Scott, Sounds of the Metropolis: The 19th-Century Popular Music Revolution in London, New York, Paris and Vienna (Oxford University Press, 2008), especially the chapter ‘A Revolution on the Dance Floor, a Revolution in Musical Style: The Viennese Waltz’

Joseph Wechsberg, The Waltz Emperors: The Life and Times and Music of the Strauss Family (Putnam, 1973)

Cheryl A. Wilson, Literature and Dance in Nineteenth-century Britain (Cambridge University Press, 2009)

Virginia Woolf, The Voyage Out (first published 1915; William Collins, 2013)

Virginia Woolf, The Years (first published 1937; Vintage Classics, 2016)

David Wyn Jones, The Strauss Dynasty and Habsburg Vienna (Cambridge University Press, 2023)

Sevin H. Yaraman, Revolving Embrace: The Waltz as Sex, Steps, and Sound (Pendragon Press, 2002)

Rishona Zimring, Social Dance and the Modernist Imagination in Interwar Britain (Ashgate Press, 2013)

Brian Armstrong is the co-founder and CEO of Coinbase, the largest cryptocurrency exchange in the United States by trading volume and users. He launched Coinbase in 2012 after working as a software engineer at Airbnb, where he experienced firsthand the frictions of global payment systems. Under his leadership, Coinbase grew into a publicly traded company on Nasdaq in 2021 and now serves over 100 million verified users in more than 100 countries. Beyond Coinbase, Armstrong has co-founded initiatives like ResearchHub and NewLimit and is a prominent advocate for an open, crypto-powered financial system.

------

Thank you to the sponsors that fuel our podcast and our team:

Athletic Nicotine

https://www.AthleticNicotine.com/tetra

Use code 'TETRA'

------

Squarespace

https://Squarespace.com/tetra

Use code 'TETRA'

------

LMNT Electrolytes

https://DrinkLMNT.com/tetra

Use code 'TETRA'

------

Sign up to receive Tetragrammaton Transmissions

https://www.tetragrammaton.com/join-newsletter

Cass Sunstein is one of the most widely cited legal scholars of all time and among the most prolific writers working today. This year alone he has five books out, including Imperfect Oracle on the strengths and limits of AI and On Liberalism: In Defense of Freedom. In his second appearance on the show, he brings his characteristic intellectual range to exploring liberalism's present precariousness and AI's implications for law and speech.

Tyler and Cass discuss whether liberalism is self-undermining or simply vulnerable to illiberal forces, the tensions in how a liberal immigration regime would work, whether new generations of liberal thinkers are emerging, if Derek Parfit counts as a liberal, Mill's liberal wokeism, the allure of Mises' "cranky enthusiasm for freedom," whether the central claim of The Road to Serfdom holds up, how to blend indigenous rights with liberal thought, whether AIs should have First Amendment protections, the argument for establishing a right not to be manipulated, better remedies for low-grade libel, whether we should have trials run by AI, how Bob Dylan embodies liberal freedom, Cass' next book about animal rights, and more.

Read a full transcript enhanced with helpful links, or watch the full video on the new dedicated Conversations with Tyler channel.

Recorded October 10th, 2025.

This episode was made possible through the support of the John Templeton Foundation.

Other ways to connect

• Follow us on X and Instagram
• Follow Tyler on X
• Follow Cass on X
• Sign up for our newsletter
• Join our Discord
• Email us: cowenconvos@mercatus.gmu.edu
• Learn more about Conversations with Tyler and other Mercatus Center podcasts here.

Episode 143

I spoke with Iason Gabriel about:

* Value alignment

* Technology and worldmaking

* How AI systems affect individuals and the social world

Iason is a philosopher and Senior Staff Research Scientist at Google DeepMind. His work focuses on the ethics of artificial intelligence, including questions about AI value alignment, distributive justice, language ethics and human rights.

You can find him on his website and Twitter/X.

Find me on Twitter (or LinkedIn if you want…) for updates, and reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.

Outline

* (00:00) Intro

* (01:18) Iason’s intellectual development

* (04:28) Aligning language models with human values, democratic civility and agonism

* (08:20) Overlapping consensus, differing norms, procedures for identifying norms

* (13:27) Rawls’ theory of justice, the justificatory and stability problems

* (19:18) Aligning LLMs and cooperation, speech acts, justification and discourse norms, literacy

* (23:45) Actor Network Theory and alignment

* (27:25) Value alignment and Iason’s starting points

* (33:10) The Ethics of Advanced AI Assistants, AI’s impacts on social processes and users, personalization

* (37:50) AGI systems and social power

* (39:00) Displays of care and compassion, Machine Love (Joel Lehman)

* (41:30) Virtue ethics, morality and language, virtue in AI systems vs. MacIntyre’s conception in After Virtue

* (45:00) The Challenge of Value Alignment

* (45:25) Technologists as worldmakers

* (51:30) Technological determinism, collective action problems

* (55:25) Iason’s goals with his work

* (58:32) Outro

Links

Papers:

* AI, Values, and Alignment (2020)

* Aligning LMs with Human Values (2023)

* Toward a Theory of Justice for AI (2023)

* The Ethics of Advanced AI Assistants (2024)

* A matter of principle? AI alignment as the fair treatment of claims (2025)

Tony Hawk is a professional skateboarder widely regarded as one of the most influential figures in the sport’s history. Rising to prominence in the 1980s and 1990s, he became known for pioneering tricks like the 900 and for pushing skateboarding into the mainstream through competitions, video games, and media appearances. He became a household name through victorious X Games performances, and his career highlights
include being the first to land the 900 in competition, earning over 70 contest victories, and dominating vert skating across two decades. As an ambassador for the sport, he also founded the Tony Hawk Foundation to support youth skateparks, further cementing his influence and legacy in skateboarding culture.

------

Thank you to the sponsors that fuel our podcast and our team:

Athletic Nicotine

https://www.AthleticNicotine.com/tetra

Use code 'TETRA'

------

LMNT Electrolytes

https://DrinkLMNT.com/tetra

Use code 'TETRA'

------

Squarespace

https://Squarespace.com/tetra

Use code 'TETRA'

------

Sign up to receive Tetragrammaton Transmissions

https://www.tetragrammaton.com/join-newsletter

In this episode, Tyler Cowen and I discuss freedom. We talk about how people talk about freedom, whether you can define freedom, freedom's relation to other concepts, the relevance of free will and consciousness, what it might mean for a country to be free, and much more. In short, we do some philosophy! I hope you enjoy it..

Vitalik Buterin is the founder of Ethereum, the world's second-largest cryptocurrency by market cap, currently valued at around $500 billion. But beyond revolutionizing blockchain technology, Vitalik has become one of the most thoughtful voices on AI safety and existential risk.

He's donated over $665 million to pandemic prevention and other causes, and has a 12% P(Doom) – putting him squarely in what I consider the "sane zone" for AI risk assessment. What makes Vitalik particularly interesting is that he's both a hardcore techno-optimist who built one of the most successful decentralized systems ever created, and someone willing to seriously consider AI regulation and coordination mechanisms.

Vitalik coined the term "d/acc" – defensive, decentralized, democratic, differential acceleration – as a middle path between uncritical AI acceleration and total pause scenarios. He argues we need to make the world more like Switzerland (defensible, decentralized) and less like the Eurasian steppes (vulnerable to conquest).

We dive deep into the tractability of AI alignment, whether current approaches like DAC can actually work when superintelligence arrives, and why he thinks a pluralistic world of competing AIs might be safer than a single aligned superintelligence. We also explore his vision for human-AI merger through brain-computer interfaces and uploading.

The crux of our disagreement is that I think we're heading for a "plants vs. animals" scenario where AI will simply operate on timescales we can't match, while Vitalik believes we can maintain agency through the right combination of defensive technologies and institutional design.

Finally, we tackle the discourse itself – I ask Vitalik to debunk the common ad hominem attacks against AI doomers, from "it's just a fringe position" to "no real builders believe in doom." His responses carry weight given his credibility as both a successful entrepreneur and someone who's maintained intellectual honesty throughout his career.

Timestamps

* 00:00:00 - Cold Open

* 00:00:37 - Introducing Vitalik Buterin

* 00:02:14 - Vitalik's altruism

* 00:04:36 - Rationalist community influence

* 00:06:30 - Opinion of Eliezer Yudkowsky and MIRI

* 00:09:00 - What’s Your P(Doom)™

* 00:24:42 - AI timelines

* 00:31:33 - AI consciousness

* 00:35:01 - Headroom above human intelligence

* 00:48:56 - Techno optimism discussion

* 00:58:38 - e/acc: Vibes-based ideology without deep arguments

* 01:02:49 - d/acc: Defensive, decentralized, democratic acceleration

* 01:11:37 - How plausible is d/acc?

* 01:20:53 - Why libertarian acceleration can paradoxically break decentralization

* 01:25:49 - Can we merge with AIs?

* 01:35:10 - Military AI concerns: How war accelerates dangerous development

* 01:42:26 - The intractability question

* 01:51:10 - Anthropic and tractability-washing the AI alignment problem

* 02:00:05 - The state of AI x-risk discourse

* 02:05:14 - Debunking ad hominem attacks against doomers

* 02:23:41 - Liron’s outro

Links

Vitalik’s website: https://vitalik.eth.limo

Vitalik’s Twitter: https://x.com/vitalikbuterin

Eliezer Yudkowsky’s explanation of p-Zombies: https://www.lesswrong.com/posts/fdEWWr8St59bXLbQr/zombies-zombies

—

Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.

Support the mission by subscribing to my Substack at DoomDebates.com and to youtube.com/@DoomDebates

Peter Thiel is unlike any other Trump tech bro. As well as a wildly successful investor, he’s seen as a thinker - the philosopher king of Silicon Valley. Thiel’s acolytes in the tech world and Washington include vice-president JD Vance but his relationship with the Trump camp is complicated. And there are still questions about what, if anything, he wants with the president.

In the final episode of this season of Tech Tonic, Murad Ahmed speaks to FT columnist Gillian Tett about Thiel’s political philosophy, and to Tabby Kinder, the FT’s West Coast financial editor, about his influence in Silicon Valley.

Free to read:

How Peter Thiel and Silicon Valley funded the sudden rise of JD Vance

A time for truth and reconciliation (written by Peter Thiel)

How a little-known French literary critic became a bellwether for the US right

Palantir’s ‘revolving door’ with government spurs huge growth

This season of Tech Tonic is presented by Murad Ahmed and produced by Josh Gabert-Doyon. The senior producer is Edwin Lane and the executive producer is Flo Phillips. Sound design by Sam Giovinco. Breen Turner and Samantha Giovinco. Original music by Metaphor Music, Manuela Saragosa and Topher Forhecz are the FT’s acting co-heads of audio.

Read a transcript of this episode on FT.com

Hosted on Acast. See acast.com/privacy for more information.

JoeBen Bevirt has spent two decades building electric vertical take-off and landing (eVTOL) aircraft, and now he's on the cusp of commercial approval and rollout. Will flying cars be as transformational as the automobile? How will air taxis impact our cities and the way we live? And how did JoeBen achieve this feat of ingenuity?

This week we're joined by the Founder and CEO of Joby Aviation, an American aviation company pioneering eVTOL aircraft for air taxi service. All-electric, virtually silent, and traveling up to 200mph with a pilot and four passengers, Joby is opening new possibilities in the skies above — starting at the price of an Uber Black. The implications for productivity and quality of life are massive, saving the average person an hour or two a day sitting in traffic and unlocking new swaths of land for development.

I'm proud that 8VC co-led Joby's first investment round about a decade ago, when many others, even flying enthusiasts, thought it was a pipedream. Since then, Joby has single-handedly shaped an entire new industry, from engineering breakthroughs to regulatory pathways, ensuring that American aviation stays ahead of China. Joby expects its first passenger rides in Dubai within a year and is working closely with the Trump administration as it nears the final stages of FAA approval. Inspired by SpaceX, Joby is vertically integrated and plans to aggressively ramp manufacturing here in the U.S., backed by a $500 million investment from Toyota (bringing Toyota's total investment near $900 million). While we await the first passenger flights, Joby is also building out its infrastructure nationwide — and they're looking for real estate and partners! You can contact JoeBen and the team here: info@jobyaviation.com

00:00 Episode Intro

01:38 Flying cars are here

04:00 JoeBen's journey

05:48 Battery progress & hydrogen breakthroughs

08:50 Air taxi for the price of Uber Black

12:35 When will commercial flights start?

20:30 Why Joby is the industry leader

24:20 Why China is copying Joby

28:00 How air taxis will change your life

32:10 How Joby will transform real estate

35:45 Solving intractable problems

This is an interview with Richard Ngo, AGI researcher and thinker - with extensive stints at both OpenAI and DeepMind.

This is an additional installment of our "Worthy Successor" series - where we explore the kinds of posthuman intelligences that deserve to steer the future beyond humanity.

This episode referred to the following other essays and resources:
-- A Worthy Successor - The Purpose of AGI: https://danfaggella.com/worthy
-- Richard's exploratory fiction writing - http://narrativeark.xyz/

Watch this episode on The Trajectory YouTube channel: https://youtu.be/UQpds4PXMjQ

See the full article from this episode: https://danfaggella.com/ngo1

...

About The Trajectory:

AGI and man-machine merger are going to radically expand the process of life beyond humanity -- so how can we ensure a good trajectory for future life?

From Yoshua Bengio to Nick Bostrom, from Michael Levin to Peter Singer, we discuss how to positively influence the trajectory of posthuman life with the greatest minds in AI, biology, philosophy, and policy.

Ask questions of our speakers in our live Philosophy Circle calls:
https://bit.ly/PhilosophyCircle

Stay in touch:
-- Newsletter: bit.ly/TrajectoryTw
-- X: x.com/danfaggella
-- Blog: danfaggella.com/trajectory
-- YouTube: youtube.com/@trajectoryai

Patrick McKenzie (patio11) is joined by Azeem Azhar, writer of the Exponential View newsletter, to discuss the massive data center buildout powering AI and its implications for our energy infrastructure. The conversation covers the physical limitations of modern datacenters, the challenges of electricity generation, the societal ripples from historical largescale infrastructure investments like railways and telecommunications, and the future of energy including solar, nuclear and geothermal power. Through their discussion, Patrick and Azeem explain why our mental models for both computing and energy systems need to be updated.
–
Full transcript available here: www.complexsystemspodcast.com/ai-llm-data-center-power-economics/

–
Sponsors:  Safebase | Check

Ready to save time and close deals faster? Inbound security reviews shouldn’t slow down your team or your sales cycle. Leading companies use SafeBase to eliminate up to 98% of inbound security questionnaires, automate workflows, and accelerate pipeline. Go to safebase.io/podcast

Check is the leading payroll infrastructure provider and pioneer of embedded payroll. Check makes it easy for any SaaS platform to build a payroll business, and already powers 60+ popular platforms. Head to checkhq.com/complex and tell them patio11 sent you.

–

Recommended in this episode:

• Azeem’s newsletter: https://www.exponentialview.co/ 
• Azeem Azhar’s guest essay: The 19th-Century Technology That Threatens A.I. https://www.nytimes.com/2024/12/28/opinion/ai-electricity-power-plants.html
• Electric Twin: https://www.electrictwin.com/ 
• Video of Elon Musk’s Colossus https://www.youtube.com/watch?v=Tw696JVSxJQ 
• Complex Systems with Travis Dauwalter on the electrical grid: https://open.spotify.com/episode/5JY8e84sEXmHFlc8IR2kRb?si=35ymIC0UQ5SKdV8rrBcgIw 
• Complex Systems with Austin Vernon on fracking: https://open.spotify.com/episode/0YDV1XyjUCM2RtuTcBGYH9?si=YshjUXPEQBiScNxrNaI-Gw 
• Complex Systems with Casey Handmer on direct capture of CO2 to turn into hydrocarbon: https://open.spotify.com/episode/0GHegWgLSubYxvATmbWhQu?si=xNYBjn0ZTX2IT_pAZ5Ozsg 

–

Twitter:

@azeem

@patio11

–

Timestamps:

(00:00) Intro 

(00:27) The power economics of data centers

(01:12) Historical infrastructure rollouts

(04:58) The telecoms bubble 

(06:22) Unprecedented enterprise spend on AI capabilities

(11:12) Let's have your LLM talk to my LLM

(16:44) Is there a saturation point?

(19:25) Sponsors: Safebase | Check

(21:55) What’s in a data center?

(24:52) The challenges of data centers

(29:40) Geographical considerations for data centers

(36:53) Energy consumption and future needs

(40:48) Challenges in building transmission lines

(41:35) The solar power learning curve

(43:51) Small modular nuclear reactors

(51:26) Geothermal energy and fracking

(01:01:34) The future of AI and energy systems

(01:12:57) Wrap

Technology doesn’t force us to do anything — it merely opens doors. But military and economic competition pushes us through.

That’s how today’s guest Allan Dafoe — director of frontier safety and governance at Google DeepMind — explains one of the deepest patterns in technological history: once a powerful new capability becomes available, societies that adopt it tend to outcompete those that don’t. Those who resist too much can find themselves taken over or rendered irrelevant.

Links to learn more, highlights, video, and full transcript.

This dynamic played out dramatically in 1853 when US Commodore Perry sailed into Tokyo Bay with steam-powered warships that seemed magical to the Japanese, who had spent centuries deliberately limiting their technological development. With far greater military power, the US was able to force Japan to open itself to trade. Within 15 years, Japan had undergone the Meiji Restoration and transformed itself in a desperate scramble to catch up.

Today we see hints of similar pressure around artificial intelligence. Even companies, countries, and researchers deeply concerned about where AI could take us feel compelled to push ahead — worried that if they don’t, less careful actors will develop transformative AI capabilities at around the same time anyway.

But Allan argues this technological determinism isn’t absolute. While broad patterns may be inevitable, history shows we do have some ability to steer how technologies are developed, by who, and what they’re used for first.

As part of that approach, Allan has been promoting efforts to make AI more capable of sophisticated cooperation, and improving the tests Google uses to measure how well its models could do things like mislead people, hack and take control of their own servers, or spread autonomously in the wild.

As of mid-2024 they didn’t seem dangerous at all, but we’ve learned that our ability to measure these capabilities is good, but imperfect. If we don’t find the right way to ‘elicit’ an ability we can miss that it’s there.

Subsequent research from Anthropic and Redwood Research suggests there’s even a risk that future models may play dumb to avoid their goals being altered.

That has led DeepMind to a “defence in depth” approach: carefully staged deployment starting with internal testing, then trusted external testers, then limited release, then watching how models are used in the real world. By not releasing model weights, DeepMind is able to back up and add additional safeguards if experience shows they’re necessary.

But with much more powerful and general models on the way, individual company policies won’t be sufficient by themselves. Drawing on his academic research into how societies handle transformative technologies, Allan argues we need coordinated international governance that balances safety with our desire to get the massive potential benefits of AI in areas like healthcare and education as quickly as possible.

Host Rob and Allan also cover:

• The most exciting beneficial applications of AI
• Whether and how we can influence the development of technology
• What DeepMind is doing to evaluate and mitigate risks from frontier AI systems
• Why cooperative AI may be as important as aligned AI
• The role of democratic input in AI governance
• What kinds of experts are most needed in AI safety and governance
• And much more

Chapters:

• Cold open (00:00:00)
• Who's Allan Dafoe? (00:00:45)
• Allan's role at DeepMind (00:01:26)
• Why join DeepMind over everyone else? (00:04:31)
• Do humans control technological change? (00:09:30)
• Arguments for technological determinism (00:20:57)
• The synthesis of agency with tech determinism (00:27:10)
• Competition took away Japan's choice (00:38:11)
• Can speeding up one tech redirect history? (00:43:20)
• Structural pushback against alignment efforts (00:49:17)
• Do AIs need to be 'cooperatively skilled'? (00:53:53)
• How AI could boost cooperation between people and states (01:03:34)
• The super-cooperative AGI hypothesis and backdoor risks (01:08:44)
• Aren’t today’s models already very cooperative? (01:15:15)
• How would we make AIs cooperative anyway? (01:18:19)
• Ways making AI more cooperative could backfire (01:24:36)
• AGI is an essential idea we should define well (01:32:37)
• It matters what AGI learns first vs last (01:43:39)
• How Google tests for dangerous capabilities (01:48:24)
• Evals 'in the wild' (02:00:57)
• What to do given no single approach works that well (02:05:05)
• We don't, but could, forecast AI capabilities (02:09:08)
• DeepMind's strategy for ensuring its frontier models don't cause harm (02:15:05)
• How 'structural risks' can force everyone into a worse world (02:18:52)
• Is AI being built democratically? Should it? (02:23:40)
• How much do AI companies really want external regulation? (02:29:01)
• Social science can contribute a lot here (02:38:07)
• How AI could make life way better: Self-driving cars, medicine, education, and sustainability (02:40:48)

Video editing: Simon Monsour
Audio engineering: Ben Cordell, Milo McGuire, Simon Monsour, and Dominic Armstrong
Camera operator: Jeremy Chevillotte
Transcriptions: Katy Moore

In this episode, Edward Hughes, researcher at Google DeepMind, and Aron Vallinder, an independent researcher and PIBBSS fellow discuss their pioneering research on cultural evolution and cooperation among large language model agents. The conversation delves into the study's design, exploring how different AI models exhibit cooperative behavior in simulated environments, the implications of these findings for future AI development, and the potential societal impacts of autonomous AI agents. They elaborate on their experimental setup involving different LLMs like Claude, Gemini 1.5, and GPT-4.0 in a cooperative donor-recipient game, shedding light on how various AI models handle cooperation and their potential societal impacts. Key points include the importance of understanding externalities, the role of punishment and communication, and future research directions involving mixed-model societies and human-AI interactions. The episode invites listeners to engage in this fast-growing field, stressing the need for more hands-on research and empirical evidence to navigate the rapidly evolving AI landscape.

Link to Aron & Edward's research paper "Cultural Evolution of Cooperation among LLM

Agents"

SPONSORS:

Oracle Cloud Infrastructure (OCI): Oracle's next-generation cloud platform delivers blazing-fast AI and ML performance with 50% less for compute and 80% less for outbound networking compared to other cloud providers. OCI powers industry leaders like Vodafone and Thomson Reuters with secure infrastructure and application development capabilities. New U.S. customers can get their cloud bill cut in half by switching to OCI before March 31, 2024 at https://oracle.com/cognitive

NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive

Shopify: Shopify is revolutionizing online selling with its market-leading checkout system and robust API ecosystem. Its exclusive library of cutting-edge AI apps empowers e-commerce businesses to thrive in a competitive market. Cognitive Revolution listeners can try Shopify for just $1 per month at https://shopify.com/cognitive

CHAPTERS:

(00:00) Teaser

(00:42) About the Episode

(03:26) Introduction

(03:40) The Rapid Evolution of AI

(04:58) Human Cooperation and Society

(07:03) Cultural Evolution and Stories

(08:39) Mechanisms of Cultural Evolution (Part 1)

(20:56) Sponsors: Oracle Cloud Infrastructure (OCI) | NetSuite

(23:35) Mechanisms of Cultural Evolution (Part 2)

(27:07) Experimental Setup: Donor Game (Part 1)

(37:35) Sponsors: Shopify

(38:55) Experimental Setup: Donor Game (Part 2)

(44:32) Exploring AI Societies: Claude, Gemini, and GPT-4

(45:50) Striking Graphical Differences

(48:08) Experiment Results and Implications

(50:54) Prompt Engineering and Cooperation

(57:40) Mixed Model Societies

(01:00:35) Future Research Directions

(01:03:10) Human-AI Interaction and Influence

(01:05:20) Complexifying AI Games

(01:18:14) Evaluations and Feedback Loops

(01:20:50) Open Source and AI Safety

(01:23:23) Reflections and Future Work

(01:30:04) Outro

In our first episode of Epoch After Hours, Ege, Tamay and Jaime dig into what they expect AI to look like by 2030; why economists are underestimating the likelihood of explosive growth; the startling regularity in technological trends like Moore's Law; Moravec’s paradox, and how we might overcome it; and much more!

Ajeya Cotra works at Open Philanthropy, a leading funder of efforts to combat existential risks from AI. She has led the foundation’s grantmaking on technical research to understand and reduce catastrophic risks from advanced AI. She is co-author of Planned Obsolescence, a newsletter about AI futurism and AI alignment.

Although a committed doomer herself, Cotra has worked hard to understand the perspectives of AI safety skeptics. In this episode, we asked her to guide us through the contentious debate over AI safety and—perhaps—explain why people with similar views on other issues frequently reach divergent views on this one. We spoke to Cotra on December 10.

Nora Belrose, Head of Interpretability Research at EleutherAI, discusses critical challenges in AI safety and development. The conversation begins with her technical work on concept erasure in neural networks through LEACE (LEAst-squares Concept Erasure), while highlighting how neural networks' progression from simple to complex learning patterns could have important implications for AI safety.

Many fear that advanced AI will pose an existential threat -- pursuing its own dangerous goals once it's powerful enough. But Belrose challenges this popular doomsday scenario with a fascinating breakdown of why it doesn't add up.

Belrose also provides a detailed critique of current AI alignment approaches, particularly examining "counting arguments" and their limitations when applied to AI safety. She argues that the Principle of Indifference may be insufficient for addressing existential risks from advanced AI systems. The discussion explores how emergent properties in complex AI systems could lead to unpredictable and potentially dangerous behaviors that simple reductionist approaches fail to capture.

The conversation concludes by exploring broader philosophical territory, where Belrose discusses her growing interest in Buddhism's potential relevance to a post-automation future. She connects concepts of moral anti-realism with Buddhist ideas about emptiness and non-attachment, suggesting these frameworks might help humans find meaning in a world where AI handles most practical tasks. Rather than viewing this automated future with alarm, she proposes that Zen Buddhism's emphasis on spontaneity and presence might complement a society freed from traditional labor.

SPONSOR MESSAGES:

CentML offers competitive pricing for GenAI model deployment, with flexible options to suit a wide range of models, from small to large-scale deployments.

https://centml.ai/pricing/

Tufa AI Labs is a brand new research lab in Zurich started by Benjamin Crouzier focussed on ARC and AGI, they just acquired MindsAI - the current winners of the ARC challenge. Are you interested in working on ARC, or getting involved in their events? Goto https://tufalabs.ai/

Nora Belrose:

https://norabelrose.com/

https://scholar.google.com/citations?user=p_oBc64AAAAJ&hl=en

https://x.com/norabelrose

SHOWNOTES:

https://www.dropbox.com/scl/fi/38fhsv2zh8gnubtjaoq4a/NORA_FINAL.pdf?rlkey=0e5r8rd261821g1em4dgv0k70&st=t5c9ckfb&dl=0

TOC:

1. Neural Network Foundations

[00:00:00] 1.1 Philosophical Foundations and Neural Network Simplicity Bias

[00:02:20] 1.2 LEACE and Concept Erasure Fundamentals

[00:13:16] 1.3 LISA Technical Implementation and Applications

[00:18:50] 1.4 Practical Implementation Challenges and Data Requirements

[00:22:13] 1.5 Performance Impact and Limitations of Concept Erasure

2. Machine Learning Theory

[00:32:23] 2.1 Neural Network Learning Progression and Simplicity Bias

[00:37:10] 2.2 Optimal Transport Theory and Image Statistics Manipulation

[00:43:05] 2.3 Grokking Phenomena and Training Dynamics

[00:44:50] 2.4 Texture vs Shape Bias in Computer Vision Models

[00:45:15] 2.5 CNN Architecture and Shape Recognition Limitations

3. AI Systems and Value Learning

[00:47:10] 3.1 Meaning, Value, and Consciousness in AI Systems

[00:53:06] 3.2 Global Connectivity vs Local Culture Preservation

[00:58:18] 3.3 AI Capabilities and Future Development Trajectory

4. Consciousness Theory

[01:03:03] 4.1 4E Cognition and Extended Mind Theory

[01:09:40] 4.2 Thompson's Views on Consciousness and Simulation

[01:12:46] 4.3 Phenomenology and Consciousness Theory

[01:15:43] 4.4 Critique of Illusionism and Embodied Experience

[01:23:16] 4.5 AI Alignment and Counting Arguments Debate

(TRUNCATED, TOC embedded in MP3 file with more information)

Andrej Karpathy joins Sarah and Elad in this week of No Priors. Andrej, who was a founding team member of OpenAI and former Senior Director of AI at Tesla, needs no introduction. In this episode, Andrej discusses the evolution of self-driving cars, comparing Tesla and Waymo’s approaches, and the technical challenges ahead. They also cover Tesla’s Optimus humanoid robot, the bottlenecks of AI development today, and  how AI capabilities could be further integrated with human cognition.  Andrej shares more about his new company Eureka Labs and his insights into AI-driven education, peer networks, and what young people should study to prepare for the reality ahead.

Sign up for new podcasts every week. Email feedback to show@no-priors.com

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @Karpathy

Show Notes: 

(0:00) Introduction

(0:33) Evolution of self-driving cars

(2:23) The Tesla  vs. Waymo approach to self-driving 

(6:32) Training Optimus  with automotive models

(10:26) Reasoning behind the humanoid form factor

(13:22) Existing challenges in robotics

(16:12) Bottlenecks of AI progress 

(20:27) Parallels between human cognition and AI models

(22:12) Merging human cognition with AI capabilities

(27:10) Building high performance small models

(30:33) Andrej’s current work in AI-enabled education

(36:17) How AI-driven education reshapes knowledge networks and status

(41:26) Eureka Labs

(42:25) What young people study to prepare for the future

Dr. Joscha Bach introduces a surprising idea called "cyber animism" in his AGI-24 talk - the notion that nature might be full of self-organizing software agents, similar to the spirits in ancient belief systems. Bach suggests that consciousness could be a kind of software running on our brains, and wonders if similar "programs" might exist in plants or even entire ecosystems.

MLST is sponsored by Brave:

The Brave Search API covers over 20 billion webpages, built from scratch without Big Tech biases or the recent extortionate price hikes on search API access. Perfect for AI model training and retrieval augmentated generation. Try it now - get 2,000 free queries monthly at http://brave.com/api.

Joscha takes us on a tour de force through history, philosophy, and cutting-edge computer science, teasing us to rethink what we know about minds, machines, and the world around us. Joscha believes we should blur the lines between human, artificial, and natural intelligence, and argues that consciousness might be more widespread and interconnected than we ever thought possible.

Dr. Joscha Bach

https://x.com/Plinz

This is video 2/9 from our coverage of AGI-24 in Seattle https://agi-conf.org/2024/

Watch the official MLST interview with Joscha which we did right after this talk on our Patreon now on early access - https://www.patreon.com/posts/joscha-bach-110199676 (you also get access to our private discord and biweekly calls)

TOC:

00:00:00 Introduction: AGI and Cyberanimism

00:03:57 The Nature of Consciousness

00:08:46 Aristotle's Concepts of Mind and Consciousness

00:13:23 The Hard Problem of Consciousness

00:16:17 Functional Definition of Consciousness

00:20:24 Comparing LLMs and Human Consciousness

00:26:52 Testing for Consciousness in AI Systems

00:30:00 Animism and Software Agents in Nature

00:37:02 Plant Consciousness and Ecosystem Intelligence

00:40:36 The California Institute for Machine Consciousness

00:44:52 Ethics of Conscious AI and Suffering

00:46:29 Philosophical Perspectives on Consciousness

00:49:55 Q&A: Formalisms for Conscious Systems

00:53:27 Coherence, Self-Organization, and Compute Resources

YT version (very high quality, filmed by us live)

https://youtu.be/34VOI_oo-qM

Refs:

Aristotle's work on the soul and consciousness

Richard Dawkins' work on genes and evolution

Gerald Edelman's concept of Neural Darwinism

Thomas Metzinger's book "Being No One"

Yoshua Bengio's concept of the "consciousness prior"

Stuart Hameroff's theories on microtubules and consciousness

Christof Koch's work on consciousness

Daniel Dennett's "Cartesian Theater" concept

Giulio Tononi's Integrated Information Theory

Mike Levin's work on organismal intelligence

The concept of animism in various cultures

Freud's model of the mind

Buddhist perspectives on consciousness and meditation

The Genesis creation narrative (for its metaphorical interpretation)

California Institute for Machine Consciousness

This is an interview with Nick Bostrom, the Founding Director of Future of Humanity Institute Oxford.

This is the first installment of The Worthy Successor series - where we unpack the preferable and non-preferable futures humanity might strive towards in the years ahead.

This episode referred to the following other essays and resources:

-- The Intelligence Trajectory Political Matrix: danfaggella.com/itpm 

-- Natural Selection Favors AIs over Humans: https://arxiv.org/abs/2303.16200

-- The SDGs of Strong AGI: https://emerj.com/ai-power/sdgs-of-ai/

Watch this episode on The Trajectory YouTube channel: https://youtu.be/_ZCE4XZ9doc?si=RXptg0y6JcxelXkF

Read the Nick Bostrom's episode highlight: danfaggella.com/bostrom1/

...

About The Trajectory:

AGI and man-machine merger are going to radically expand the process of life beyond humanity -- so how can we ensure a good trajectory for future life?

From Yoshua Bengio to Nick Bostrom, from Michael Levin to Peter Singer, we discuss how to positively influence the trajectory of posthuman life with the greatest minds in AI, biology, philosophy, and policy.

Ask questions of our speakers in our live Philosophy Circle calls:
https://bit.ly/PhilosophyCircle

Stay in touch:
-- Newsletter: bit.ly/TrajectoryTw
-- X: x.com/danfaggella
-- Blog: danfaggella.com/trajectory
-- YouTube: youtube.com/@trajectoryai

I talked with Patrick McKenzie (known online as patio11) about how a small team he ran over a Discord server got vaccines into Americans' arms: A story of broken incentives, outrageous incompetence, and how a few individuals with high agency saved 1000s of lives.

Enjoy!

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here.

Follow me on Twitter for updates on future episodes.

Sponsor

This episode is brought to you by Stripe, financial infrastructure for the internet. Millions of companies from Anthropic to Amazon use Stripe to accept payments, automate financial processes and grow their revenue.

Timestamps

(00:00:00) – Why hackers on Discord had to save thousands of lives

(00:17:26) – How politics crippled vaccine distribution

(00:38:19) – Fundraising for VaccinateCA

(00:51:09) – Why tech needs to understand how government works

(00:58:58) – What is crypto good for?

(01:13:07) – How the US government leverages big tech to violate rights

(01:24:36) – Can the US have nice things like Japan?

(01:26:41) – Financial plumbing & money laundering: a how-not-to guide

(01:37:42) – Maximizing your value: why some people negotiate better

(01:42:14) – Are young people too busy playing Factorio to found startups?

(01:57:30) – The need for a post-mortem

"You can’t charge what something is worth during a pandemic. So we estimated that the value of one course of COVID vaccine in January 2021 was over $5,000. They were selling for between $6 and $40. So nothing like their social value. Now, don’t get me wrong. I don’t think that they should have charged $5,000 or $6,000. That’s not ethical. It’s also not economically efficient, because they didn’t cost $5,000 at the marginal cost. So you actually want low price, getting out to lots of people.

"But it shows you that the market is not going to reward people who do the investment in preparation for a pandemic — because when a pandemic hits, they’re not going to get the reward in line with the social value. They may even have to charge less than they would in a non-pandemic time. So prepping for a pandemic is not an efficient market strategy if I’m a firm, but it’s a very efficient strategy for society, and so we’ve got to bridge that gap." —Rachel Glennerster

In today’s episode, host Luisa Rodriguez speaks to Rachel Glennerster — associate professor of economics at the University of Chicago and a pioneer in the field of development economics — about how her team’s new Market Shaping Accelerator aims to leverage market forces to drive innovations that can solve pressing world problems.

Links to learn more, highlights, and full transcript.

They cover:

• How market failures and misaligned incentives stifle critical innovations for social goods like pandemic preparedness, climate change interventions, and vaccine development.
• How “pull mechanisms” like advance market commitments (AMCs) can help overcome these challenges — including concrete examples like how one AMC led to speeding up the development of three vaccines which saved around 700,000 lives in low-income countries.
• The challenges in designing effective pull mechanisms, from design to implementation.
• Why it’s important to tie innovation incentives to real-world impact and uptake, not just the invention of a new technology.
• The massive benefits of accelerating vaccine development, in some cases, even if it’s only by a few days or weeks.
• The case for a $6 billion advance market commitment to spur work on a universal COVID-19 vaccine.
• The shortlist of ideas from the Market Shaping Accelerator’s recent Innovation Challenge that use pull mechanisms to address market failures around improving indoor air quality, repurposing generic drugs for alternative uses, and developing eco-friendly air conditioners for a warming planet.
• “Best Buys” and “Bad Buys” for improving education systems in low- and middle-income countries, based on evidence from over 400 studies.
• Lessons from Rachel’s career at the forefront of global development, and how insights from economics can drive transformative change.
• And much more.

Chapters:

• The Market Shaping Accelerator (00:03:33)
• Pull mechanisms for innovation (00:13:10)
• Accelerating the pneumococcal and COVID vaccines (00:19:05)
• Advance market commitments (00:41:46)
• Is this uncertainty hard for funders to plan around? (00:49:17)
• The story of the malaria vaccine that wasn’t (00:57:15)
• Challenges with designing and implementing AMCs and other pull mechanisms (01:01:40)
• Universal COVID vaccine (01:18:14)
• Climate-resilient crops (01:34:09)
• The Market Shaping Accelerator’s Innovation Challenge (01:45:40)
• Indoor air quality to reduce respiratory infections (01:49:09)
• Repurposing generic drugs (01:55:50)
• Clean air conditioning units (02:02:41)
• Broad-spectrum antivirals for pandemic prevention (02:09:11)
• Improving education in low- and middle-income countries (02:15:53)
• What’s still weird for Rachel about living in the US? (02:45:06)

Producer and editor: Keiran Harris
Audio Engineering Lead: Ben Cordell
Technical editing: Simon Monsour, Milo McGuire, and Dominic Armstrong
Additional content editing: Katy Moore and Luisa Rodriguez
Transcriptions: Katy Moore

Episode 122

I spoke with Professor David Thorstad about:

* The practical difficulties of doing interdisciplinary work

* Why theories of human rationality should account for boundedness, heuristics, and other cognitive limitations

* why EA epistemics suck (ok, it’s a little more nuanced than that)

Professor Thorstad is an Assistant Professor of Philosophy at Vanderbilt University, a Senior Research Affiliate at the Global Priorities Institute at Oxford, and a Research Affiliate at the MINT Lab at Australian National University. One strand of his research asks how cognitively limited agents should decide what to do and believe. A second strand asks how altruists should use limited funds to do good effectively.

Reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (01:15) David’s interest in rationality

* (02:45) David’s crisis of confidence, models abstracted from psychology

* (05:00) Blending formal models with studies of the mind

* (06:25) Interaction between academic communities

* (08:24) Recognition of and incentives for interdisciplinary work

* (09:40) Movement towards interdisciplinary work

* (12:10) The Standard Picture of rationality

* (14:11) Why the Standard Picture was attractive

* (16:30) Violations of and rebellion against the Standard Picture

* (19:32) Mistakes made by critics of the Standard Picture

* (22:35) Other competing programs vs Standard Picture

* (26:27) Characterizing Bounded Rationality

* (27:00) A worry: faculties criticizing themselves

* (29:28) Self-improving critique and longtermism

* (30:25) Central claims in bounded rationality and controversies

* (32:33) Heuristics and formal theorizing

* (35:02) Violations of Standard Picture, vindicatory epistemology

* (37:03) The Reason Responsive Consequentialist View (RRCV)

* (38:30) Objective and subjective pictures

* (41:35) Reason responsiveness

* (43:37) There are no epistemic norms for inquiry

* (44:00) Norms vs reasons

* (45:15) Arguments against epistemic nihilism for belief

* (47:30) Norms and self-delusion

* (49:55) Difficulty of holding beliefs for pragmatic reasons

* (50:50) The Gibbardian picture, inquiry as an action

* (52:15) Thinking how to act and thinking how to live — the power of inquiry

* (53:55) Overthinking and conducting inquiry

* (56:30) Is thinking how to inquire as an all-things-considered matter?

* (58:00) Arguments for the RRCV

* (1:00:40) Deciding on minimal criteria for the view, stereotyping

* (1:02:15) Eliminating stereotypes from the theory

* (1:04:20) Theory construction in epistemology and moral intuition

* (1:08:20) Refusing theories for moral reasons and disciplinary boundaries

* (1:10:30) The argument from minimal criteria, evaluating against competing views

* (1:13:45) Comparing to other theories

* (1:15:00) The explanatory argument

* (1:17:53) Parfit and Railton, norms of friendship vs utility

* (1:20:00) Should you call out your friend for being a womanizer

* (1:22:00) Vindicatory Epistemology

* (1:23:05) Panglossianism and meliorative epistemology

* (1:24:42) Heuristics and recognition-driven investigation

* (1:26:33) Rational inquiry leading to irrational beliefs — metacognitive processing

* (1:29:08) Stakes of inquiry and costs of metacognitive processing

* (1:30:00) When agents are incoherent, focuses on inquiry

* (1:32:05) Indirect normative assessment and its consequences

* (1:37:47) Against the Singularity Hypothesis

* (1:39:00) Superintelligence and the ontological argument

* (1:41:50) Hardware growth and general intelligence growth, AGI definitions

* (1:43:55) Difficulties in arguing for hyperbolic growth

* (1:46:07) Chalmers and the proportionality argument

* (1:47:53) Arguments for/against diminishing growth, research productivity, Moore’s Law

* (1:50:08) On progress studies

* (1:52:40) Improving research productivity and technology growth

* (1:54:00) Mistakes in the moral mathematics of existential risk, longtermist epistemics

* (1:55:30) Cumulative and per-unit risk

* (1:57:37) Back and forth with longtermists, time of perils

* (1:59:05) Background risk — risks we can and can’t intervene on, total existential risk

* (2:00:56) The case for longtermism is inflated

* (2:01:40) Epistemic humility and longtermism

* (2:03:15) Knowledge production — reliable sources, blog posts vs peer review

* (2:04:50) Compounding potential errors in knowledge

* (2:06:38) Group deliberation dynamics, academic consensus

* (2:08:30) The scope of longtermism

* (2:08:30) Money in effective altruism and processes of inquiry

* (2:10:15) Swamping longtermist options

* (2:12:00) Washing out arguments and justified belief

* (2:13:50) The difficulty of long-term forecasting and interventions

* (2:15:50) Theory of change in the bounded rationality program

* (2:18:45) Outro

Links:

* David’s homepage and Twitter and blog

* Papers mentioned/read

* Bounded rationality and inquiry

* Why bounded rationality (in epistemology)?

* Against the newer evidentialists

* The accuracy-coherence tradeoff in cognition

* There are no epistemic norms of inquiry

* Permissive metaepistemology

* Global priorities and effective altruism

* What David likes about EA

* Against the singularity hypothesis (+ blog posts)

* Three mistakes in the moral mathematics of existential risk (+ blog posts)

* The scope of longtermism

* Epistemics

In this conversation recorded live in Miami, Tyler and Peter Thiel dive deep into the complexities of political theology, including why it's a concept we still need today, why Peter's against Calvinism (and rationalism), whether the Old Testament should lead us to be woke, why Carl Schmitt is enjoying a resurgence, whether we're entering a new age of millenarian thought, the one existential risk Peter thinks we're overlooking, why everyone just muddling through leads to disaster, the role of the katechon, the political vision in Shakespeare, how AI will affect the influence of wordcels, Straussian messages in the Bible, what worries Peter about Miami, and more.

Read a full transcript enhanced with helpful links, or watch the full video.

Recorded February 21st, 2024.

Other ways to connect

• Follow us on X and Instagram
• Follow Tyler on X
• Follow Peter on X
• Sign up for our newsletter
• Join our Discord
• Email us: cowenconvos@mercatus.gmu.edu
• Learn more about Conversations with Tyler and other Mercatus Center podcasts here.

Sam Harris speaks with William MacAskill about the implosion of FTX and the effect that it has had on the Effective Altruism movement. They discuss the logic of "earning to give," the mind of SBF, his philanthropy, the character of the EA community, potential problems with focusing on long-term outcomes, AI risk, the effects of the FTX collapse on Will personally, and other topics.

If the Making Sense podcast logo in your player is BLACK, you can SUBSCRIBE to gain access to all full-length episodes at samharris.org/subscribe.

Learning how to train your mind is the single greatest investment you can make in life. That's why Sam Harris created the Waking Up app. From rational mindfulness practice to lessons on some of life's most important topics, join Sam as he demystifies the practice of meditation and explores the theory behind it.

Had so much fun chatting with my good friends Trenton Bricken and Sholto Douglas on the podcast.

No way to summarize it, except: 

This is the best context dump out there on how LLMs are trained, what capabilities they're likely to soon have, and what exactly is going on inside them.

You would be shocked how much of what I know about this field, I've learned just from talking with them.

To the extent that you've enjoyed my other AI interviews, now you know why.

So excited to put this out. Enjoy! I certainly did :)

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. 

There's a transcript with links to all the papers the boys were throwing down - may help you follow along.

Follow Trenton and Sholto on Twitter.

Timestamps

(00:00:00) - Long contexts

(00:16:12) - Intelligence is just associations

(00:32:35) - Intelligence explosion & great researchers

(01:06:52) - Superposition & secret communication

(01:22:34) - Agents & true reasoning

(01:34:40) - How Sholto & Trenton got into AI research

(02:07:16) - Are feature spaces the wrong way to think about intelligence?

(02:21:12) - Will interp actually work on superhuman models

(02:45:05) - Sholto’s technical challenge for the audience

(03:03:57) - Rapid fire

Who isn't a paperclipper?

Text version here: https://joecarlsmith.com/2024/01/11/an-even-deeper-atheism

This essay is part of a series I'm calling "Otherness and control in the age of AGI." I'm hoping that individual essays can be read fairly well on their own, but see here for brief summaries of the essays that have been released thus far: https://joecarlsmith.com/2024/01/02/otherness-and-control-in-the-age-of-agi

Flock is a public safety technology platform that operates in over 4,000 cities across the United States, and solves about 2,200 crimes daily. That’s 10 percent of reported crimes nationwide.

Taken from a16z’s recent LP Summit, a16z General Partner David Ulevitch  joins forces with Flock Safety’s founder, Garrett Langley and Sheriff Kevin McMahill of the Las Vegas Metropolitan Police Department. Together, they cover the delicate balance between using technology to combat crime and respecting individual privacy, and explore the use of drones and facial recognition, building trust within communities, and the essence of objective policing.

Resources: 

Find Garret on Twitter: https://twitter.com/glangley

Find Sheriff McMahill on Twitter:  https://twitter.com/Sheriff_LVMPD

Find David on Twitter:https://twitter.com/davidu

Learn more about Flock Safety: https://www.flocksafety.com

Stay Updated: 

Find a16z on Twitter: https://twitter.com/a16z 

Find a16z on LinkedIn: https://www.linkedin.com/company/a16z 

Subscribe on your favorite podcast app: https://a16z.simplecast.com/ 

Follow our host: https://twitter.com/stephsmithio 

Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

Stay Updated:

Find a16z on YouTube: YouTube

Find a16z on X

Find a16z on LinkedIn

Listen to the a16z Show on Spotify

Listen to the a16z Show on Apple Podcasts

Follow our host: https://twitter.com/eriktorenberg

Please note that the content here is for informational purposes only; should NOT be taken as legal, business, tax, or investment advice or be used to evaluate any investment or security; and is not directed at any investors or potential investors in any a16z fund. a16z and its affiliates may maintain investments in the companies discussed. For more details please see a16z.com/disclosures.

Hosted by Simplecast, an AdsWizz company. See pcm.adswizz.com for information about our collection and use of personal data for advertising.

In this episode, Flo Crivello, founder of Lindy AI, joins Nathan to chat about Biden’s executive order, and the state of AI safety. They discuss Flo’s thoughts on the executive order, building AGI kill switches, self driving cars, and more. If you need an ERP platform, check out our sponsor NetSuite: http://netsuite.com/cognitive.

SPONSORS: Netsuite | Omneky

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of ALL eCommerce in the US. And Shopify's the global force behind Allbirds, Rothy's, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries.From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you're selling, Shopify's got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/cognitive

NetSuite has 25 years of providing financial software for all your business needs. More than 36,000 businesses have already upgraded to NetSuite by Oracle, gaining visibility and control over their financials, inventory, HR, eCommerce, and more. If you're looking for an ERP platform ✅ head to NetSuite: http://netsuite.com/cognitive and download your own customized KPI checklist.

Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

RECOMMENDED PODCAST: 

Every week investor and writer of the popular newsletter The Diff, Byrne Hobart, and co-host Erik Torenberg discuss today’s major inflection points in technology, business, and markets – and help listeners build a diversified portfolio of trends and ideas for the future. Subscribe to “The Riff” with Byrne Hobart and Erik Torenberg: https://link.chtbl.com/theriff

TIMESTAMPS:

(00:00) Episode Preview

(00:06:42) The natural order of technological progress

(00:07:00) Self driving cars

(00:10:57) Where is Flo accelerationist?

(00:12:34) Artificial intelligence as a new form of life

(00:17:08) - Sponsors: Oracle | Omneky

(00:18:05) Silicon-based intelligence vs carbon-based intelligence

(00:24:36) Executive Order

(00:29:32) How would a GPU kill switch work?

(00:31:24) “Let’s not regulate model development, but applications”

(00:32:08) - Sponsor: Netsuite

(00:36:00) GPT-4 is the most critical component for AGI

(00:38:00) AGI in 2-8 years

(00:39:26) Eureka moment from a general system

(00:48:00) AI research with China

(00:52:00) Does AI have subjective experience? The Mu response

This show is produced by Turpentine: a network of podcasts, newsletters, and more, covering technology, business, and culture — all from the perspective of industry insiders and experts. We’re launching new shows every week, and we’re looking for industry-leading sponsors — if you think that might be you and your company, email us at erik@turpentine.co.

Producer: Vivian Meng

Executive Producers: Amelia Salyers, and Erik Torenberg

Editor: Graham Bessellieu

Paul Christiano is the world’s leading AI safety researcher. My full episode with him is out!

We discuss:

- Does he regret inventing RLHF, and is alignment necessarily dual-use?

- Why he has relatively modest timelines (40% by 2040, 15% by 2030),

- What do we want post-AGI world to look like (do we want to keep gods enslaved forever)?

- Why he’s leading the push to get to labs develop responsible scaling policies, and what it would take to prevent an AI coup or bioweapon,

- His current research into a new proof system, and how this could solve alignment by explaining model's behavior

- and much more.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Open Philanthropy

Open Philanthropy is currently hiring for twenty-two different roles to reduce catastrophic risks from fast-moving advances in AI and biotechnology, including grantmaking, research, and operations.

For more information and to apply, please see the application: https://www.openphilanthropy.org/research/new-roles-on-our-gcr-team/

The deadline to apply is November 9th; make sure to check out those roles before they close.

Timestamps

(00:00:00) - What do we want post-AGI world to look like?

(00:24:25) - Timelines

(00:45:28) - Evolution vs gradient descent

(00:54:53) - Misalignment and takeover

(01:17:23) - Is alignment dual-use?

(01:31:38) - Responsible scaling policies

(01:58:25) - Paul’s alignment research

(02:35:01) - Will this revolutionize theoretical CS and math?

(02:46:11) - How Paul invented RLHF

(02:55:10) - Disagreements with Carl Shulman

(03:01:53) - Long TSMC but not NVIDIA

Tyler Cowen has long nurtured an obsession with music. It’s one of the few addictions Tyler believes is actually conducive to a fulfilling intellectual life.

In this bonus episode, an addendum to Rick’s conversation with Tyler, Rick sits with Tyler as he plays and talks through the music that moves him: from the outer bounds of the avant-garde to contemporary pop music and all points in between.

Here at the Mindscape Podcast, we are firmly pro-reason. But what does that mean, fundamentally and in practice? How did humanity come into the idea of not just doing things, but doing things for reasons? In this episode we talk with cognitive scientist Hugo Mercier about these issues. He is the co-author (with Dan Sperber) of The Enigma of Reason, about how the notion of reason came to be, and more recently author of Not Born Yesterday, about who we trust and what we believe. He argues that our main shortcoming is not being insufficiently skeptical of radical claims, but of being too skeptical of claims that don't fit our views.

Support Mindscape on Patreon.

Hugo Mercier received a Ph.D. in cognitive sciences from the École des Hautes Études en Sciences Sociales. He is currently a Permanent CNRS Research Scientist at the Institut Jean Nicod, Paris. Among his awards are the Prime d'excellence from the CNRS.

• Web site
• Google Scholar publications
• Amazon author page
• Twitter

Become a Spectator subscriber today to access this podcast without adverts. Go to spectator.co.uk/adfree to find out more.

For more Spectator podcasts, go to spectator.co.uk/podcasts

Contact us: podcast@spectator.co.uk

Hosted on Acast. See acast.com/privacy for more information.

Read the full transcript here.

My guest today is Samo Burja. Samo is the founder of consulting firm, Bismark Analysis, and has dedicated his life’s work to understanding why there has never been an immortal society. His research focuses on institutions, the founders behind them, how they rise and why they always fall in the end. As you’ll hear, Samo has an encyclopedic grasp of history and his work has led him to some fascinating theories about human progress, the nature of exceptional founders, and the future of different societies across the world. Please enjoy my conversation with Samo Burja.

Listen to Founders Podcast

Founders Episode 311: James Cameron

For the full show notes, transcript, and links to mentioned content, check out the episode page here.

-----

This episode is brought to you by Tegus. Tegus is the modern research platform for leading investors. Tired of running your own expert calls to get up to speed on a company? Tegus lets you ramp faster and find answers to critical questions more efficiently than any alternative method. The gold standard for research, the Tegus platform delivers unmatched access to timely, qualitative insights through the largest and most differentiated expert call transcript database. With over 60,000 transcripts spanning 22,000 public and private companies, investors can accelerate their fundamental research process by discovering highly-differentiated and reliable insights that can’t be found anywhere else in the market. As a listener, drive your next investment thesis forward with Tegus for free at tegus.co/patrick.

-----

Invest Like the Best is a property of Colossus, LLC. For more episodes of Invest Like the Best, visit joincolossus.com/episodes. 

Past guests include Tobi Lutke, Kevin Systrom, Mike Krieger, John Collison, Kat Cole, Marc Andreessen, Matthew Ball, Bill Gurley, Anu Hariharan, Ben Thompson, and many more.

Stay up to date on all our podcasts by signing up to Colossus Weekly, our quick dive every Sunday highlighting the top business and investing concepts from our podcasts and the best of what we read that week. Sign up here.

Follow us on Twitter: @patrick_oshag | @JoinColossus

Show Notes

(00:02:52) - (First question) - The core thesis behind the Great Founder Theory

(00:06:40) - Great ideas inevitably being discovered at some point in history 

(00:08:45) - The historic implications of a global adoption of the Great Founder Theory

(00:10:51) - The different possible directions of future trends

(00:17:08) - Distinctions between great founders versus live players  

(00:22:15) - Common misconceptions about what qualifies one as a great founder

(00:24:38) - Noteworthy great founders in the United States

(00:28:34) - Recurring observable traits and common themes of great founders  

(00:31:29) - Using caution when projecting a mythic lens onto great founders 

(00:37:53) - Social technology as the upstream effects of prior material technology

(00:43:32) - Whether or not institutions play a role in propagating the work of great founders   

(00:49:08) - The role of power and differences between owned and borrowed power  

(00:56:51) - Additional ideas that play an outsized role in shaping the world 

(01:01:09) - A differing worldview to his own that he finds interesting

(01:04:53) - Whether or not capital allocators can benefit from the Great Founder Theory 

(01:07:37) - The kindest thing anyone has ever done for him

Stephen Wolfram is a physicist, computer scientist and businessman. He is the founder and CEO of Wolfram Research, the creator of Mathematica and Wolfram Alpha, and the author of A New Kind of Science.

Full transcript available at: jnwpod.com.

See omnystudio.com/listener for privacy information.

Here is my conversation with Dario Amodei, CEO of Anthropic.

Dario is hilarious and has fascinating takes on what these models are doing, why they scale so well, and what it will take to align them.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Timestamps

(00:00:00) - Introduction

(00:01:00) - Scaling

(00:15:46) - Language

(00:22:58) - Economic Usefulness

(00:38:05) - Bioterrorism

(00:43:35) - Cybersecurity

(00:47:19) - Alignment & mechanistic interpretability

(00:57:43) - Does alignment research require scale?

(01:05:30) - Misuse vs misalignment

(01:09:06) - What if AI goes well?

(01:11:05) - China

(01:15:11) - How to think about alignment

(01:31:31) - Is modern security good enough?

(01:36:09) - Inefficiencies in training

(01:45:53) - Anthropic’s Long Term Benefit Trust

(01:51:18) - Is Claude conscious?

(01:56:14) - Keeping a low profile

Mustafa Suleyman, co-founder of DeepMind and now co-founder and CEO of Inflection AI, joins Sarah and Elad to discuss how his interests in counseling, conflict resolution, and intelligence led him to start an AI lab that pioneered deep reinforcement learning, lead applied AI and policy efforts at Google, and more recently found Inflection and launch Pi.

Mustafa offers insights on the changing structure of the web, the pressure Google faces in the age of AI personalization, predictions for model architectures, how to measure emotional intelligence in AIs, and the thinking behind Pi: the AI companion that knows you, is aligned to your interests, and provides companionship.

Sarah and Elad also discuss Mustafa’s upcoming book, The Coming Wave (release September 12, 2023), which examines the political ramifications of AI and digital biology revolutions.

No Priors is now on YouTube! Subscribe to the channel on YouTube and like this episode.

Show Links:

• Forbes - Startup From Reid Hoffman and Mustafa Suleyman Debuts ChatBot
• Inflection.ai
• Mustafa-Suleyman.ai

Sign up for new podcasts every week. Email feedback to show@no-priors.com

Follow us on Twitter: @NoPriorsPod | @Saranormous | @EladGil | @mustafasuleymn

Show Notes:

[00:06] - From Conflict Resolution to AI Pioneering

[10:36] - Defining Intelligence

[15:32] - DeepMind's Journey and Breakthroughs

[24:45] - The Future of Personal AI Companionship

[33:22] - AI and the Future of Personalized Content

[41:49] - The Launch of Pi

[51:12] - Mustafa’s New Book The Coming Wave

The second half of my 7 hour conversation with Carl Shulman is out!

My favorite part! And the one that had the biggest impact on my worldview.

Here, Carl lays out how an AI takeover might happen:

* AI can threaten mutually assured destruction from bioweapons,

* use cyber attacks to take over physical infrastructure,

* build mechanical armies,

* spread seed AIs we can never exterminate,

* offer tech and other advantages to collaborating countries, etc

Plus we talk about a whole bunch of weird and interesting topics which Carl has thought about:

* what is the far future best case scenario for humanity

* what it would look like to have AI make thousands of years of intellectual progress in a month

* how do we detect deception in superhuman models

* does space warfare favor defense or offense

* is a Malthusian state inevitable in the long run

* why markets haven't priced in explosive economic growth

* & much more

Carl also explains how he developed such a rigorous, thoughtful, and interdisciplinary model of the biggest problems in the world.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Catch part 1 here

Timestamps

(0:00:00) - Intro

(0:00:47) - AI takeover via cyber or bio

(0:32:27) - Can we coordinate against AI?

(0:53:49) - Human vs AI colonizers

(1:04:55) - Probability of AI takeover

(1:21:56) - Can we detect deception?

(1:47:25) - Using AI to solve coordination problems

(1:56:01) - Partial alignment

(2:11:41) - AI far future

(2:23:04) - Markets & other evidence

(2:33:26) - Day in the life of Carl Shulman

(2:47:05) - Space warfare, Malthusian long run, & other rapid fire

How worried about AI risk will we feel in the future, when we can see advanced machine intelligence up close? We should worry accordingly now. Text version here: https://joecarlsmith.com/2023/05/08/predictable-updating-about-ai-risk 

In terms of the depth and range of topics, this episode is the best I’ve done.

No part of my worldview is the same after talking with Carl Shulman. He's the most interesting intellectual you've never heard of.

We ended up talking for 8 hours, so I'm splitting this episode into 2 parts.

This part is about Carl’s model of an intelligence explosion, which integrates everything from:

* how fast algorithmic progress & hardware improvements in AI are happening,

* what primate evolution suggests about the scaling hypothesis,

* how soon before AIs could do large parts of AI research themselves, and whether there would be faster and faster doublings of AI researchers,

* how quickly robots produced from existing factories could take over the economy.

We also discuss the odds of a takeover based on whether the AI is aligned before the intelligence explosion happens, and Carl explains why he’s more optimistic than Eliezer.

The next part, which I’ll release next week, is about all the specific mechanisms of an AI takeover, plus a whole bunch of other galaxy brain stuff.

Maybe 3 people in the world have thought as rigorously as Carl about so many interesting topics. This was a huge pleasure.

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Timestamps

(00:00:00) - Intro

(00:01:32) - Intelligence Explosion

(00:18:03) - Can AIs do AI research?

(00:39:00) - Primate evolution

(01:03:30) - Forecasting AI progress

(01:34:20) - After human-level AGI

(02:08:39) - AI takeover scenarios

Peter Singer is one of the world's most influential living philosophers, whose ideas have motivated millions of people to change how they eat, how they give, and how they interact with each other and the natural world.

Peter joined Tyler to discuss whether utilitarianism is only tractable at the margin, how Peter thinks about the meat-eater problem, why he might side with aliens over humans, at what margins he would police nature, the utilitarian approach to secularism and abortion, what he's learned producing the Journal of Controversial Ideas, what he'd change about the current Effective Altruism movement, where Derek Parfit went wrong, to what extent we should respect the wishes of the dead, why professional philosophy is so boring, his advice on how to enjoy our lives, what he'll be doing after retiring from teaching, and more.

Read a full transcript enhanced with helpful links, or watch the full video.

Recorded May 25th, 2023

Other ways to connect

• Follow us on Twitter and Instagram
• Follow Tyler on Twitter
• Follow Peter on Twitter
• Email us: cowenconvos@mercatus.gmu.edu
• Learn more about Conversations with Tyler and other Mercatus Center podcasts here.

Photo credit: Katarzyna de Lazari-Radek

What is the nature of the universe? How do we make decisions correctly? What differentiates right actions from wrong ones?

Such fundamental questions have been the subject of philosophical and theological debates for millennia. But, as we all know, and surveys of expert opinion make clear, we are very far from agreement. So... with these most basic questions unresolved, what’s a species to do?

In today's episode, philosopher Joe Carlsmith — Senior Research Analyst at Open Philanthropy — makes the case that many current debates in philosophy ought to leave us confused and humbled. These are themes he discusses in his PhD thesis, A stranger priority? Topics at the outer reaches of effective altruism.

Links to learn more, summary and full transcript.

To help transmit the disorientation he thinks is appropriate, Joe presents three disconcerting theories — originating from him and his peers — that challenge humanity's self-assured understanding of the world.

The first idea is that we might be living in a computer simulation, because, in the classic formulation, if most civilisations go on to run many computer simulations of their past history, then most beings who perceive themselves as living in such a history must themselves be in computer simulations. Joe prefers a somewhat different way of making the point, but, having looked into it, he hasn't identified any particular rebuttal to this 'simulation argument.'

If true, it could revolutionise our comprehension of the universe and the way we ought to live...

Other two ideas cut for length — click here to read the full post.

These are just three particular instances of a much broader set of ideas that some have dubbed the "train to crazy town." Basically, if you commit to always take philosophy and arguments seriously, and try to act on them, it can lead to what seem like some pretty crazy and impractical places. So what should we do with this buffet of plausible-sounding but bewildering arguments?

Joe and Rob discuss to what extent this should prompt us to pay less attention to philosophy, and how we as individuals can cope psychologically with feeling out of our depth just trying to make the most basic sense of the world.

In today's challenging conversation, Joe and Rob discuss all of the above, as well as:

• What Joe doesn't like about the drowning child thought experiment
• An alternative thought experiment about helping a stranger that might better highlight our intrinsic desire to help others
• What Joe doesn't like about the expression “the train to crazy town”
• Whether Elon Musk should place a higher probability on living in a simulation than most other people
• Whether the deterministic twin prisoner’s dilemma, if fully appreciated, gives us an extra reason to keep promises
• To what extent learning to doubt our own judgement about difficult questions -- so-called “epistemic learned helplessness” -- is a good thing
• How strong the case is that advanced AI will engage in generalised power-seeking behaviour

Chapters:

• Rob’s intro (00:00:00)
• The interview begins (00:09:21)
• Downsides of the drowning child thought experiment (00:12:24)
• Making demanding moral values more resonant (00:24:56)
• The crazy train (00:36:48)
• Whether we’re living in a simulation (00:48:50)
• Reasons to doubt we’re living in a simulation, and practical implications if we are (00:57:02)
• Rob's explainer about anthropics (01:12:27)
• Back to the interview (01:19:53)
• Decision theory and affecting the past (01:23:33)
• Rob's explainer about decision theory (01:29:19)
• Back to the interview (01:39:55)
• Newcomb's problem (01:46:14)
• Practical implications of acausal decision theory (01:50:04)
• The hitchhiker in the desert (01:55:57)
• Acceptance within philosophy (02:01:22)
• Infinite ethics (02:04:35)
• Rob's explainer about the expanding spheres approach (02:17:05)
• Back to the interview (02:20:27)
• Infinite ethics and the utilitarian dream (02:27:42)
• Rob's explainer about epicycles (02:29:30)
• Back to the interview (02:31:26)
• What to do with all of these weird philosophical ideas (02:35:28)
• Welfare longtermism and wisdom longtermism (02:53:23)
• Epistemic learned helplessness (03:03:10)
• Power-seeking AI (03:12:41)
• Rob’s outro (03:25:45)

Producer: Keiran Harris

Audio mastering: Milo McGuire and Ben Cordell

Transcriptions: Katy Moore

Patreon: https://www.patreon.com/mlst

The ultimate goal of neuroscience is to learn how the human brain gives rise to human intelligence and what it means to be intelligent. Understanding how the brain works is considered one of humanity’s greatest challenges. 

Jeff Hawkins thinks that the reality we perceive is a kind of simulation, a hallucination, a confabulation. He thinks that our brains are a model reality based on thousands of information streams originating from the sensors in our body.  Critically - Hawkins doesn’t think there is just one model but rather; thousands. 

Jeff has just released his new book, A thousand brains: a new theory of intelligence. It’s an inspiring and well-written book and I hope after watching this show; you will be inspired to read it too. 

https://numenta.com/a-thousand-brains-by-jeff-hawkins/

https://numenta.com/blog/2019/01/16/the-thousand-brains-theory-of-intelligence/

Panel:

Dr. Keith Duggar https://twitter.com/DoctorDuggar

Connor Leahy https://twitter.com/npcollapse

Ben Garfinkel is a Research Fellow at the University of Oxford and Acting Director of the Centre for the Governance of AI.

In this episode we talk about:

• An overview of AI governance space, and disentangling concrete research questions that Ben would like to see more work on
• Seeing how existing arguments for the risks from transformative AI have held up and Ben’s personal motivations for working on global risks from AI
• GovAI’s own work and opportunities for listeners to get involved

Further reading and a transcript is available on our website: hearthisidea.com/episodes/garfinkel

If you have any feedback, you can get a free book for filling out our new feedback form. You can also get in touch through our website or on Twitter. Consider leaving us a review wherever you're listening to this — it's the best free way to support the show. Thanks for listening!

Demis Hassabis is the CEO and co-founder of DeepMind. Please support this podcast by checking out our sponsors:
– Mailgun: https://lexfridman.com/mailgun
– InsideTracker: https://insidetracker.com/lex to get 20% off
– Onnit: https://lexfridman.com/onnit to get up to 10% off
– Indeed: https://indeed.com/lex to get $75 credit
– Magic Spoon: https://magicspoon.com/lex and use code LEX to get $5 off

EPISODE LINKS:
Demis’s Twitter: https://twitter.com/demishassabis
DeepMind’s Twitter: https://twitter.com/DeepMind
DeepMind’s Instagram: https://instagram.com/deepmind
DeepMind’s Website: https://deepmind.com
Plasma control paper: https://nature.com/articles/s41586-021-04301-9
Quantum simulation paper: https://science.org/doi/10.1126/science.abj6511
The Emperor’s New Mind (book): https://amzn.to/3bx03lo
Life Ascending (book): https://amzn.to/3AhUP7z

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips

SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman

OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(07:17) – Turing Test
(14:43) – Video games
(36:18) – Simulation
(38:29) – Consciousness
(43:29) – AlphaFold
(57:09) – Solving intelligence
(1:09:28) – Open sourcing AlphaFold & MuJoCo
(1:19:34) – Nuclear fusion
(1:23:38) – Quantum simulation
(1:26:46) – Physics
(1:30:13) – Origin of life
(1:34:52) – Aliens
(1:42:59) – Intelligent life
(1:46:08) – Conscious AI
(1:59:23) – Power
(2:03:53) – Advice for young people
(2:11:59) – Meaning of life

It’s easy to dismiss alarming AI-related predictions when you don’t know where the numbers came from.

For example: what if we told you that within 15 years, it’s likely that we’ll see a 1,000x improvement in AI capabilities in a single year? And what if we then told you that those improvements would lead to explosive economic growth unlike anything humanity has seen before?

You might think, “Congratulations, you said a big number — but this kind of stuff seems crazy, so I’m going to keep scrolling through Twitter.”

But this 1,000x yearly improvement is a prediction based on *real economic models* created by today’s guest Tom Davidson, Senior Research Analyst at Open Philanthropy. By the end of the episode, you’ll either be able to point out specific flaws in his step-by-step reasoning, or have to at least consider the idea that the world is about to get — at a minimum — incredibly weird.

Links to learn more, summary and full transcript.

As a teaser, consider the following:

Developing artificial general intelligence (AGI) — AI that can do 100% of cognitive tasks at least as well as the best humans can — could very easily lead us to an unrecognisable world.

You might think having to train AI systems individually to do every conceivable cognitive task — one for diagnosing diseases, one for doing your taxes, one for teaching your kids, etc. — sounds implausible, or at least like it’ll take decades.

But Tom thinks we might not need to train AI to do every single job — we might just need to train it to do one: AI research.

And building AI capable of doing research and development might be a much easier task — especially given that the researchers training the AI are AI researchers themselves.

And once an AI system is as good at accelerating future AI progress as the best humans are today — and we can run billions of copies of it round the clock — it’s hard to make the case that we won’t achieve AGI very quickly.

To give you some perspective: 17 years ago we saw the launch of Twitter, the release of Al Gore's *An Inconvenient Truth*, and your first chance to play the Nintendo Wii.

Tom thinks that if we have AI that significantly accelerates AI R&D, then it’s hard to imagine not having AGI 17 years from now.

Wild.

Host Luisa Rodriguez gets Tom to walk us through his careful reports on the topic, and how he came up with these numbers, across a terrifying but fascinating three hours.

Luisa and Tom also discuss:

• How we might go from GPT-4 to AI disaster
• Tom’s journey from finding AI risk to be kind of scary to really scary
• Whether international cooperation or an anti-AI social movement can slow AI progress down
• Why it might take just a few years to go from pretty good AI to superhuman AI
• How quickly the number and quality of computer chips we’ve been using for AI have been increasing
• The pace of algorithmic progress
• What ants can teach us about AI
• And much more

Chapters:

• Rob’s intro (00:00:00)
• The interview begins (00:04:53)
• How we might go from GPT-4 to disaster (00:13:50)
• Explosive economic growth (00:24:15)
• Are there any limits for AI scientists? (00:33:17)
• This seems really crazy (00:44:16)
• How is this going to go for humanity? (00:50:49)
• Why AI won’t go the way of nuclear power (01:00:13)
• Can we definitely not come up with an international treaty? (01:05:24)
• How quickly we should expect AI to “take off” (01:08:41)
• Tom’s report on AI takeoff speeds (01:22:28)
• How quickly will we go from 20% to 100% of tasks being automated by AI systems? (01:28:34)
• What percent of cognitive tasks AI can currently perform (01:34:27)
• Compute (01:39:48)
• Using effective compute to predict AI takeoff speeds (01:48:01)
• How quickly effective compute might increase (02:00:59)
• How quickly chips and algorithms might improve (02:12:31)
• How to check whether large AI models have dangerous capabilities (02:21:22)
• Reasons AI takeoff might take longer (02:28:39)
• Why AI takeoff might be very fast (02:31:52)
• Fast AI takeoff speeds probably means shorter AI timelines (02:34:44)
• Going from human-level AI to superhuman AI (02:41:34)
• Going from AGI to AI deployment (02:46:59)
• Were these arguments ever far-fetched to Tom? (02:49:54)
• What ants can teach us about AI (02:52:45)
• Rob’s outro (03:00:32)

Producer: Keiran Harris
Audio mastering: Simon Monsour and Ben Cordell
Transcriptions: Katy Moore

Paul Christiano runs the Alignment Research Center, a non-profit research organization whose mission is to align future machine learning systems with human interests. Paul previously ran the language model alignment team at OpenAI, the creators of ChatGPT. 

Today, we’re hoping to explore the solution-landscape to the AI Alignment problem, and hoping Paul can guide us on that journey. 

------ ✨ DEBRIEF | Unpacking the episode:  https://www.bankless.com/debrief-paul-christiano    ------ ✨ COLLECTIBLES | Collect this episode:  https://collectibles.bankless.com/mint 

------ ✨ Always wanted to become a Token Analyst? Bankless Citizens get exclusive access to Token Hub. Join Them. https://bankless.cc/TokenHubRSS  

------ In today’s episode, Paul answers many questions, but the overarching ones are:  1) How BIG is the AI Alignment problem?  2) How HARD is the AI Alighment problem? 3) How SOLVABLE is the AI Alignment problem? 

Does humanity have a chance? Tune in to hear Paul’s thoughts. 

------ BANKLESS SPONSOR TOOLS: 

⚖️ ARBITRUM | SCALING ETHEREUM https://bankless.cc/Arbitrum 

🐙KRAKEN | MOST-TRUSTED CRYPTO EXCHANGE https://bankless.cc/kraken 

🦄UNISWAP | ON-CHAIN MARKETPLACE https://bankless.cc/uniswap 

👻 PHANTOM | FRIENDLY MULTICHAIN WALLET https://bankless.cc/phantom-waitlist 

🦊METAMASK LEARN | HELPFUL WEB3 RESOURCE https://bankless.cc/MetaMask 

------ Topics Covered

0:00 Intro 9:20 Percentage Likelihood of Death by AI 11:24 Timing  19:15 Chimps to Human Jump 21:55 Thoughts on ChatGPT 27:51 LLMs & AGI 32:49 Time to React? 38:29 AI Takeover  41:51 AI Agency  49:35 Loopholes  51:14 Training AIs to Be Honest  58:00 Psychology  59:36 How Solvable Is the AI Alignment Problem? 1:03:48 The Technical Solutions (Scalable Oversight)  1:16:14 Training AIs to be Bad?!  1:18:22 More Solutions 1:21:36 Stabby AIs  1:26:03 Public vs. Private (Lab) AIs 1:28:31 Inside Neural Nets 1:32:11 4th Solution  1:35:00 Manpower & Funding  1:38:15 Pause AI? 1:43:29 Resources & Education on AI Safety  1:46:13 Talent   1:49:00 Paul’s Day Job 1:50:15 Nobel Prize  1:52:35 Treating AIs with Respect  1:53:41 Uptopia Scenario  1:55:50 Closing & Disclaimers 

------ Resources:

Alignment Research Center https://www.alignment.org/ 

Paul Christiano’s Website https://paulfchristiano.com/ai/ 

----- Not financial or tax advice. This channel is strictly educational and is not investment advice or a solicitation to buy or sell any assets or to make any financial decisions. This video is not tax advice. Talk to your accountant. Do your own research.

Disclosure. From time-to-time I may add links in this newsletter to products I use. I may receive commission if you make a purchase through one of these links. Additionally, the Bankless writers hold crypto assets. See our investment disclosures here: https://www.bankless.com/disclosures 

I went over to the OpenAI offices in San Fransisco to ask the Chief Scientist and cofounder of OpenAI, Ilya Sutskever, about:

* time to AGI

* leaks and spies

* what's after generative models

* post AGI futures

* working with Microsoft and competing with Google

* difficulty of aligning superhuman AI

Watch on YouTube. Listen on Apple Podcasts, Spotify, or any other podcast platform. Read the full transcript here. Follow me on Twitter for updates on future episodes.

Timestamps

(00:00) - Time to AGI

(05:57) - What’s after generative models?

(10:57) - Data, models, and research

(15:27) - Alignment

(20:53) - Post AGI Future

(26:56) - New ideas are overrated

(36:22) - Is progress inevitable?

(41:27) - Future Breakthroughs

Historian Tom Holland joined Tyler to discuss in what ways his Christianity is influenced by Lord Byron, how the Book of Revelation precipitated a revolutionary tradition, which book of the Bible is most foundational for Western liberalism, the political differences between Paul and Jesus, why America is more pro-technology than Europe, why Herodotus is his favorite writer, why the Greeks and Persians didn't industrialize despite having advanced technology, how he feels about devolution in the United Kingdom and the potential of Irish unification, what existential problem the Church of England faces, how the music of Ennio Morricone helps him write for a popular audience, why Jurassic Park is his favorite movie, and more.

Read a full transcript enhanced with helpful links, or watch the full video.

Recorded February 1st, 2023

Other ways to connect

• Follow us on Twitter and Instagram
• Follow Tyler on Twitter
• Follow Tom on Twitter
• Email us: cowenconvos@mercatus.gmu.edu
• Learn more about Conversations with Tyler and other Mercatus Center podcasts here.

Photo credit: Sadie Holland

Should we think of morality in terms of objective truth or social consensus?

Looking for a link we mentioned? It's here: https://linktr.ee/philosophyforourtimes

Once the fashion of a postmodern age, moral relativism has always had its detractors, many of them religious. But now a new breed of atheist celebrity thinkers, from Sam Harris to Peter Singer, are making claims for the existence of absolute moral truths. Critics argue that like authoritarian moralists of the past, they use so-called 'objective' morality to shore up to their own prejudices and silence dissent.

Firebrand philosopher Slavoj Žižek, bestselling author of Zed Joanna Kavenna, and philosopher and author of Truth Simon Blackburn debate objective morality in a postmodern age. Hosted by Professor and Chair of Jurisprudence at the University of Oxford, Ruth Chang.

There are thousands of big ideas to discover at IAI.tv – videos, articles, and courses waiting for you to explore. Find out more: https://iai.tv/podcast-offers?utm_source=podcast&utm_medium=shownotes&utm_campaign=[iai-tv-episode-title]

See Privacy Policy at https://art19.com/privacy and California Privacy Notice at https://art19.com/privacy#do-not-sell-my-info.

Yasheng Huang has written two of Tyler's favorite books on China: Capitalism with Chinese Characteristics, which contrasts an entrepreneurial rural China and a state-controlled urban China, and The Rise and Fall of the EAST, which argues that Keju—China's civil service exam system—played a key role in the growth and expanding power of the Chinese state.

Yasheng joined Tyler to discuss China's lackluster technological innovation, why declining foreign investment is more of a concern than a declining population, why Chinese literacy stagnated in the 19th century, how he believes the imperial exam system deprived China of a thriving civil society, why Chinese succession has been so stable, why the Six Dynasties is his favorite period in Chinese history, why there were so few female emperors, why Chinese and Chinese Americans have done less well becoming top CEOs of American companies compared to Indians and Indian Americans, where he'd send someone on a two week trip to China, what he learned from János Kornai, and more.

Read a full transcript enhanced with helpful links, or watch the full video.

Recorded January 17th, 2023

Other ways to connect

• Follow us on Twitter and Instagram
• Follow Tyler on Twitter
• Follow Yasheng on Twitter
• Email us: cowenconvos@mercatus.gmu.edu
• Learn more about Conversations with Tyler and other Mercatus Center podcasts here.

Photo credit: MIT Sloan School

Anonymous founders of the Effective Accelerationist (e/acc) movement @Bayeslord and Beff Jezoz (@BasedBeff) join Erik Torenberg, Dan Romero, and Nathan Labenz to debate views on AI safety.

We're hiring across the board at Turpentine and for Erik's personal team on other projects he's incubating. He's hiring a Chief of Staff, EA, Head of Special Projects, Investment Associate, and more. For a list of JDs, check out: eriktorenberg.com.

RECOMMENDED PODCAST:

The HR industry is at a crossroads. What will it take to construct the next generation of incredible businesses – and where can people leaders have the most business impact? Hosts Nolan Church and Kelli Dragovich have been through it all, the highs and the lows – IPOs, layoffs, executive turnover, board meetings, culture changes, and more. With a lineup of industry vets and experts, Nolan and Kelli break down the nitty-gritty details, trade offs, and dynamics of constructing high performing companies. Through unfiltered conversations that can only happen between seasoned practitioners, Kelli and Nolan dive deep into the kind of leadership-level strategy that often happens behind closed doors. Check out the first episode with the architect of Netflix’s culture deck Patty McCord.

https://link.chtbl.com/hrheretics

TIMESTAMPS:

(00:00) Episode preview

(03:00) Intro to effective accelerationism

(08:00) Differences between effective accelerationism and effective altruism

(23:00) Effective accelerationism is bottoms-up

(42:00) Transhumanism

(46:00) "Equanimity amidst the singularity"

(48:30) Why AI safety is the wrong frame

(56:00) Pushing back against effective accelerationism

(01:06:00) The case for AI safety

(01:24:00) Upgrading civilizational infrastructure

(01:33:00) Effective accelerationism is anti-fragile

(01:39:00) Will we botch AI like we botched nuclear?

(01:46:00) Hidden costs of emphasizing downsides

(2:00:00) Are we in the same position as neanderthals, before humans?

(2:09:00) "Doomerism has an unpriced opportunity cost of upside"

SPONSORS: Beehiiv | Shopify | Secureframe

Head to Beehiiv, the newsletter platform built for growth, to power your own. Connect with premium brands, scale your audience, and deliver a beautiful UX that stands out in an inbox. 🐝 to https://Beehiiv.com and use code “MOZ” for 20% off your first three months

-Shopify: https://shopify.com/momentofzen for a $1/month trial period

Shopify is the global commerce platform that helps you sell at every stage of your business. Shopify powers 10% of all e-commerce in the US. And Shopify’s the global force behind Allbirds, Rothy’s, and Brooklinen, and 1,000,000s of other entrepreneurs across 175 countries. From their all-in-one e-commerce platform, to their in-person POS system – wherever and whatever you’re selling, Shopify’s got you covered. With free Shopify Magic, sell more with less effort by whipping up captivating content that converts – from blog posts to product descriptions using AI. Sign up for $1/month trial period: https://shopify.com/momentofzen

-Secureframe (www.secureframe.com)

Secureframe is the leading all-in-one platform for security and privacy compliance. Get SOC-2 audit ready in weeks, not months. I believe in Secureframe so much that I invested in it, and I recommend it to all my portfolio companies. Sign up for a free demo and mention MOMENT OF ZEN during your demo to get 20% off your first year.

Robin Dale Hanson is an associate professor of economics at George Mason University and a research associate at the Future of Humanity Institute of Oxford University. He is known as an expert on idea futures and markets, and he was involved in the creation of the Foresight Exchange and DARPA’s FutureMAP project. He invented market scoring rules like LMSR (Logarithmic Market Scoring Rule) used by prediction markets such as Consensus Point (where Hanson is Chief Scientist), and has conducted research on signaling.

When attempting to affect change in the world, you will inevitably run up against concepts that others consider sacred. This creates a very tough barrier to change, especially if you are trying to change something like democracy, family, or religion. The essence of sacred is the bond shared between those who consider a particular idea sacred. It’s difficult to see things the same when seeing them in high resolution, so sacred things tend to be seen in abstract detail, even when looking at them up close, to allow for concensus.


Session Summary: Robin Hanson, George Mason University | Deflecting The Sacred - Foresight Institute

The Foresight Institute is a research organization and non-profit that supports the beneficial development of high-impact technologies. Since our founding in 1987 on a vision of guiding powerful technologies, we have continued to evolve into a many-armed organization that focuses on several fields of science and technology that are too ambitious for legacy institutions to support.

Allison Duettmann is the president and CEO of Foresight Institute. She directs the Intelligent Cooperation, Molecular Machines, Biotech & Health Extension, Neurotech, and Space Programs, Fellowships, Prizes, and Tech Trees, and shares this work with the public. She founded Existentialhope.com, co-edited Superintelligence: Coordination & Strategy, co-authored Gaming the Future, and co-initiated The Longevity Prize. 

Apply to Foresight’s virtual salons and in person workshops here!

We are entirely funded by your donations. If you enjoy what we do please consider donating through our donation page.

Visit our website for more content, or join us here:

• Twitter
• Facebook
• LinkedIn

Every word ever spoken on this podcast is now AI-searchable using Fathom.fm, a search engine for podcasts. 

Hosted on Acast. See acast.com/privacy for more information.

Chris Miller is an Associate Professor of International History at Tufts University and author of the book “Chip War: The Fight for the World's Most Critical Technology” (the Financial Times Business Book of the Year). He is also a Visiting Fellow at the American Enterprise Institute, and Eurasia Director at the Foreign Policy Research Institute.

Over the next few episodes we will be exploring the potential for catastrophe cause by advanced artificial intelligence. But before we look ahead, we wanted to give a primer on where we are today: on the history and trends behind the development of AI so far. In this episode, we discuss:

• How semiconductors have historically been related to US military strategy
• How the Taiwanese company TSMC became such an important player in this space — while other countries’ attempts have failed
• What the CHIPS Act signals about attitudes to compute governance in the decade ahead

Further reading is available on our website: hearthisidea.com/episodes/miller

If you have any feedback, you can get a free book for filling out our new feedback form. You can also get in touch through our website or on Twitter. Consider leaving us a review wherever you're listening to this — it's the best free way to support the show. Thanks for listening!