Summary

In this episode of The Test Set, Wes McKinney and co-hosts interview Alenka Frim, an Apache Arrow committer and yoga teacher, about open source community health, Arrow's growing role in the data ecosystem, and how AI is reshaping software engineering. McKinney reflects on Arrow's first decade, describing it as 'AI-resistant' technology that becomes more important as AI advances. He shares his experience using coding agents daily, noting they make him more productive than ever while the human judgment layer remains essential. The conversation explores how pedagogy and software careers are being transformed by AI tools, with McKinney predicting an order-of-magnitude increase in shipped software products enabled by dramatically lower development costs.

Key Insight

AI coding agents are a force multiplier that will dramatically increase the volume and quality of shipped software, but human judgment in system design and critical thinking remains irreplaceable -- making foundational technologies like Arrow more important, not less.

Spicy Quotes (click to share)

• 3

  I think the big signs are whether the community is adding people -- like people are coming and staying -- and also are members of the community comfortable raising concerns about what's not working and having constructive dialogue about how to fix it without blaming or pointing fingers.

• 5

  Maybe the field of software engineering will shrink, but I think maybe it will stay the same size and we'll all be building 10 to 100 times more software.

• 5

  I described the Arrow project as being what I felt like an AI-resistant project or AI-resistant technology. And I wanted to see what you thought about that, both in the sense that AI doesn't make Arrow irrelevant -- in some ways it makes it even more important, as a grand unifier.

• 5

  Yes, they're tremendously useful. I'm building more software, more productive than I ever have been in my life, which is insane. But at the same time, the human element of judgment feels more essential than ever.

• 6

  When beautifying the code base has one thousandth of the cost that it used to have, of course the code base should be beautiful. Of course it should be well-structured and easy to reason about.

• 5

  I think the result is I expect the number of unique shipped software products to be a minimum an order of magnitude what it was in the past, if not two orders of magnitude.

• 3

  I approached software engineering from a very mathematician-centric viewpoint of thinking about things in terms of stacks of lemmas and theorems and trying to prove things by layering things on top of each other.

Tone

reflective, optimistic, conversational