A Learning System Made of Learning Parts

My friend Jessica Kerr, welcome to Still Burning.

Thank you, Kent. It's the fires making me feel really welcome right now.

Yeah, it's the fire equivalent of that. Great to have you on the podcast. We've been friends a long time. I wanted to have you on since we had the idea for the podcast.

I finally got out to the cool house.

You got to the cool house — to the cool side of the country. A talk to geeks who still care and are still doing something about it. And you fit that definition for sure. I kind of watched you from afar for a while and you kept coming up with these powerful metaphors that made me think differently about my situation. Like symmathesy. And then we became friends and worked together on systems thinking — developers during COVID.

Yeah. COVID was an interesting, challenging time. And now we're in a completely different challenging time.

Yeah. In these challenging times, we need creativity. And we need people who are willing to dig in technically as the rulebook is changed — for everybody.

Yes. But the meaning of "technical" — like the kind of technical digging in we need to do is at another level.

Yeah. Things that I wrote whole books about just don't matter anymore. Like carefully crafting a function. This is no leverage. I mean, right — it still feels good. But you made the analogy earlier of woodworking.

Right. Right now, if you want to build a bookshelf or a table — that's great, that's a craft. But there's also IKEA.

That's right. There's cheaper ways to do it, and you could choose to do it. When did you first know that you were a geek?

Well, in high school I used to program my graphing calculator because I was bored in class.

Sure, who didn't?

Well, many, many people around me did not. Also, I wish I knew how to sing, but I quit choir in seventh grade because the teacher wouldn't let me read my Star Trek book while other people were singing. Okay, but she would let her favorites — the cheerleaders — talk amongst themselves. So there's like a different value system, right? I'm like, what is important? And what forms of mental stimulation are worthwhile?

Yeah, and socially acceptable. When's the first time you felt comfortable being a geek?

Oh, Missouri Scholars Academy. Between sophomore and junior year of high school, Missouri has a thing where you go to Mizzou and they gather close to a thousand people from across the state who are like top of their class academically. And to be around people who valued each other for being smart — where we can have geeky conversations and talk about philosophy or technology — that was the first time that I felt desired. My sister went to MSA, and then just a few years ago my kid went to MSA. I was so happy.

I contrast this with the first time I came to San Francisco, which was maybe 12 years ago, and the billboards are targeted at me — "Developer" — and I was like, what? That's too much, man. I want my value system to be shared by my friends, but not the whole world.

There's something I want to get back to in the IKEA analogy related to the billboards. Developers have gone through a period recently of being incredibly valued. When I got out of college in 1999 — this beautiful year to get out of college — it was so easy to get hired as a programmer. I could get a job in any city I wanted, whenever I wanted. And now this year — maybe starting last year — we feel threatened. Because the IKEA of software development has arrived.

Yes. It feels like it's arrived because part of our job has been taken out of our hands.

Yeah, part of our job can be easily replaced with something — with a computer. With something that anyone can access in their browser. But basically they can go to IKEA — and IKEA here is ChatGPT or Claude — and they can get the software that they need. And it'll be good enough. No, they don't have to analyze it. They just have to look at the output and say, is it good enough? Is it what I want? You do have to know what you want — that's a trick. But I don't think you have to understand the code to get useful software now.

Okay, we're programmers and our job has just gone away — because anybody can do it. I think that's a message a lot of people hear, but I disagree.

Our job is completely different. Yes. And somebody's got to assemble the IKEA furniture. And my mom actually didn't know how to make her own website — I just did it for her in a few hours instead of a week. Our job is completely different because the parts — the code crafting and the understanding software through code — that part has been commoditized. But that's not all of our job. It's some people's favorite parts. It's not my favorite part. So I'm okay.

I think it emphasizes once again that relating to other people is a more highly levered skill now, even more than it was before.

Yes, because understanding what to build is the hardest part. My pinned tweet for many years is: "I don't want to build software so much as build understanding and express it in software." And I feel like I can do that more than ever now. But I have to do it at a different layer. It's not just writing the code. I need to express that understanding in some sort of verification layer.

Yes, and that's where the analytical skills come in — to say there aren't enough tests here.

Yeah, because there's like — an accountant building software for themselves — if it works for them, it works for them, and they understand it well enough. But as soon as you're building software for other people, and stuff is going to scale and be depended upon by other software that's going to be used by people who don't understand it in and out — then: how do we know it works? That requires all the analytical skill we ever had.

Yeah. And some more. Because the tools are set up to give us the finger guns. You asked for this feature? Gotcha.

And that's where the creativity comes in now. The creativity isn't — what shall I name this variable, how shall I refactor this function to make it look pretty? The creativity is in: how do I design a system that produces code that works and I can say why I know it works? I find that really satisfying.

Of the vocabulary you introduced — some of which took off — explain what symmathesy means.

Yes, let's talk about symmathesy. It's a word coined by Nora Bateson, who's an anthropologist. It means a learning system made of learning parts. This contrasts with the word "system" by itself — which often we think of as something mechanical. A machine or a program — you can understand all the parts and all the relationships, at least theoretically. In a living system, those parts are constantly changing and the relationships between them are constantly changing. Nora sees those as flows of learning. Within our team, we're always learning from each other and changing each other. And the software participates in that — the people learn from the software and then they influence the software. The whole system is learning and growing together. You cannot model that. Everything biological, everything human and sociological is a symmathesy.

I already had people and software in the system. Now agents are a completely third kind of node in this system. Because they learn completely differently. Unlike code, you don't program into them to do this thing differently. Unlike people, they don't remember things and have their attitude shifted by everything they've ever seen. But in the course of working with agents, we totally influence them with their context — the memories they write down, the code and the way it looks totally changes how they learn.

And their learning is on a weirdly short timescale. Suddenly documentation matters tremendously more. Because people really only read the docs for a project when they start. Maybe that happens twice a year. But now the agent needs to start from scratch in that project and read that documentation every 15 minutes or an hour. So suddenly that documentation is getting used regularly, and that's why we can keep it up to date.

Because it's actually worthwhile.

It's worthwhile, and we notice when it's wrong. Because the agent gets dumb. And when it gets dumb, you look at the wider system. So we are stewards of its environment. And when I'm asking it, "what made you think that?" — usually it can tell me. Well, I read this document over here and it says this. And I'm like, okay, let's spin off a sub-agent to update that document. Often I'll just slash-clear, update the document, start over. That's the other thing — agents can forget, and people can't.

Yeah, and I think we're not using that nearly enough. This idea that coding is free.

Now you know, if parenting worked like this — if you're gonna have an interaction with a kid and be like, fuck, that didn't work, slash-clear — reload my game from before you came in the door. Let me start over. Step one: be happy to see you. Step two: ask you to pick that stuff up.

At Olicon, Charity had a bunch of stickers about the human in the loop — "it's my loop." Right, I'm in charge here. And Corey Quinn said sometimes the loop could become a noose. Which is true — if you start caring at the wrong level, if you start trying to understand everything it did, that's almost never useful. The other time is when it's like, oh, I counted this, you go test it for me — no, that's the noose. Don't stick your head in that. Tell that thing to test it itself. And if it doesn't know how to test it itself, then it's time to have it write that Playwright script or whatever.

By the time you got to programming, the playbook for how to do good programming was fairly established. Is that fair?

It was probably established somewhere. But that doesn't mean it was in the particular culture that I was programming in. We had established procedures — we would write test code to exercise our functions and then we would delete it. Because CI was not a thing. We had version control, but if you locked the file no one else could change it. We had internal libraries. No open source. We had strong coding standards so you could make your code look like everyone else's code. Later we switched to Java and guess what our Java looked like.

Yeah, exactly.

I used to tell people: what programming language should you use? Well, whichever one you already know — that's the one you're going to be best at. But now it's a little different. Which one is the agent good at? Has a lot to do with it. But what I know and can read when I need to is also valuable. Understanding a language and understanding a runtime are two different things. Because if I'm actually going to operate this — if I'm going to put this capability out into the world — I need it to do more than just meet my requirements. I need it to run smoothly.

So my learning has been accelerated so much. Because I can be coding in Rust, never seen Rust before, see a weird symbol, and ask what does this mean — and I'm asking that question in a concrete situation, at a time when I care about it.

And unlike Googling, you're not having to translate some generic example to your situation. A younger programmer at Honeycomb — Ruthie — was telling me that they love this because they're able to learn so much faster. Their learning used to be limited by the people available to answer their questions. Now they can go to Claude for all the easy questions, all the generic questions, and then go to the human with the specific "why do we do it this way?" question after they've got all the background. And the answers come a lot faster because when they'd ask a person, they'd be like, I don't quite get it, can you explain it another way? A person has to think about it. Claude is all over that.

Yeah, loves doing it.

Our flow of learning is highly accelerated in that way — if we use it right. But it's tempting to skip that and just assemble the furniture and start using it.

I got that feature done. You want the next feature?

That's the thing — if we focus on doing the same stuff faster, then we are missing the step function. We get a step function in productivity by an old definition — just outputting features. Outputting features is easy. Understanding features is harder. Understanding the language and the runtime and what do we need to care about? We have so much opportunity to learn faster, but you gotta take the step back and use the tool in a different way to maximize learning instead of skipping it.

Especially when we go fiddle with our hooks or whatever to inject a different workflow and these reminders — to not neglect learning. It feels like play. And in many cultures, play doesn't feel like productivity. But I promise: that feeling of play is a sign that you're learning something. And we need a lot of that right now because we do not know what we're doing.

When in the 3X kind of world, when you're in an Explorer phase — farting around is a positive good. If you're in a highly tuned, very precise, highly scaled system and somebody wants to come along and say "let me fiddle around with the deployment" — no. But when you're in the Explorer world, you've got nothing to lose. And learning — trust that little voice that goes, "hey, I wonder."

That's a good signal. And often it's better to try it. Especially when what we're wondering about is how to work with the agents themselves. Because that's where we get to play and that's where we get to be in charge and do stuff we never could before.

Do you get the same kind of pushback — "I don't want to hear about all this AI stuff"?

My kids will have none of it. I get it from software developers too. And I'm like, okay, well, you don't have to listen to my podcast. I'll let you know if we ever put out an episode that's not about AI. But this is what's happening right now, and it is where we need to learn, and it is terrifying and exciting — and I choose to be excited. I don't make my kids use it. They're going to need to learn, but right now they're like, it's terrible for artists.

How did that happen? We're supposed to be the fuddy-daddies. We're supposed to be digging our heels in, and the kids are supposed to be doing cool new stuff we can't understand.

There's so much corporate push and AI shoved in their faces all the time. So no — they are rebelling by rejecting it. We are conforming to the capitalist bro-ligar-key by leaning into it. I know. It's so exciting. I can still do stuff I couldn't before. I'm not going to reject something so amazing just because it's also the center of a gigantic bubble. No, I'm in there. I'm participating. And that gets us back to symmathesy — because to understand and improve a symmathesy, you have to get in it.

I define symmathesist as someone who participates in a symmathesy and works to improve it consciously. That's what I'm doing as a developer — as I improve my code, as I level up my team and myself, as I build relationships within the team and outside, as I improve the observability so I can learn more from my running software. And even more now that I'm working with agents — learning how to work with them better, learning how to get them the context they need, learning how to get them to build the verification systems that we all need to build something useful. I am a symmathesist.

Hmm. And that's like — there's definitely a set of skills there. There's a two-level kind of thinking where you're in the system and also thinking about the system.

There's a certain kind of f*** to give. And that gets back to the premise of Still Burning — geeks who still care and are still doing something about it.

This is a tough one. I get the ethical objections in terms of energy usage, and I'm concerned about — as an author — a little piece of me is out there in all this training data.

I wanted them to read all my stuff, please. I would like the entire world to think more like me. Please train on my output. You don't get paid for that, but everybody knows that writing a book is a labor of love and a service — and that's why we applaud people for it. Because you don't get paid enough. Now I'm not worried about the plagiarism. I totally want my output to be part of what the world is.

I was in the middle of writing a book about software design and augmented development and got some bad news and sat back and thought — do I really want to be doing this? What is my incentive to write this book at all if it's gonna go into the training data for something that's gonna make somebody else a whole bunch of money? The principles won't be out of date. But why am I busting my hump writing a book? I don't like the feeling that other people will profit economically and I won't.

I'm over that. I just don't have that fairness bug. As long as I'm doing okay and writing and contributing to the community gets me better jobs — good enough. I just need enough. And it doesn't hurt my feelings for someone else to profit. That's their problem. The idea that the world is fair or ever should be — it causes a lot of suffering. Should we try to make our little piece of the world more fair to the people in it? Yes. Should we expect the world to be fair? I don't want to be miserable.

Ours is the work, not the fruits. If you asked me to vote — if I had some influence on whether the money went to the artists or the record companies, I would vote for the money to go to the artists. But I'm not going to not participate in the system just because it's not fair enough.

Last question, Jess. What about this whole situation scares you enough to wake you up at night?

I have two kids to put through college and I can't look around and say I'm gonna have a job available to me in any city that makes the kind of money I want to make. The jobs are different now, the market is different. I do think that demand for software will increase compared to what it is today — drastically. We will make so much more software because it's cheaper. But that's going to take a while. And right now, for the software that people do demand and pay for — that can be done by ten times fewer people in the right situations. Somebody's got to design the IKEA furniture. Somebody's going to write those instructions and decide how the shelf should fit together and make sure it does. And so I'm gonna get even better at that part of the job. There's plenty of work. But yeah, the market's not what it was, and that does keep me up.

Thank you so much for being a guest on Still Burning.

Thank you for continuing to care.

I do. It's a little too much sometimes.

I'll take it over not caring. Not wanting anything is one of the worst things. Yeah.