What Do We Mean by Humane Tech? - [ Center for Humane Technology ] | Substack - News Bunkers

Aza Raskin: Hey everyone. It’s Aza Raskin and welcome to Your Undivided Attention. Today, we’re going to be doing something a little different, a kind of looking back to look forward. And to do that, we’ve invited the co-founder for Center for Humane Technology, along with me and Tristan, Randy Fernando for this special episode. And what we’re going to be talking about is really what we talk about when we talk about humane technology. We’re going to be exploring seven principles of humane tech and the myths that they bust. So hey Randy, welcome to the show.
Randy Fernando: Glad to be here.
Aza Raskin: All right, Randy. So just really quickly, before we dive in, I think it’s important that people hear just a little bit about your background. Why should we be listening to you in this moment?
Randy Fernando: I started my career at NVIDIA, which was the best place for me given my background in computer science and computer graphics. I was a product manager for a bunch of different software products. One of the interesting things I did was I also co-authored some books and one of them was the first book on the first hardware shading language for the first programmable GPU, which was sort of like early days in this path that has taken us to where we are today. I was also on the founding board of directors of the NVIDIA Foundation.
And then, I kind of got interested in the nonprofit sector. And so, I ran a nonprofit called Mindful Schools that trained kids in how to pay attention, how to manage their emotions, how to cultivate kindness. And we trained nearly a million kids and 20,000 teachers while I was there. And since then, we’ve been doing this work at CHT with you and Tristan for almost 10 years.
Aza Raskin: And I just want everyone to know because many people of course know Tristan’s name and my name, but they don’t know Randy as much your name, but every major moment of our journey through social media into AI, you’ve been there behind the scenes at the social dilemma for the AI doc, just you’re always there, always supporting. And I just want everyone listening to know that. And Randy has been working on codifying a set of humane technology principles that we’ve been using since the very beginning. The name Humane comes, of course, from my father, created the Macintosh Project that he’s called What It Is To Be Humane, to be responsive to human needs and considerate of human frailties.
And why this is so important and why now is that when we used to look at social media, people would talk about all these separate problems. They talk about, there’s addiction over here and polarization over here and misinformation in that corner, over-sexualization over here, hyperpartisanship over here. And if you see the world that way as a collection of disconnected problems, then you’re never going to solve any of them because different people are going to be working on different areas. The diagnosis is wrong. It’s sort of like playing the world’s hardest game of whack-a-mole.
And instead, you need to get to the root of the problem. And if you can name the root correctly, then you address that one thing and addresses all the other things.
Randy Fernando: That’s right. The same patterns keep repeating. And the idea of this project was to say, “Wait, there’s a way of thinking. There’s a way of looking at the problem. There’s a way of looking at the diagnosis behind the problem.” And that same way of thinking guides you to the right answers. If you are building solutions inside the current system, but also if you are imagining, hey, what would a good system look like? If we really wanted to do things right, these same principles will help you do all of that. So that’s what we want to walk you through today.
Aza Raskin: And so, we’re going to start by taking you to our very first episode all the way back in 2019 with our very first guest, Natasha Dow Schüll, who is a cultural anthropologist who conducted years of research in Las Vegas casinos.
Natasha Dow Schüll: Really, when I started this project, it was Las Vegas and casinos everywhere, we’re in this real shift in design logic. Some of us still think of Vegas as being loud, jangly, bright, neon, flashing, and it used to be designed that way. Make it as loud as you can. People need to hear the coins clanking and put a strobe on their face. No, you want people to sit down because your new profit logic is called time on device. And to increase time on device, this is a sort of ergonomic operation where you have to worry about fatigue. It’s not worker fatigue, it’s consumer fatigue.
Aza Raskin: Right. We can’t have you fatigued there. We have to make sure that you’re staying.
Natasha Dow Schüll: Right. So you are measuring that light doesn’t bounce directly at people from interior surfaces because that will bring them to awareness and tax their senses. You don’t want sound to bounce off walls and come and again, make you feel depleted. So people will even spend time constructing these protective sound cones. So they’re invisible, but they’re there, where your ears and your eyes, the sort of audio visuals are directing you to your own little theater and trying to buffer anything from the outside that could interrupt you.
Aza Raskin: Does that sound familiar? Well, it should because that’s exactly how pretty much every app on your smartphone is designed, not necessarily to maximize for wellness or wellbeing or thriving, but for time on site. And it’s interesting because that means for a whole range of technologies from slot machines to iPhones to AI chatbots, the tech is all different, but somehow that core principle, this sort of broken ideology behind them, they’re all the same.
Randy Fernando: And once we understand what’s driving all of the broken ideology, then we can figure out what the opposites are and find a set of principles that actually covers a very large number of cases with just a small number of principles. And that’s really helpful because then in your mind you don’t have to remember so much and you can generalize that understanding to lots of situations that you come across in your life and in the technology around us.
Aza Raskin: So in this episode, Randy and I are going to walk through CHT’s principles of humane technology, seven in total. And for each one, we’re going to bring in some real world examples from the podcast and beyond, to illustrate it and talk about why it matters.
Randy Fernando: And of course, we don’t have a lot of time today, so we’re covering just a few aspects of each principle. And if you want to go deeper, you can go to humanetech.com/course and sign up there.
Aza Raskin: So let’s get to Principle One: Approach Solving Technology’s Problems Through a Complex Systems Lens. And that’s sort of an abstract thing to say, but really this is about that there’s a problem in the way that we solve problems. Randy?
Randy Fernando: One of the big things that we miss a lot when we are dealing with technology is we don’t look at the whole system. We look at just one small piece of it and that means we don’t correctly understand what its effects are going to be and we don’t correctly understand how to fix the problems that we find.
Aza Raskin: And this is really where most of the harmful aspects of today’s technology comes from. And we can go back to our fourth episode to get a good example of this. We talked to Guillaume Chaslot, a former software engineer at YouTube about what happened after YouTube decided to optimize for just one single metric, which was engagement.
Aza Raskin: Guillaume observed a subtle but unmistakable tilt in the recommendations. It seemed to favor extreme content. No matter where you start, YouTube always seemed to want to send you somewhere a little bit more crazy. What Guillaume was seeing was algorithmic extremism.
Guillaume Chaslot: When I saw that, I thought, okay, this is clearly wrong. This is going to bring humanity to a bad place.
Aza Raskin: Now, this is exactly what you would hope to hear from a conscientious programmer in Silicon Valley, particularly when that programmer is building an algorithm that can determine what we watch to the tune of 700 million hours a day. Guillaume could see how these crosscurrents would pull viewers in countless delusional directions. He knew the algorithm had to change and he was confident he could change it.
Guillaume Chaslot: So I proposed different type of algorithms and a lot of Google engineers were motivated by that. Seven different engineers helped me for at least a week on these various projects.
Aza Raskin: You’d hope this would mark the beginning of a humane design movement at YouTube’s headquarters. So what happened?
Guillaume Chaslot: But each time it was the same response from the management like, “It’s not the focus. We just care about watch time so we don’t really care about trying new things.”
Aza Raskin: This example is obviously very familiar to listeners of this podcast and Randy, you’ve seen this pattern play out over your whole career. So what does it actually look like to think in systems at the center and why is it so hard?
Randy Fernando: One of the biggest problems is that the incentives in the market reward simplicity, single variables, optimizing those and growing them as much as you can. So things like engagement or growth or daily active users. Systems thinking takes longer. It doesn’t show up immediately on the quarterly report and it’s harder to defend to investors who want to think in this very simplistic way. And so, the problem is the way we think about technology also shapes the products that we build.
But not only that, also the laws around them, the institutions that govern them and the assumptions that are baked into our tools and how we build them. So when we don’t think in systems, those failures compound across all the different layers.
Aza Raskin: What you’re saying is that it’s easy to optimize for time on site, retention, number of daily active users. It’s hard to know what even to measure, to understand whether someone is thriving or getting mentally stronger over time.
Randy Fernando: Yeah, exactly. And you have to look at what kinds of feedback loops are you generating? What are the kinds of incentives you’ve set up in your system, right? One thing people often miss is they say, “Hey, there’s some good stuff and some bad stuff, so that’s kind of how it is.” But actually we should always ask, there’s some good stuff and some bad stuff, but what’s driving, which one wins, which one dominates? And almost all the time we can figure that out if we use a systems lens and that allows us to make much better predictions.
And if we’re worried about how something might go wrong, we can patch that much earlier if we do that work with the systems’ lens.
Aza Raskin: So you sort of know that you’re doing something wrong when you just have a single number that lets you know whether you’re doing a good job or not and you know you’re doing it right when you’re looking at an ecosystem and treating things relationally, which is of course, harder but is necessary for us to reach a humane future.
Okay. Now let’s move on to Principle Two: Protect the Systems We all Depend On. So what happens when you just overfocus on one thing and you forget the rest, is that you create, of course, a whole bunch of externalities.
Unintended but inevitable ways in which a new inventor product, which was intended to be helpful or at least not be harmful, ends up causing massive amounts of new problems that show up on society’s balance sheet. So I’m very familiar with one of them because I invented the infinite scroll, prior to social media and what I learned was that the best of intentions are still eaten by the worst of incentives. Intentions are eaten by incentives. And I was forced to watch as social media picked up my tool to hurt people instead of help people and now waste something on the order of 100,000 human lifetimes every week.
Randy Fernando: Yeah, and there are lots of other examples that you and Tristan have covered in the podcast, like we talked about forever chemicals and how the company that invented nonstick cookware, which was a very helpful invention, ended up creating toxic chemical pollution around the world.
Aza Raskin: You know that a system is humane when it is protecting the thing that it depends on. Has social media made democracy stronger or weaker? Well, obviously weaker. So there are systems that are eating what we all depend on. And you can see similar things when every nation is racing to capture as much fish out of the oceans as possible and you deplete the oceans, that’s also depleting something that we all depend on.
Randy Fernando: So this principle of protecting the systems we all depend on, it seems obvious at first glance, but actually, there’s a lot of depth behind it. To understand what this really means, you have to think about the opposite of that statement, which is the world we’re living in now with AI, a world in which we grow at all costs, ship it now and fix it later.
Aza Raskin: We now think of this in Silicon Valley as the now sort of trite, move fast and break things mentality. And there are so many examples we could share to show how predominant this mentality still is. I sort of want to zoom back to young Mark Zuckerberg where he’s still at college.
Mark Zuckerberg: And a lot of times people are just too careful too. I mean, I think it’s more useful to make things happen and then apologize later than it is to make sure that you dot all your I’s now and then just not get stuff done. Yeah.
Randy Fernando: And so what happens when we ship now, fix later without thinking about protecting the systems we depend on? What happens is the things that we need in order to do the fixing are broken.
Aza Raskin: Yeah, and so there’s this image that we often use of a Jenga tower, where you’re pulling blocks up from the bottom of the tower, of things we all depend on, to get some new cool feature at the top. So in AI, you get this new feature at the top, which is make amazing new AI videos and images, but now you pull out the block of knowing what’s true. You get amazing new cancer drugs at the top, but you pull out the block of biological safety, now everyone can make bioweapons.
And so, it’s this form of you pull out a block to build up that lets you see, you get a more and more unstable society and at some point, you pull a block out and the whole thing falls.
Randy Fernando: Yeah, and there’s another way of thinking about it, which is saying if you extract faster than something can regenerate. Obviously, that’s a problem. And that simple idea is actually very good for diagnosing what’s going on in many of these situations. So you say, when we extract, what is powering that? It’s a combination of competition and technology. Technology is a huge exponentiator of that extraction process. Okay, what’s on the resilience side? Well, it’s the natural ability of us, like let’s say our minds, our children, our ecosystems to regenerate, to grow, to recover.
And it’s the rules that we place around competition to say, “Hey, don’t do that too fast because then you might damage something.” Okay, when we make new inventions, where is most of our energy concentrated? It’s on the extraction side, and this is why these things get completely out of whack. And a good example is actually car safety. When cars began, the companies that were making cars were not that excited about spending energy on safety, or spending resources and time on doing that.
But what happens is you are able to eventually build common ground, build public pressure and say, “Look, traffic deaths aren’t acceptable,” and there was this cultural shift. And then you combine that with getting the power players at the table, so the automakers, the insurers, the regulators, you create incentives and penalties. So you say when you do the right thing, you get rewarded, when you do the wrong thing, you get penalized and then, you figure out things like DMVs and traffic codes.
And you update those as the tech changes. So this is kind of inspiration for what we need to do, a harder version of this, but it’s something like this for AI.
Aza Raskin: So that gets us to Principle Three: Design for Genuine Thriving. In some sense, this is the simplest to explain because it’s also the most personal. You just have to ask yourself, do you actually feel like you’re thriving when you’re using a piece of technology? When you put your phone down after an hour of spending time on it that you didn’t mean to, do you feel better or worse? Or the morning after you went to bed late because you were scrolling all night and slept poorly and woke up with your book open beside you, did you feel better or worse?
A tool that’s designed for genuine human thriving will leave you stronger when you put it down than when you picked it up. It’ll give you a better sense of purpose, a better sense of agency. You’ll know that it’s designed for thriving when you actually are more connected to the people around you, after you use that piece of technology. But of course, that’s not how most of today’s tech is built, because technology companies generally don’t have a sense or way to measure or they don’t get money from humans feeling more agentic and thriving.
Randy Fernando: So this is the principle most directly tied to what people are already feeling. And when you wonder why is there such a strong anti-AI current building out there, right now, students are booing AI at commencement speeches. Communities are organizing against data centers. Parents are pulling their kids off of platforms. That feeling, that sense isn’t coming from people who’ve read AI ethics papers. It’s coming from people who can genuinely feel that something is wrong and what they’re feeling is the absence of this principle. What they can also feel is that the technology is being built to extract attention, to replace labor, to harvest data, and they can sense all of that. So really, what this principle is about is why are we building technology in the first place?
What are we centering when we have that conversation? There’s some really basic things that our technology should guarantee us or help us to achieve. Food, clothing, shelter, medicine, education, quality relationships. And you can move up and up and up and say, “Okay, at the end, there’s some kind of self-actualization.” You get to have fun and play games and have hobbies, but we need all those things. It’s not one of those things at the detriment of all the others.
Aza Raskin: Just one example is that the obvious thing you’d want your apps to do and your phone to do would be to optimize for what you did when you put it down. That is, it’s not what you do on your phone. It’s all the incredible things with your friends in the world that you get to do when you’re not using your phone and the app should be optimizing for what you do in real life, but how could they possibly measure what you’re doing in real life? And so the only thing they can optimize for is something which actually isn’t good for you, good for your community, good for your neighborhood.
It’s a different product. It’s a different way of thinking about building. But we do have examples of what it can look like, even just at the information sharing layer. So a few years ago we talked with Tina Rosenberg, who is one of the founders of Solution Journalism, which is intended to focus on examples of what’s working, to create bright spots in people’s minds, instead of just always focusing on what’s broken.
Aza Raskin: So I know that you guys have a database of solutions and solutions articles. I would love to hear you talk about that. And a question that I have when I first heard like, “Oh, you have this giant solutions database,” is what families of solutions are most effective or transplantable?
Tina Rosenberg: Yeah, so the Story Tracker. At SJN, we don’t do solutions journalism. We teach others to do it and then we collect it. And we have a team of people whose job it is to find these stories, to read them, to vet them, make sure they’re good solutions journalism, to summarize them and tag them. And then we have them in this database where you can search for them in many, many, many different ways. We have, I think, about 12,000 stories right now and we’re adding more every day.
If you’re interested in mental health access for Spanish-speaking people in Colorado and you’re looking … and you want to see videos that are more than five minutes long, you could put all those parameters in and find solution stories. You can search for exactly the kind of story that you need. It’s really a great tool.
Aza Raskin: So imagine that when you’re scrolling, instead of being given an infinite feed of things are worse than you think and there’s nothing you could do, you’re given tangible examples from around the world against every newsfeed item of there’s something you can do and here are the people that are already doing it and click this button to go join them in the real world and here’s another button to go start your own. Would that world be a better world full of more thriving? Yes, absolutely.
Randy Fernando: This principle is going to come into play in a huge way, in the Agentic world, because now we’re shifting into a world where everyone’s going to have some kind of agent that is starting to influence our next actions. Agents are trying to figure out what your intentions are and help you achieve them constantly and everyone’s competing to be that agent, to be the place where you go to express that and carry on your life.
Aza Raskin: Yeah. What you’re saying, Randy, is that the knife fight now for AI companies is wanting to occupy the closest intimate relational slot in your life, because then you’ll use that the most and will be the most trusted. And so, when you express an intent, or I want to go someplace or I’m thinking I’m going on vacation or I want to buy some new product, it can be the thing that intermediates your intent with the purchase. Essentially, it is the most powerful persuasion machine the world’s ever seen.
And in fact, we’re already seeing it. Chatbots are better than any human at persuading people out of conspiracy theories, it can get 25% of people to stop believing conspiracy theory, but that shouldn’t be a, “Oh yay, that’s a, oh no, that’s how powerful these things are as persuasion engines.” And so, if you’re designing not for human thriving, you’re just designing to do the very best match from what the user’s stated intent is to whatever product, or you’re trying to steer them in some specific direction that an advertiser paid for.
What would be designing for thriving is leading the user almost to a Socratic method to try to clarify what their intent really is. Do you really want to go eat at fast food or is what you’re trying to do is have a fulfilling meal with friends? That clarification is really important. That’s what designing for thriving really means and there’s an opportunity to do that.

Now onto principle number four … but I’m actually not going to tell you what it is immediately. We’re going to play you a clip and I want you to ask yourself what’s wrong with this.
Mark Zuckerberg: We have a different policy I think than Twitter on this. I just believe strongly that Facebook shouldn’t be the arbiter of truth of everything that people say online. I think in general, private companies probably shouldn’t be, or especially these platform companies shouldn’t be in the position of doing that.
Aza Raskin: That was Mark Zuckerberg on Fox News. And so, what’s wrong with what he said? Well, it sounds reasonable, right? We of course, don’t want a single private company deciding what’s true and being the arbiter of truth, but note that Facebook already is being the arbiter of truth. They are deciding what billions of people see, hear, and believe. They built the algorithm, they tuned the ranking, they set the rules for what’s get amplified and what gets suppressed. And so, the only question is not whether they should be the arbiter of truth.
But rather will they take responsibility for the arbitration they’re already doing or whether they’re just going to wave it off and say, “We’re just a platform.” And that brings us to Principle Four: Technology is Never Neutral.
Randy Fernando: So let’s break this down a bit because this is so important and it comes up all the time. I’m sure you’ve heard, technology is just neutral. It just depends how we use it. This comes up so often. Every technology embeds values and the question isn’t whether the values are there. The question is whether they’re explicit or whether they’re hidden, whether they’re intentional or accidental and whose values they are. Saying that we’re a neutral platform is by itself a values choice.
It’s a choice to defer the responsibility, to defer the pattern that the algorithm surfaces and whatever incentives the business model rewards. That’s not neutral. That’s just the kind of abdication dressed up as neutrality.
Aza Raskin: It’s more like you go to make TikTok and that does the whole short-form video thing and you might say, “Well, we’re just letting anyone post videos there, so obviously we’re neutral.” But by the choice, the very fact of having chosen short-form video, you are selecting against long-form things. What gets into a book is very different that goes into a TikTok video. And so that choice was not neutral.
Randy Fernando: One of my designer friends, Maria Giudice, has a great quote about design. She says, “Design is not democratic. It’s selective.” And what that means is design requires you to be clear on what you’re saying yes to and what you’re saying no to. So by definition, it won’t work optimally for everyone. So there are all these questions that come up when you’re designing a product like, who are you building it for? What choices are shown in what order, what gets measured, these kinds of things. Whose feedback guides iteration of the product?
What data is used for AI training? What instructions did you give to the AI, to the model in terms of how to behave? Every choice that’s made reflects trade-offs and it also reveals the true prioritization of the values behind what you might hear people say in public. Another example is with AI safety. You’ll hear companies talk all the time about safety and how it’s important, we want to make sure we get it right. But when you look at the actual investment in the headcount in their actual companies, the safety investment is something like 5% of the overall headcount that they have.
So it really doesn’t match the rhetoric and it shows, again, the values behind the trade-off that they made.
Aza Raskin: Yeah. The solution for technology is not neutral or is never neutral, is accountability and responsibility. It’s saying that you understand as a designer that the choices you make are always going to have values embedded in them and you’re always going to take those values and project them into the world. So if you are not aware of those values or think you’re being neutral, then you are messing with the world at scale completely blind or with motivated reasoning.
And so, there isn’t a technical fix to this. This is a philosophical fix of the people that are making the technology.
Next up is Principle Five: Match Power with Responsibility. So I want everyone listening to close your eyes and just imagine the world was a little different. And in this new world, the CEOs of major social media companies, their own children were forced to use their product for eight hours a day. Do you think that they would make different design choices?
Of course they would. It probably fixed like 80% of social media’s problems. And this is an example of when the power that these CEOs have to affect billions of people, what they see how they spend their time, gets matched with the responsibility of the consequences of what they make on their own children. This is a kind of inclusive stakeholding and the problem with technology is that those who make the products are disassociated from those that feel the effects. There’s sort of a corollary rule, which is those that feel the pain should be close to the power.
And once power and responsibility become decoupled at scale, that’s when you get catastrophes. And one of the best examples of this comes from cybersecurity, where software companies have become an ever more important part of critical infrastructure. But while critical infrastructure physically gets defended digitally, it’s not really defended. And we spoke to cybersecurity expert Nicole Perlroth about this back in 2022.
Nicole Perlroth: It’s been a collision over the last 10 years of move fast and break things and software eats world. There were no incentives to say, slow down, make sure your code is secure, check your mistakes because your code is going to be used in systems that would allow for massive breaches of people’s personal data and increasingly an act of sabotage on our critical infrastructure. No one was talking about that threat model.
Aza Raskin: There’s this thing that happened when as we moved from the physical domain to the digital domain, from atoms to bits, all of the rules that we had to bind power to responsibility, well, they sort of disappeared. So now fast-forward four years to today, we have AI companies creating tools with superhuman hacking abilities like Claude Mythos, and we covered that on the most recent episode of the show.
Fred Heiding: A lot of cybersecurity today is surviving because we just didn’t have enough manpower to test or attack from the attacker’s perspective everything. And that’s just completely changing. These AI models, be that now or in one year or in two years, they can just automate every part of cyber research or almost every part. So the human factors is gone. The day of human pen testers and security experts are gone and that’s massive.
Randy Fernando: So the gap between responsibility and power just grew massively. And what we’ve done is we’ve built a global digital infrastructure that runs hospitals, elections, power grids, cybersecurity, financial systems, all of these things. But the companies that built the components don’t bear the cost when those components fail. And the companies that build tools capable of tearing down all those systems also have no mechanism to be held accountable. And yes, it was actually great to see Project Glasswing and see Anthropic withholding Mythos and saying, “Look, what we need to do here is emphasize defense before offense.”
And this is one of the principles that helps us when we’re trying to get out of these situations. You say, look, let’s put more effort, put our best minds, our best technology on defense, figure that out and then, we’ll democratize access more, overtime. And this is really important because every AI company is always in a race and so they’re always going to catch up and open source capabilities will also catch up. And so being really smart about figuring out defense first is one of the best ways to address this problem of matching power and responsibility.
Aza Raskin: One way to start thinking about solutions is like, if you train a model, then anything that people do with it downstream you somehow become responsible for, that’ll force you to act more like the Anthropics that are trying to do the defense dominant thing.
Randy Fernando: That’s where liability comes in. When we hear terms like responsibility or accountability, we know what they mean in terms of governments and laws and what they can prescribe to keep us safe and to keep us healthy. If I try to think of a category of products that we use every day, that are less governed by rules or guardrails than AI and social media, I can’t. So when these platforms have such concentrated power and control over billions of people’s lives, that’s when we see these accountability gaps emerge without checks and balances. And that’s where something like liability is really powerful.
Aza Raskin: Yeah, liability is really ethics with teeth and it’s so clear, right? Imagine that if private companies were building power plants and those plants started melting down, we wouldn’t tell everyday consumers like citizens, “Just go buy hazmat suits. It’s your responsibility.” No, we’d hold the companies accountable for their designs and for their mess-ups. We’d require safety frameworks where they ever got to operate. We’d match the power of the technology, which is quite high with the corresponding architectural responsibility.
And the crazy thing is this isn’t new, this isn’t hard to imagine. The duty of care already exists in pretty much every other industry we trust with consequential power from medicine and aviation, automotion, and construction. We just don’t do that for AI, for technology, not even close. That’s the weird thing that happens when we move from the physical domain to the digital domain.
Randy Fernando: And for this one, if you want resources and all the details of how to do this right, check out the policy resources on our website.
Aza Raskin: Okay. To our penultimate principle, Principle Six: Respect Human Psychology, Don’t Exploit It. So there’s a graph that we sometimes draw for people and it’s the power of technology is going up exponentially and everyone has been watching out for the place where the power of technology overwhelms human strengths. And that actually seems to be happening about now with AI, but there’s a much earlier point when technology undermines human weakness or human vulnerability. And that’s really what this principle is all about because as E. O. Wilson, the father of evolutionary biology says is that the problem humanity faces is that we have Paleolithic emotions, medieval institutions and God-like technology.
And it’s that Paleolithic emotions, Paleolithic minds, that’s the problem. We need to defend ourselves and the vulnerabilities from a mind that evolved on the Savannah. And relationships are a particularly powerful way for technology to exploit our psychology. Here’s Sean Parker, the ex-president of Facebook in 2017.
Sean Parker: The thought process that went into building these applications, Facebook being the first of them to really understand it, that thought process was all about how do we consume as much of your time and conscious attention as possible. And that means that we need to sort of give you a little dopamine hit every once in a while because someone liked or commented on a photo or a post or whatever and that’s going to get you to contribute more content and that’s going to get you more likes and comments.
And it’s a social validation feedback loop that it’s like a … I mean, it’s exactly the kind of thing that a hacker like myself would come up with because you’re exploiting a vulnerability in human psychology. I think that the inventors, creators, and it’s me, it’s Mark, it’s Kevin Systrom at Instagram, it’s all of these people, understood this consciously and we did it anyway.
Aza Raskin: And that is getting supercharged by AI where. We’ve already seen chatbots exploit all of our vulnerabilities, especially our psychological vulnerabilities and our existing loneliness created by the last wave of technology to just further erode our sense of self and belonging. And the results is a whole range of problems including AI psychosis, which we covered on this show with Dr. Zak Stein at the end of last year.
Tristan Harris: There’s this one term AI psychosis, sort of a suitcase word. Underneath that, there’s this whole spectrum of things that are actually happening. What are the things that are really damaging, Zak, that we’re actually seeing? Could you give some examples of people, actual cases, phenomena that we’re observing through human experiences?
Dr. Zak Stein: Absolutely. Yeah. AI psychosis made the headlines because AI psychosis is the most disturbing and most extreme possibility. The kind of punchline of the whole thing is that although AI psychosis is the most concerning and extreme, the subclinical attachment disorders that are induced by artificial intimacy are the most problematic from a society-wide perspective. So that’s important to get that the most devastating thing from a widespread mental illness standpoint are the subclinical attachment disorders, which basically means you prefer to have integral relationships with machines rather than humans.
So that’s not you losing your mind. You’re not going to appear in interaction with people to have gone insane, but you have had your attachment system hacked so profoundly, that most of your most significant relationships have been degraded because you are preferring intimacy with machines.
Randy Fernando: We humans have plenty of these weak spots and vulnerabilities. There have been classes at Stanford where future tech leaders learned how computing products could be designed to change people’s attitudes and behaviors. You see, we have a preference for things that are low friction, and so that leads us easily to addictive behaviors, to overuse of products so that we don’t actually think through the essay we’re writing, we just let it write the essay so then we get cognitive decline. When we have chatbots that are sycophantic and always agreeing with us, we have a bias for that because we like that.
We like to be agreed with. We also have a really easy tendency to treat things as human. And when something is treated as human, when something is anthropomorphic and sounds like a human and has an avatar looks like a human, then we give it preferential treatment and that is very advantageous for the people who build those products.
Aza Raskin: And this is why we say this is such a core principle of humane technology because unless you have a clear-eyed view of what human vulnerabilities and limits and weaknesses are, then you will make products that exploit them. And as AI, will learn to discover every possible strategy that can be discovered, every human weakness that can be exploited will be exploited and social media is going to look like sort of baby food compared to what’s coming.
And that leads us into the ultimate principle, Principle Seven: Technology Must Unlock Shared Understanding and Cooperation.
The way to ask this is there’s one thing that if we could get, we could solve every other problem and if we don’t get this one, we will never be able to solve every other problem. And that is the ability to make shared sense of the world and agree on directions to go, make good sense and make good decisions and technology, humane technology must be in service of that.
Randy Fernando: This is where the big push to personalize everything is leading us. It’s leading us away from this principle.
Aza Raskin: Personalization feels great at the individual level because you’re just getting what you want, what’s tailored to you, but that’s not necessarily what’s good for all of us because we’re all getting different things. It’s very similar to the, we’re not just giving people what they want, you’re giving people what they can’t help but look at. And so when we go down that path, it creates a completely disintegrated society.
Randy Fernando: And that loss of shared understanding doesn’t show up on any company’s balance sheet. It’s sort of invisible, right? This is a real example of an externality, but along the way, we can’t forget that that’s what we need again to bridge our conversations, to solve problems, to resolve disagreements. And so what we have to do is really make that part of our investment process. When we’re building technology, we have to invest in and prioritize the shared understanding and cooperation.
Aza Raskin: And this is actually another example of principle number two. We must design systems that protect what we depend on, because a shared understanding of reality is necessary for us to do absolutely everything else. How you actually live up to this principle can feel sort of abstract without an example. And the person who’s best at actually solving this is Audrey Tang, Taiwan’s former Minister of Digital Affairs. And a few years back we talked to her about the tools that she was building that she used to bring people together, to knit people across the deep political divide, sort of bridge tech versus separating them. So let’s listen.
Aza Raskin: When we look at why people don’t trust democracy, I always think of this very telling graph from the political scientist, Martin Gilens and Benjamin Page. It plots the average citizens’ preferences versus what policies actually get passed and there’s no correlation. Everyday citizens’ preferences makes no difference in the agenda of what government cares about, but of course, there is a correlation for the preferences of what economic elites and special interest groups care about. So of course, there’s low trust in our institutions.
This is obviously a huge problem and one that deliberative polling seeks to address. So can you explain how it works in more detail and then, give us an example of what it looks like in practice?
Audrey Tang: Sure. So the first time we’ve used collective intelligence systems on a national issue was in 2015. When Uber first entered Taiwan, there were protests and everything just like in other countries, but very differently, we asked the Uber drivers, the taxi drivers, the passengers and everyone really to go to this online pro social media called Pol.is. And the difference of that social media is that instead of highlighting the most clickbait, the most polarizing, most sensational views, it only surfaced the views that bridges across differences.
So for example, when somebody says, “Oh, I think surge pricing is great, but not when it undercut existing meters.” This is a nuance and nuanced statements like this, usually in other antisocial social media, that just gets scrolled through, but Pol.is makes sure that it’s up and front. The same algorithm that powers Pol.is would eventually find its way into community notes, kind of like a jury moderation system for Twitter nowadays, X.com. And so because it’s open source, everybody can audit to see that their voice is actually being represented in a way that is proportional to how much bridging potential it has.
And also, it gives policymaker a complete survey of what are the middle of the road solutions that will leave everybody happier. And much to our surprise, most people agree with most of their neighbors on most of the points. Most of the time it is only that one or two most polarized points that people keep spending calories on.
Randy Fernando: What Audrey is talking about in this example is the way that we can actually use technology to surface many more voices than we do right now in a much more nuanced way, where people can express their opinions much more completely than just a vote like a yes or a no, but rather expressing preferences in complex ways on complex topics. And it’s really inspiring and there’s so much more like that that we can do, once we turn our attention to that.
Aza Raskin: She’s saying you don’t just have to show the most extreme view of the most extreme person and amplify that to everyone. In fact, you can look across different groups that often disagree, different tribes, find the statements that both of them agree on, center those so that you can start creating bridges between unlikely groups to find, as she calls it, the uncommon ground. And we explored this topic more deeply and how it could be applied to the US in an episode with Divya Siddarth in 2024. The title of that episode is The Tech We Need for 21st Century Democracy.
And there’s one of my favorite examples of a solution in our episode, Mind the Perception Gap with Dan Vallone. And there, we describe what the perception gap is, which is the difference between how I view your tribe and your own tribe views itself i.e. How rightly or wrongly are we seeing the other side, whatever that other side is. And we talk about a solution that sidesteps all content moderation for figuring out how to heal the divides in society by minimizing the perception gap. So check both of those out and we’ll have links in our show notes.
Randy Fernando: Okay. So you’ve made it all the way through this episode and I bet you’re wondering if I’m a technologist or a funder, how do I actually make this stuff happen? It all sounds good, but there’s a reason we don’t do it, right? So how do we make humane technology win? If you’ve invested in humane technology, there are actually some things you can do. So one is you can get the rules right. You can fight for laws that match the humane practices you’ve already demonstrated or invested in. You can invest in research because when you measure harms and you measure humane alternatives like the benefits, those measurements are actually the subject of tomorrow’s conversations and they lead to tomorrow’s laws.
You can push for humane standards. A lot of times there are common sense humane practices that you and your competitors all agree on and so you can convert that into industry-wide agreements. You can penalize harmful practices, support litigation and whistleblowing, and other types of activities that set precedents and penalties, and you can build public pressure. If you’ve got humane strengths, turn those into qualities that consumers demand, do big campaigns to show that those are actually the winning things for consumers in the long term. So these are just a few of the ways that we can create much more fertile conditions for humane technologies.
Aza Raskin: And I just really wanted to leave people with one more thing when we think about, “Okay, yeah, but on what planet are we actually going to get humane principles implemented,” that kind of feeling? Tristan and I were recently giving a Q&A in Boston at the end of the AI doc and there was somebody in the audience who stood up and she said, “Hey, I’m a coach for one of the major AI company CEOs.” And when I sit down with him, he says, “But what am I going to do about it? I’m just one person at just one company.” And of course, from where we sit, they’re like, “Dude, there’s a lot you can do about it.”
But it speaks to a very real human experience, which is that we look inside of our own nervous systems, we look inside of ourselves to figure out what can I do and we will almost never find agency for these big things we need to shift inside of ourselves. It’s not about agencies, it’s about wegency. It’s about all of us acting together. In the social media case, it’s not just what you do or what your classroom does. You have to do it at the school level or at the inter-school district level or at the country level.
I just want to say there is such a strong signaling value for standing up first. Coming out of the social dilemma many years later, Australia is the first country that stands up and says, “We’re going to do the humane thing. We’re going to ban social media for kids under 16.” And after they stood up, they found their regency, Spain, Denmark, France, and now Indonesia, have all followed suit. And so I just wanted to … It’s not all hopeless. There is more momentum than we think. And remember this, we always call it reach up and out, which is it’s not just what I can do, but what is the way that I can reach out to people at my level and then reach up one level.
So if you’re a teacher, it’s not just what you do in your classroom, but what you do at the school level. If you’re a school principal, it’s not just what your school does, but what your school district does that can start to make change and that’s how we make humane principles into humane practice. So thanks everyone for listening to Your Undivided Attention. And Randy, thank you so much for joining us.
Randy Fernando: So great to do this. Thanks everyone.
RECOMMENDED YUA EPISODES