Klarna CEO Sebastian Siemiatkowski on Getting AI to Do the Work of 700 Customer Service Reps

[00:00] I feel it's different with copy and image. In copy, though, [00:04] there in the LLMs, [00:05] are much less impressed. [00:07] And I think the reason for that is that [00:10] At least how the LLMs work is they work towards the average. So they are trained towards the average. And creativity is not the average. [00:35] It was 2010 when we first got into business with a young man named Sebastian... [00:42] based in Stockholm. [00:43] Fast forward to 2024 and Klarna. [00:47] is a global payments and commerce behemoth. [00:51] Klarna recently made its mark in the world of AI by sharing some of the results [00:56] of a product that they built for customer facing workflows. [01:00] Cloud has been one of the more aggressive experimenters in the world of AI, both with external workflows as well as internal use cases. [01:08] Sebastian joins us today. [01:11] to say a few words about what they've built. [01:13] and where he sees this world headed. [01:15] Sebastian, welcome to the show. Thank you for having me. [01:20] So you have become... [01:23] a poster child, perhaps the canonical example, [01:26] of putting AI into production inside of your business.

[01:30] to make life better for your customers and to make things more efficient internally. [01:35] And so the thing that everybody is desperate to know about... [01:37] is how did you do it? Why did you do it? What lessons have you learned? What are the pros and cons? Everything related to the customer support implementation that you guys have done. [01:47] But, [01:48] Before we get into that, unless we put the cart before the horse... [01:51] Can you just give us two words for people who may not be customers? What is Klarna? Give us a sense for the size and the scope and the business that Klarna is in. [02:00] Sure. So, I mean, it basically started as a payment solution for shopping online, often associated with this buy now, pay later thing. [02:09] But actually today, [02:11] We do about $100 billion worth of volume across the world. We have... [02:15] half a million merchants. We have about 100 million consumers [02:19] They can both pay the full amount, what we call debit, and they can pay in installments and use credit. [02:26] And it's also a fintech and a neobank in the sense that we have card sources, balances, [02:31] The whole thing, we are a fully regulated bank. [02:33] And there's about 4,000 employees. [02:36] Got it. [02:37] In 100 million-plus customers in 20-plus countries, we can see how the customer support implementation got to the scale that it did. [02:44] I think there are a lot of people who have contemplated doing something like what you guys did. There are very few people who have actually executed against it. [02:53] And so maybe the first question we'll ask you about this is, how did you do it? Like, how did you guys get into production so quickly with something that seems to me pretty darn effective?

[03:02] Sure. So I think you can start at the scratch. I mean, the first thing that happened to me, at least, was like November 2022 on Twitter. [03:11] And then I see people say, you should really try out this thing called Judge at B. I try it out and I'm blown away. I was like, wow, this is amazing. I've never seen anything like this. Or at least maybe when I tried Google 20 years ago. But this is even more impressive. [03:24] At that point in time, I was like, okay... [03:26] It's really cool But then holidays came Christmas And then after that I was like Oh [03:30] you know, we have to lean into this and let's see if we can get hold of Sam Altman and OpenAI and [03:37] and so forth, and I realized that soon Sam is going to be a person who's going to be impossible to get a meeting with, so I've got to try before everyone else. [03:43] And so finally, you know, lucky for me, I had Sequoia as a shareholder in both Klana and OpenAI. So that was a good opener. [03:50] I got it and I flew to San Francisco pretending that I was going there for other business, but meeting Sam was the only reason. [03:56] Originally, my time for meeting was two hours. By the time I got there, it was only 30 minutes left because the secretary was down-prioritizing my meeting. [04:04] And I came in and I sat down with Sam and I got a pitch to him that working with this [04:09] European bank, Fintech, is going to be a great client to test OpenAI products on because what I really wanted to accomplish was to [04:17] use us as a guinea pig. I wanted to make sure that we would always... [04:21] try the latest greatest and that they would find us as a great [04:24] clients to work and develop things with. And we managed to establish that relationship and [04:29] we had a joint Slack channel to start experimenting a lot. And then we...

[04:33] The second thing that we did was important to me was to encourage people internally to really lean in and try it. And originally, there was tons of these concerns. What about data? What about this? So we made sure to very quickly solve these things so we were GDPR compliant and that we could – [04:49] set the right structures around it since we're a bank and all that. [04:52] and really make sure that everyone in the company would experiment with it. [04:56] And then it just happened to be so that some people were more curious and more passionate, and some people were less, and that was fine. But some people leaned in, and it happened to be so that one of the teams that leaned in started looking at a fairly, actually... [05:09] complex challenge in a way which was what we call dispute resolution and dispute resolution in the bank is basically a customer calls us and says hey i didn't get that package [05:18] And the merchant says, but I did ship that package. [05:20] and then we have to be like a small mini court, [05:23] that basically gathers all the evidence and decides whether [05:26] who's done wrong and who's done right, right? And what are we gonna do on this transaction? Who's gonna cover the cost? [05:32] And these errands are very complex. [05:34] They require a lot of evidence, a lot of emails back and forth, and communication with both the consumer and the merchant takes a lot of time. And there's always been a backlog, and it's always frustrating because customers wait a lot before they get the final outcome. And so we started experimenting. [05:48] just to see if Chachapi and these services could help us. [05:52] make basically like a co-pilot help us [05:55] Take those decisions, Foster. [05:57] and [05:58] I think to some degree what's important here is that [06:01] You know, a lot of it comes back to the creativity of the team

[06:05] And we had a lot of teams in Klana. Some were more successful than some of this. This team happened just to be very, very strong and really good at what they were doing and very creative. [06:12] and they found and built away what is actually today referred to as rags, [06:17] they built already back then. They realized that would be a good solution. And within two months, they were demoing internally to us and others, [06:25] that they had managed to build a co-pilot that basically [06:28] helped [06:29] accelerate the process [06:31] and also increase the quality of the decision making because it was [06:34] it was making sure that we actually really [06:37] you know, took in all the irrelevant [06:39] information and then took a decision on these disputes. [06:43] And we said, look, this is amazing. Let's put it in production. [06:46] So far, just as a co-pilot. [06:48] And then the team, which was crazy, like two months later, [06:52] we suddenly get this Slack message internally which says, [06:55] We're out of errands. Can you send us more tickets? And that's never happened. Like this has been a constant backlog. It was just like, this is really impressive. [07:05] And then we said, look, it'd be crazy to try to see how we could... [07:09] you know, increase the pace of this and actually even answer customer service errands. [07:15] And then that team went on that challenge. [07:20] I think to us what was most critical, [07:22] was... [07:23] One of the rules that we agreed on was that the customer should always know if they speak to AI or if they speak to human. [07:30] and that's been important but [07:33] When we wanted to start testing this, we had a bug.

[07:36] And the bug was that for a few thousand conversations, [07:40] It wasn't clear that it was AI. [07:42] And... [07:44] When we then looked at, which was not intended, but the conclusion was we could read a few thousand transcripts of conversations. [07:51] where [07:52] The human wasn't aware, the customer wasn't aware that it was AI answering. [07:56] And we realized that the AI was doing a heck of a good job. [08:00] And that made us conclude that... [08:02] It's. [08:03] The most important thing to us was that customer satisfaction would be equal or greater. [08:08] then [08:09] what it was with a human agent. And as we saw that we started to reach that point, [08:17] Then we became less nervous about putting this into real production and actually try it, right? [08:24] So I think that like [08:25] it was really the effort of that amazing team that kind of tested and iterated [08:29] And that if you... [08:30] lucky shots that are along the way that then allowed us to put it into production. [08:35] Amazing. [08:36] There are so many questions that we want to ask to follow up on this. So maybe we'll start with, [08:41] You mentioned that RAG is part of the architecture. Can you say a few more words about the implementation itself and sort of where does OpenAI end and Klarna begin in terms of what you guys have done? [08:51] built on top to make this work. [08:54] Well, I think that the... [08:55] Uh... [08:56] It's funny because in general I'm always so freaking transparent and this is one time in my life that I actually feel a bit cagey about telling too much about the secret sauce. [09:05] because I should think about this as a fairly important strategic advantage.

[09:11] What I can say is that... [09:13] In our case, one of the key elements was that, like, [09:18] It's about... [09:19] making sure [09:21] that the instructions are clear. [09:25] If you onboard a human, [09:28] and you... [09:30] Ask a random human to sit in our customer service and try to answer a question. [09:34] And the documentation that's available to that human [09:37] is subpar. [09:39] because [09:41] there's an assumption that you can rely on [09:43] what people have learned. [09:45] in different sessions or assumptions. [09:47] you're not going to be successful. [09:49] But if you've written documentation that is detailed enough, so you could... [09:53] Even if very slowly put any random individual, and they could slowly go through your FAQ and manuals, but actually answer a question correctly, [10:01] because it was documented at level detail. [10:04] then it works. [10:06] And that's how I think about the AI today. It's basically... [10:09] an employee that turns out to work every day and has forgotten everything about what Klana is, how it works, [10:14] and every time you need to tell it again. [10:17] And that may change over time, but currently that's partially the game. And then so that helped us a lot to think about it that way. [10:24] that we just needed to make sure that the documentation and the manuals were clear enough. [10:27] and of quality enough. And then it can actually execute. Because many times [10:31] You know, if you... [10:32] It's, I mean, the truth that has been the truth for data scientists for a long period of time. Shit in, shit out, right? [10:39] If you feed data models with bad things, you're going to get bad results, right? So you need to make sure what you feed in is good, and then you can get better outcomes.

[10:47] Sebastian, I think you tweeted that your customer support agent is now handling two-thirds of your customer service inquiries. [10:54] You got a question from your audience on Twitter this week. [10:57] Are you planning to replace your CS department 100%? [11:01] with AI, I guess, from a technological perspective? Do you think that's possible on what time span and what are your plans? [11:07] Well, I mean, it's very hard, obviously, to predict. [11:10] how far will I go and what can I do in the future and so forth. But I think that the... [11:15] It's definitely not going to happen anytime soon. And I do think that [11:19] There will be customers that prefer a human for, you know, could be for any reason, could be because they have such a belief or, you know. [11:27] conviction or preference or whatever. And obviously you want to serve those customers as well. So, [11:32] There's no chance. [11:33] that the human ages are going away anytime soon. [11:37] With that said, though, I think actually the biggest quality [11:42] improvement that we see [11:44] is that [11:46] um [11:49] generally speaking and obviously we as every other companies to some degree want to avoid this but it's not uncommon [11:54] that are human agents. [11:55] have multiple chats going on. And we as customers all know that because you go and chat, [12:02] And you're like, you write a question and you don't get the answer immediately. And you're a little bit like, come on. And they forgot about you. [12:08] And they're like, "Hello, John, where are you? Why are you not answering?" And they're like, "Oh, I'm..." and so forth. And I would ask that because when I have tried to work in customer service myself, I did the same thing. It doesn't make sense because you also sometimes say, "Customer is slow when not answering." and so forth. So you start doing something else. You can't just sit in idle and wait for that. You want to resolve more things. You get it. But the consequence of that is that the average time of resolution

[12:31] of a customer service chat is about 14 minutes. [12:37] And when... [12:38] we move to AI. It's two minutes. [12:40] And the reason of that is because you get instant response as a customer, as opposed to that delay that happens due to that parallel handling of errands. And this is actually the biggest advantage. And so as a customer, a lot of customers that try that say, wow, I want this experience. But at the same point of time, we have something else which is funny. [13:02] which is that [13:03] AI chatbots have been around for 10 years or something, and they've all been of [13:08] horrible quality. So each one of us have gone to some airline and tried to converse about some tickets and been like, my God, this is the dumbest thing I've ever talked to. And [13:18] So the funny thing is that of those 30% currently... [13:21] that do not use [13:23] our AI chatbot. [13:24] The most common reason is the [13:26] when we start the conversation with them, [13:29] The first thing they write is agent. [13:31] Right. Agent. [13:32] which basically means they want to speak to a human. And that's not necessarily because [13:36] they so deeply want to speak to a human, as some of them are sore, but... [13:41] a lot of others is just because they had these horrible experiences and they won't avoid it. They just don't trust it to be good. [13:46] And so actually what I'm seeing is that what's happening right now is time. [13:50] Well, it will take some time to educate customers on the fact that like, [13:53] you know what, but this experience is actually many times better. And a lot of the people that tried it, they want to use it more because they find it more [14:00] you know, foster, foster, [14:01] um and i think that's the that takes a little bit longer time right there's

[14:06] the actual experience, but then there's the perception and expectation of what the experience is going to look like. [14:10] And changing that takes a bit longer than changing the experience itself. So, [14:13] I suspect we're going to see an even higher proportion [14:17] of things dealt with AI. But there's obviously a lot of complex queries that it doesn't resolve well today. [14:23] And that still needs to be improved on and there's still tons of work to be done. [14:28] What are the trade-offs? Are there ways in which it is consistently worse than what you had before? [14:36] Um... [14:40] Actually, no, but it's not entirely as I said. [14:43] That is not entirely due to the fact of AI. That is partly due to the fact that... [14:50] some of the instructions [14:53] that [14:54] or manuals that were written to help our human agents. [14:57] were subpar. [14:59] And... [15:00] The experience already before suffered from that. [15:03] but not enough managerial attention and focus was put to improving that and helping our agents become better at work. [15:10] So actually our agents have better [15:13] tools today to be successful in helping the customers [15:16] as does the AI. So the consequences both experiences are improving as a consequence of that. [15:20] But, you know, so I think that like... [15:22] partially it's true that like [15:25] things would be worse like yeah no i think both sides get better by doing this actually [15:30] because just you realize the importance of these things and I think sometimes to some degree previously you just [15:35] you know [15:35] There wasn't enough focus on the topic.

[15:37] Yeah, that's great. [15:39] And you mentioned the 14 minutes down to two minutes. Are there other statistics you can share that help to kind of illustrate the impact of this? [15:46] Well, I think the one that we were most famously quoted on was obviously the 700 full-time employees. [15:54] I think that one, and we were very... [15:56] you know, it's a difficult number to share, and I understand, like, [16:01] You know, we understood that people would [16:03] react to it. But at the same point of time, [16:05] I also feel to some degree that like politicians are too slow. [16:09] on [16:10] considering and thinking proactively what this is going to do. [16:13] how this is going to impact society. And we felt that there's [16:16] Some... [16:17] level of importance of sharing such statistics to Kenoff. [16:20] a little bit say look this isn't just fun demos on twitter this is actually having [16:25] real life business and real life implications. Now, with that said, in our case, [16:30] We are [16:30] or have been using customer service [16:33] on contracting firms, those firms [16:35] employ hundreds of thousands of people, [16:38] And if we historically have improved our products somehow, that may also have led [16:43] to less customer service errands because, you know, we fixed some issue or flaw in the product. [16:47] But obviously I've never seen... [16:49] an improvement to our product that [16:51] at a push of a button had this dramatic impact [16:55] a number of customer service agents that we [16:57] need. [16:58] And now fortunately for those agents, [17:00] There are tons of other customers out there, so nobody has lost their job. [17:04] as of today, as far as I know at least, as a consequence of this. But obviously in the longer term, it will have implications on these kind of jobs.

[17:11] But that was the statistic that a lot of people obviously reacted to. [17:14] um... [17:15] and the fact that it's about $40 million. [17:17] $. [17:18] of improved profitability for the company on an annual basis, right? So it's fairly [17:22] significant. I mean, we do about $2 billion of revenue. So, [17:26] Gives you kind of a sound sort of size. [17:28] Awesome. [17:29] and [17:30] And since you've been thoughtful about... [17:32] kind of the broader societal or economic impacts of this technology. [17:37] I'm curious if you had a magic wand and you could craft a policy or procedure that would help get us through what's likely to be an era of disruption... [17:50] Do you have any thoughts on what you would do with that magic wand? Yeah, for sure. Using programs you'd put in place. [17:56] The first thing that I think is super critical [17:59] is actually may sound a bit surprising, but it's an electronic identification methodology for humans. [18:05] Currently, there's no globally applied such methodology. [18:08] just like a passport but an electronic one. [18:11] And why that is so critical is because the amount of fraud and scams you're going to see increase. [18:15] is due to the fact that there's no ability for you, Pat and Sonia, right now, [18:20] to ask me for my electronic identification to verify that I am not [18:24] a bot. [18:26] or I'm not some, you know, what's it called? [18:30] fake... [18:32] you know, video created or pretending to be an AI, you know, not an AI talking to you. It's actually the human, right? [18:38] And I think being able to verify that you are talking to the real human

[18:42] is critical. [18:43] If we can... [18:44] supply [18:45] that on a global level, but preferably even on a country level, that will at least [18:51] it reduced the risk of fraud and so forth because you'll be able to authenticate. Like, am I talking to Pat, the real Pat, or am I talking to your bot? Right? Like, [18:58] I want to be able to know about. [18:59] So that is very critical. I think that needs to be resolved fairly quickly because otherwise we're just going to see an explosion. [19:04] as these. [19:05] I have seen videos of myself talking to customers that, [19:08] we have produced. [19:09] that look identically to me, sound like me, which we are about to send out to over 1,000 of our top merchants. [19:17] And so like... [19:19] it is crazy to see those avatars, you know, and being able to impersonate themselves. So I think that's one thing. The second thing, though, [19:26] is if you're left-leaning, I hear people on the left side of the political spectra, they are saying, like, stop this, stop the progress. [19:34] I have a hard time. [19:36] especially considering that there are less democratical... [19:38] Countries in the world that may. [19:40] push this agenda as well. And so I think that's not necessarily the best outcome. But if you're on the right wing, [19:46] of the political spectrum. A lot of people say, oh, don't worry, there's going to be new jobs. There's always new jobs. This happens all the time. There's always new jobs. [19:53] And I think that's a little bit of simplification as well. [19:55] When I was in Brussels... [19:57] There are, I think if I remember correctly, 10,000 translators [20:01] that are employed in Brussels to translate all the European legislation into the local languages of Europe. [20:06] Those 10,000 translators are basically... [20:10] almost redundant today with the technology of DeepL and ChatGP and so forth.

[20:14] to some degree, right? I mean, you could at least reduce it dramatically. And I don't think it's easy to say to a 55-year-old translator, don't worry, you're going to become a YouTube influencer. [20:25] What you can do from a society political perspective is you could think about, okay, maybe I don't want to stop progress. [20:31] but maybe I can offer... [20:33] something to people being affected. [20:36] Maybe I can offer something to them. Maybe society can... [20:39] have the luxury of at least support individuals that are affected by these changes, because not everyone will be able to just retrain into something different. [20:47] in that facility. And I hope if [20:49] If there would be such measures, or at least plans or ideas among politicians, [20:53] then maybe you can take society through this change with a little bit or more of empathy and care. [20:58] about the people are... [21:00] who are affected, while at the same time not saying that we have to stop progress, right? [21:07] Yeah, thank you for being so thoughtful about it. It's encouraging to see people in leadership positions like yours being so thoughtful. [21:13] Thank you. [21:15] Sebastian, what types of jobs do you think are going to be most affected and... [21:20] What type of jobs do you think will be, you know, what skills are you teaching your kids to learn so that their future livelihoods are AI aligned, so to speak? [21:28] Yeah, so I think that like, it's funny you say that because when I met Sam back then and I got that meeting, I said to him, look, Sam, one thing that's going to happen is people will, this is going to have impact on jobs. So I think if you want to make this a very popular technology, you should identify like what are the job categories that people hate the most?

[21:47] I happen to have two of their three ones because... [21:51] I'm both a CEO and I'm both a banker. And those are two of the ones. And then you have only the lawyers, right? So those are the three ones. So I said to Sam like, what you should focus on, try to build AI that replaces CEOs, bankers and lawyers, and nobody will make a big fuss about it. [22:07] Unfortunately, you know, and I saw it very clearly, 'cause when, [22:10] You know, when we did a tweet later on about the marketing things we're doing about AI... [22:14] where we have less need for money. [22:17] photographers and and side copies and things less needs we still need them [22:22] But we need them predominantly for the very creative stuff and less for the kind of day-to-day stuff. [22:28] that had a violent reaction. [22:30] online and I can understand why. [22:32] because people, you know, really feel emotionally [22:36] resonate a lot to that. While when you see online tweets about [22:40] AI lawyers, nobody seems to react much. I feel sad for the lawyers in the world. I hope people remember you as well, actually. But anyways... [22:49] You know, it's scary, right? It's scary because I don't know. [22:52] I... I... [22:53] I find it... [22:56] This is a very difficult question to answer. [22:59] I just... [23:01] I mean... [23:02] to some degree [23:04] definitely physical jobs, right? Like, I mean, it just looks currently as [23:10] On the very long-term perspective, it's going to be easier to replace knowledge jobs than it's going to be to replace... [23:16] you know

[23:17] driving a truck or even though we were so convinced about self-driving cars and all that. [23:23] Or, you know, proper robots seems... [23:26] a little bit further out than, you know, [23:29] AI. [23:30] So it's difficult. [23:33] But that also assumes that everyone wants to work. I'm not sure. Like some people would like work. Some people... [23:38] we'd enjoy a society in which robots serve us and we just, you know, hang out and play football. So, like, it's a little bit... [23:44] It depends. It's hard to predict where all of this is going to go, right? [23:47] I preferably love work, so I will be one of the depressed people when I'm [23:52] AI takes my job and I'm going to sit and like [23:54] and be like, okay, that was the end of the fun, because I really enjoy it a lot. But like, you know, people are different, everyone's different, right? [24:00] Yeah. [24:01] So let's talk about some more of the stuff that you guys have built internally. So you've mentioned a little bit what you guys have done in marketing. Can you say more about that? [24:08] Mm-hmm. [24:09] Yeah, I mean, it's been very interesting as well. I mean, because, again, there's so many demos, right? And it's like... [24:15] I mean, I think everyone has tried and you've gone, you know, try to create an image or create a video and you are blown away first with what you can do. [24:24] But then you're like, yeah, but I want it to look like exactly like this. And I want it to be consistent with my brand feeling. And I wanted to, you know, et cetera. And then you start. [24:31] being more challenged with it, right? Like, and I think that's why also sometimes I feel a little bit like, [24:36] For example... [24:37] You know, people say like, oh, [24:39] it's, [24:40] unfair to the creators that these tools are being created in marketing and so forth. And I partially understand why people say that, but at the same point of time,

[24:49] We have this guy. [24:50] who [24:53] has [24:53] totally immersed himself in this video [24:56] you know, video creation, sound creation, marketing, creating things. And, [25:01] and created this amazing [25:04] basically just [25:05] scripted together a lot of these different technologies to create like these automatic marketing videos. [25:10] with me making presentations to Mershift, as an example, me as an avatar. And like, [25:16] And it's really nice. It looks really great. [25:18] It's on brand and so forth. [25:20] He is a creator. He is extremely creative. And it's a little bit like I can assume that when... [25:27] you know, [25:28] when the music industry evolved and synthesizers came along, [25:31] and computers to make music, [25:33] You know, some people were like complaining, that's not playing a guitar. That's not creative. You're sitting by a freaking computer. Nobody would say that anymore. But I'm sure there was a lot of that criticism. [25:41] To some degree, I feel the people that are adopting these technologies today, they are just... [25:46] They're very creative, but they're using new tools. [25:50] to be able to create what they see in their minds. [25:53] And so, you know, I think that's what I'm seeing. So we are basically... [25:57] having people who may not themselves be photographers, who may not themselves be great at Photoshop, or who may not themselves be great at all these things. But now, [26:07] with text and communication with the computers, they can create [26:10] what's in their minds, and they can explore ideas and concepts. [26:14] of marketing campaigns, of marketing material, of doing things in a way [26:19] That was unprecedented before, right?

[26:21] That's really exciting. And they obviously there's yeah, it's true. There's less people involved, right? Like, [26:27] there are less people involved because previously if you wanted to produce [26:30] a commercial, there are tons of people involved. And some of that is beneficial because you have [26:37] you know, different people coming with ideas and thoughts. [26:40] But some of it is also... [26:41] less beneficial because you have a [26:43] a few people have this [26:44] amazing idea, but they're not capable of turning that idea into a reality because they themselves aren't the photographer and they're themselves not all of this. And now they can actually [26:53] bring their ideas to life at a different level than they could before. [26:57] But those are the things that we're kind of doing. A lot of that is just like, [27:00] How do we go from [27:02] you know [27:03] Um... [27:04] Basically, [27:05] we want to market our credit card in Germany. And like, how do we go from that to actually having a campaign live that looks really great, uses the right copy, [27:15] etc. [27:16] And... [27:17] we have seen... [27:18] that we have already seen internal examples where something like that historically may have taken [27:23] a month or two months to prepare and go through different teams and approvals. You also have to remember as a bank we need [27:29] to make sure that we are [27:30] communicating in a regulatory compliant way because credit cards are regulated products. So there's like [27:35] tons of complexity associated with these things. [27:37] and nowadays we can see a few individuals [27:40] go from idea to actually having a marketing campaign live in a week. [27:44] or at a time frame that was impossible historically. And the quality of the campaign is higher, and the lawfulness of it is better, and all things are better.

[27:57] Sebastian, you're a creative soul and an artist, and I think the Klarna brand has always been just so special and quirky, vivid, creative, all of that. What do you think of the quality of the AI-generated creative copy? [28:09] What do you think can be outsourced today and AI and what can't? And do you still prefer... [28:14] you know the [28:15] You know, the gorgeous photo shoots that you all do in-house. [28:18] It's a good question. [28:20] uh, [28:21] I feel it's different with copy and image. [28:25] In my opinion... [28:27] when I look at the imagery, [28:29] I feel it's more fun because it can be... [28:32] to some degree more [28:34] crazy and imaginary. [28:36] so there I see less but in copy though [28:40] There in the LLMs. [28:42] much less impressed and I think the reason for that is that I [28:47] At least how the LLMs work is they work towards the average. [28:51] So they are trained towards the average. And creativity is not the average. Creativity is the extreme of recognizing that this is a total new way or a new way to combine things and stuff like that. And that's why I still think that for some period of time, creativity will... [29:07] out-compete these things. And that's what I mean. It's one thing. If I want to write [29:13] If I need to write a text about a product, [29:16] Like we have obviously, we have a product comparison website [29:21] where we have millions of products listed, right? Like clothing, iPhones, whatever. And we need product descriptions, right? For those cases, LLMs are great.

[29:31] Um, and they're very efficient and stuff like that. But when you want that perfect, quirky copy, that's going to catch the attention of, you know, of a human audience. And, you know, they're going to talk about it and thought that was funny or something. [29:44] much worse like well you can obviously generate a [29:47] fast a lot of versions but I still feel that like [29:50] It's pushing towards the average, and the average is not... [29:54] Sorry, like the average is the average, which is the average. It just doesn't stand out much, right? So there I still feel that like humans are much better at that. [30:02] of thinking outside of the box, so to speak, because the LLMs are almost like thinking in the box, that's [30:07] That's basically what they do. They're supposed to think in the box, right? Like according to the box. [30:13] That's such a fascinating dichotomy. Thanks for sharing. [30:16] Can you tell us about Kiki, I think is what it's called? [30:23] for [30:24] For Klarna, at least, it happened to be so that [30:29] coinciding with the [30:31] with the [30:33] Um... [30:35] with the AI [30:37] Revolution. [30:38] we also started obsessing about the concept of collaboration on information. [30:44] And it's actually also one of the technologies we've started using extensively in-house is Neo4j and graphs, which we didn't really explore much beforehand. [30:56] And we've also looked a lot to Wikipedia and other... [31:00] knowledge graphs and how people have built...

[31:04] You know, how people collaborate on building great information. [31:07] and [31:09] So a big initiative internally has been to start [31:13] bringing together information that are sitting in silos across multiple systems. [31:18] and improving the quality of that and really creating collaboration, which actually has the side effect [31:25] of us also deprecating [31:28] Salesforce. [31:29] deprecating [31:30] a lot of enterprise software systems, [31:33] because we move that data into one. I'm not saying, I mean, for example, Slack, we're great users of, so we're still big customers of Salesforce because of Slack instead. [31:40] But some of that takes like we've had too many of these enterprise software systems. [31:45] and as a consequence information about what we do and how we work is dispersed. [31:49] And it's inconsistent. [31:51] And so a big piece has been bringing that together, standardizing and harmonizing that. [31:57] And then on top of that, we have Kiki. [32:00] who then explores that information [32:02] And [32:03] brings it to life. So we can go and ask Kiki about anything, about how many employees are we in that part or what does this team work on? [32:12] you know, what's important to consider when you launch a system internally, what are the steps that you're going to go through and all of that. [32:18] is getting centralized into one place and connected through the knowledge graph, we're seeing that that is having [32:24] a tremendous impact on productivity, [32:26] internally. So Kiki is basically our own internal chatbot. [32:31] Based on that. [32:32] growing internal knowledge graph.

[32:35] How and where does Kiki show up for employees? Is it a Slack bot? Where do people interact with it? Both in Slack, but also we have something that looks like a Wikipedia. The knowledge graph, when you look at it, looks like a Wikipedia basically for our employees. [32:50] And you can both kind of read the articles themselves, but you can also interact with [32:53] Kiki to find information in that, right? So it's a combination of semantic search and AI to interpret the information. And that has proven to work really well. Like, [33:05] It has been a tremendous adoption internally, and I think it's created... [33:08] tons of value for us. So we're very excited about that. [33:13] And the comment on deprecating Salesforce is really interesting. I can see how... [33:18] the system of record functionality can get replaced. [33:22] For the system of engagement functionality, the workflows that people might have been doing on top of Salesforce... [33:28] Where have those gone? How do people... [33:30] whatever jobs to be done there were on top of Salesforce previously. How are people doing those jobs now? [33:35] a mix of things actually but [33:38] It's less about like [33:41] I think it's less about the fact... So some of this is actually as simple as Slack workflow. Actually, the workflows in Slack are pretty good. So you can just joke at us and laugh at us that you just moved from one proprietary system to another. But I think that the... [33:55] But it's not about that, it's the number of such systems, right? [33:59] Because... [33:59] I want people at Klana to collaborate genuinely. And one of the things that's been so revealing throughout this process is that,

[34:07] Whenever people have a new system, a new place to go and look for information, it all creates these silos. [34:13] And it reduces the ability for us to collaborate across the organization on information and providing value. [34:19] So just remember, [34:21] Removing the number of systems is important. [34:24] to have fewer. [34:25] and more quality and standard across the organization so so some of those workflows are implemented directly in our own tech stack and some of those workflows we're still using proprietary system for like i mean [34:38] where for example, [34:39] Moving our HRS. [34:41] out of Workday into Deal. [34:43] with great success. It's not like we're entirely, but we are reshaping it. We're [34:47] We're using only the payroll stuff and... [34:50] We are also within a few weeks deprecating workday. [34:52] because there was also too much information that was important. Think about [34:57] For us, understanding the organization and how it's tied together [35:01] If we're ever going to get kicky, [35:04] and our internal knowledge graph to function properly, the understanding of our organization, the teams... [35:09] The reporting lines is important. [35:11] So that could not sit in a proprietary system that needs to come in-house. [35:15] but obviously generating payroll and making sure we pay pay on salary on time and so forth. [35:21] that we are a happy customer deal nowadays. So like, [35:23] So it's just been a change in our tech stack. [35:26] Sebastian, how do you think about buy versus build decisions differently? [35:30] You have AI for customer support, you have AI for your knowledge graph, you have AI for marketing. [35:35] Each of these categories now has companies and vendors serving them.

[35:39] like Sierra and Glean and companies like that. I realized that, you know, you kind of built what you had, you know, before these solutions existed. [35:46] But I guess if you were to start over from scratch today, [35:50] Or what advice would you give other founders who are just kind of embarking on this journey? Should they buy or should they build? [35:56] It's a great question, and it's one obviously we ask ourselves all the time. [36:00] But I have to say, I'll give you an example, right? [36:02] when [36:04] when we started [36:05] encouraging people in Klono to use AI. [36:09] We didn't mandate them. [36:11] To do things. [36:12] That was core for the business. [36:14] We said, [36:15] Take the idea that you're passionate about and explore it. And one of the examples that we built early days was Google. [36:22] We said, look, one of the things that we hate in a big company is these employee engagement forms. [36:27] Because they go like, hey, how are you feeling at Klana? Great. On a scale one to five, you know. And then we're sitting and trying to interpret the answers to these forms. [36:35] We felt it was a very... [36:37] imprecise and open for a lot of interpretation and subjectivity. So we said, it's not a great way. So we said, hey, wouldn't it be fun if we could do a deep interview with every employee? [36:51] But maybe we can't do it. Maybe the AI can do it. And so we built that. [36:55] We built a deep interview. [36:58] robot based on ChatGP that we then deployed to all of our employees and said, hey, [37:03] Would you be fine with interacting with 30 minutes with this? [37:06] interviewer [37:08] in order to tell about how it is to work with Kulana and benefits and strengths and so forth. And then it took that information, summarized it,

[37:14] and basically came back with like, you know, what are the strengths and weaknesses of working Klonah could be improved, et cetera, right? Now, [37:20] The point is, [37:22] We built that, and today there are already AI tools and startups out there offering similar solutions that you can use for customer surveys or for employee surveys or engagement surveys, et cetera. [37:33] So it's not like we were the only ones doing that. Neither are we in the business of doing that. So obviously you could say today we should have bought it, but... [37:41] With that said, I am so happy we built that because we learned so much. [37:47] And the employees in Tonle Kvanna learned so much from building that, that we're now applying those learnings to other things. [37:53] And we're not [37:54] using that. So then maybe today we'd go and buy that from somebody, but I'm still happy that we did it. So I feel a little bit like this is such an emerging industry. Obviously, if you see something, [38:03] that you feel, [38:04] intuitively [38:05] It's just better than anything you can build yourselves right now. I would do it. [38:09] But there's also like so much power in just letting... [38:12] people learn how to use these things and deploy them and develop them because it's such a new technology and there's such a [38:17] massive value created from people learning to do these things themselves. [38:21] So that's why we're a little bit cautious still about buying too much. [38:25] from these even though we really want to be supportive of the startup community and so forth we're a little bit cautious because [38:29] We just want to try ourselves first to learn [38:33] And so that's how we're thinking about it. [38:35] And I think then I would also add one more thing on it. [38:39] like when we for example initiated that discussion about should we keep workday or not

[38:44] I contacted the CEO at that point of time and all of a sudden I said like, hey, convince us. [38:49] about it. But then I realized one thing that was funny, which is [38:53] And this is an advice to all companies. [38:55] that [38:57] If I go to chat to the P and I say, [38:59] What? [39:00] does the API document, what is the API calls that I can do with Workday? [39:04] I'm sure they fixed it now, but at that point of time, I gave them that feedback. At that point of time, their API documentation was behind the login. [39:11] So as a consequence of that Chachis B had not been trained to [39:15] on the APIs of Workday. [39:17] It is familiar with the APIs of Slack, because those are public documentation. And it's even more familiar [39:23] with things that are open source. [39:25] because it's been trained on the open source libraries. So there's suddenly this massive benefit [39:30] from being open source software [39:33] and even more so to make sure that you have public APIs and public documentation of your software. [39:39] because then suddenly, you know, Chachapi understands that he can interact with and support you in your interaction with that. [39:44] This is like a funny reflection. [39:45] So I really encourage you like these. [39:48] you know, more traditional companies to like make sure that everything you have is actually, you know, publicly available, easy. And, you know, don't lock this behind doors, right? Because [39:56] then it's not going to be used. [39:59] to the same degree. [40:00] yeah yeah yeah [40:02] Um... [40:03] Speaking of locked behind doors, a lot of the stuff we've talked about so far is kind of the internal to QARNA operations benefits of AI. [40:10] Let's talk about the product. What have you seen or what do you see coming for AI in your product?

[40:19] Ah, now I'm going to be even more kedgy. [40:23] I look, I... [40:26] I am extremely excited. We have some stuff that's going to go live in a few weeks that is like... [40:32] So, [40:33] It's like a beta, but it would basically be the customer service assistant on steroids. [40:39] in a sense that it will be... [40:41] even better but it would also start advising you and giving you some ideas and thoughts around [40:47] you know, the type of services that Klona offers that I think people will find quite [40:51] cool. [40:53] But [40:54] it's still beta, right? It's not [40:56] It's not going to be something yet of that kind. [41:00] I, when I then look at our internal projects, [41:03] I think... [41:05] within six to twelve months. [41:07] we will be able to start launching things. [41:10] that are truly [41:12] are disruptive in the way of services. But the funny thing with this and [41:17] is that [41:20] back in 2015. [41:22] long before all of this happened. At that point of time, Klarna was trying to compete [41:27] would strive Benadion on being a... [41:29] payment service provider. And when Adyen signed Spotify, which is a neighbor of ours, we just had to look ourselves in the mirror and say, shit, they're beating the crap out of us, both Stripe and Adyen. So we had to change direction. [41:44] And at that point of time, we cannot pivot it. [41:46] And when we... [41:48] sat down in '15 and asked ourselves, where is financial services going? Already then we said, well, eventually in the future,

[41:54] You wake up in the morning, [41:56] and your digital financial assistant says, hey, Pat, I've analyzed your mortgage, and I realize, I'll save you $10. [42:02] by switching from bank A to bank B. [42:05] And by the way, the only thing you need to do is say yes. [42:07] Right? [42:08] And so like... [42:09] we realized that [42:10] Shit, that's going to happen. [42:11] That was a revelation to us in '15. Now we, just like self-driving cars, we couldn't predict how fast or when that's gonna happen. But it was very clear to us that like, eventually that's gonna happen. And for an industry like banks, [42:23] banking? What does that mean? First and foremost, what's cool about it? [42:26] it means tea. [42:28] evaporation of all the excess profits. [42:31] in banking industry, because a lot of the bank profits are built on the lack of customer mobility, the unwillingness of us as customers to move between banks and the friction associated with it. [42:42] And when AI assistants will allow you to do that, [42:45] Just because... [42:46] You say yes. [42:48] then that will make a big difference in the market dynamics and the competitiveness. [42:54] of fintech and banks. [42:56] And I see that happening in the coming... [42:58] two, three years, for sure. And so... [43:01] ever since then, that's been the direction of the company. And [43:04] That's still the direction. We want to, because we realized ourselves like, [43:07] we don't want to be [43:09] One of those banks. We want to be that digital financial advisor of yours. [43:13] That's what we want to be. We want to be that AI digital finance assistant that helps you save time, save money. [43:19] make you feel more in control of your finances. And I think that's the natural evolution of every fintech. [43:24] And that's where we're going to go.

[43:26] So that is the direction and that's the type of services that [43:30] that we're building and trying to accomplish and [43:33] you know, [43:34] this vision we've had with the current management team that I work with has all been with me almost 10 years. [43:39] This has been the vision for 10 years. But obviously, when we saw Chattopi, we felt like, oh, it's going to happen sooner than later. It's going to go a little bit faster than we thought. [43:48] And the services we're building are all in that direction. [43:52] It's just about helping people save time, save money, be more in control of their finances. [43:57] But not on the shopping side. [43:59] Like, I'm addicted to your app as a, you know, avid shopper, too avid shopper. [44:04] And I guess my dream is to have an AI. [44:07] That would be incredible. Do you guys think you'll make plays there as well? [44:12] I think there will be... I mean, it's interesting... [44:16] It's interesting. I think in general, if you look at... [44:19] e-commerce. You can basically think about it as three things. There's a curation job to be done, which is what you're talking about. There is the brand, the product and the brand, [44:30] and then there is the infrastructure, [44:32] that helps you, you know, payments, shipping, all the stuff that's needed between. I think that's why if [44:38] If I think about the AI evolution in commerce, [44:41] I'm... [44:41] less worried about the [44:43] Brands like Nike will be Nike and people will want to buy Nike, right? [44:47] And retailers is a bit more different because, [44:51] The curation has already been split up. We have TikTok. We have Instagram. We have influencers. And we have the retailer, the Best Buy agent, who's trying to help you recommend which TV you're supposed to buy, right?

[45:03] And so I think that... [45:05] Within curation, recommendation of products and selection... [45:09] I am 100% convinced that you're right, Sonia. You're going to see... [45:13] arise in such, right? But it's also a very difficult case to do. I've seen a lot of the attempts, for example, to do travel AI suggestions. I tried it myself since I'm planning a road trip in the US with the family this summer. [45:25] And like, and it was pretty bad. It's just hard because like, you need to understand who I am and what my preferences are and what do I think and it's just, [45:34] It's more complex than we think, right? So... [45:37] I think it will happen. [45:39] but I think it will take a little bit longer time maybe. There's a lot of things still that needs to come into place, but it... [45:46] I think it's a little bit different if you're thinking like, [45:49] you know, there's obviously easier things. It's easier things are like, I mean, Amazon is already doing some of this stuff already, but like, [45:55] Easier things would be like [45:56] "Hey Pat, I know you're using contact lenses and I saw you bought them a month ago. You're probably running out of them. Do you want a new one?" [46:03] That's easier. [46:04] Right? Then like... [46:06] Hey, Sonia, you know, I think this dress would fit you because like your style is according to this. So I'm a little bit. But I show you, I can tell you one cool thing on this topic, which I think. [46:16] at least blew my mind away. [46:18] And that was... [46:19] internally we had done a test this is just a test right [46:22] The idea was [46:24] that within the Klon app when you open it, there's category pictures. Okay, so there's like a category picture like shoes, [46:30] you know, there's [46:31] home garden products, whatever.

[46:34] So what they've done [46:35] is they had taken, [46:37] and taken me. [46:38] my customer profile, all of my transactions that I've done with Klana, everything. They've taken my profile as a user, Sebastian. [46:44] And then based on that, [46:46] They had generated a category image, which was a shoe. [46:50] Well, it looks a bit like a Nike shoe. [46:52] So, [46:52] And they just wanted to create a more personalized category image that would catch my attention. So it was a... [46:59] imaginary AI created image of a shoe. [47:02] But the crazy thing is, [47:04] I looked at it. [47:05] And I was like, I want to buy that one. And I am not a big shopper. I'm not a big shopper, right? [47:12] But I was just like, that is insane. There's something in the fact that you fed my profile. [47:18] into that, my preferences of brands and purchases and all that. [47:21] And the image you generated was actually attractive. [47:25] And we already know that she and then the others are doing amazing stuff where they are predicting purchase behaviors and testing products on small quantities. [47:33] and they [47:34] And people start buying them and then they produce more quantities and they're super fast. It's super impressive to see what these guys are doing. [47:40] And sometimes I feel also people forget that actually that leads to less waste because [47:44] The bigger retailers buy a lot of products that never get sold and it's bad for the environment. So this is actually better for the environment to some degree, even though people are very critical about it. [47:52] But [47:53] What I thought here is just like, wow, I just felt like I got a glimpse into the future. I was like, the next thing is I'm going to be out shopping and the images that I will see are things that doesn't even exist. They're just, you know, created on the fly based on my profile. And then if I do click them and want to buy them, they're going to be.

[48:10] Produced [48:11] post me saying I want this right and that was just like [48:15] And again, these are like the self-driving cars things. I don't know when, but I felt very convinced that it will happen eventually. I think that was pretty cool. I was just like, wow. [48:24] That's the next level. You know, products generated. It was funny because it was a shoe. And the other thing it had created was an image of a – apparently I bought a lot of home gardening stuff, which sounds odd because I'm not a big home garden person. And it had created a lawnmower, you know, like one of those that you cut a lawn with. [48:41] But it was super nicely designed. And I was like, yeah, that's how I would like a lawnmower to look like. That's really, really nice design. Can I get that one? [48:48] So I think that's a glimpse into the future. [48:51] The future will be generated. Let's move into a sort of rapid fire round and we'll start with a question we like to ask people. Who do you admire most in the world of AI? [49:03] Nah, but it has to be Sam. I'm sorry. [49:06] It's an easy question. [49:08] Great. Next question, Sonia. [49:11] Sebastian, you and your wife are both patrons of the arts and avid art collectors. Do you have any AI art in your collection? And do you think you will ever have any AI art in your collection? [49:24] I don't have. I think I could have. I think, to me, an image is something that's supposed to create an emotional reaction of some sorts, right? Right. [49:33] And I think it's fascinating. I don't mind if the emotional reaction is created. [49:37] by an AI, if it touches me and it means something to me, that's what's important.

[49:42] But that doesn't mean that I don't think we will continue to buy a lot of art from human maids as well, right? [49:50] What is your best piece of advice for founders who are building with AI today? [49:56] Thank you. [49:56] I don't know. I think it depends. [49:58] I think the founders are doing well. I think the smaller companies are doing well. [50:02] I think it's the big companies that should stop [50:05] discarding this as [50:08] Bitcoin or some kind of temporary trend. Now, we're not going to get into the Bitcoin because I know some of the people on this call have different opinions, but... [50:17] But, yeah, I think that's the important, like, don't, like... [50:23] Be cautious. Lean into it. Try it. [50:26] Test it yourself. [50:28] Explore it. Learn it. Don't fear it. [50:31] Just try to learn. [50:32] I think that's the best thing. [50:34] You know, and you can always start talking about yeah, what about AGI in the world? [50:38] will end and this and that. [50:40] Like, I get it. I can also sit down at dinner sometimes and talk about these things because they're fascinating. But in the end, like a meteor might hit us in, hit me in the head tomorrow as well. Like, I mean... [50:49] Things can happen, you can't predict these things. [50:51] So the only thing you can do is try to lean in and learn and explore. That's my opinion. [50:56] At least try to understand it better. [50:59] Awesome. [51:00] Sebastian, thank you for doing this. [51:02] Thank you for having me. [51:04] *music*