Episode 16: $500 Million in Tokens, AI Slop at Interview & Trigger’s Broom 🍺
Week 16. The pantomime horse rolls on — and the transcript called it pantyhose again. Neil is blaming his Northern accent. Kieron is firmly in the back end. Nothing has changed.
Someone burned $500 million of Claude tokens in a month 💸
An unnamed organisation gave all staff unlimited access to Claude Enterprise with no budget limits, no kill switches, no guardrails. By the end of the month, they’d burned through $500 million of tokens. Presumably the CTO is available for “new opportunities”. KnowledgeFlow builds trigger warnings and kill switches into every deployment as standard. This is precisely why.
Kieron’s big talk 🎤
Next Wednesday, Kieron is on stage in front of 60 senior leaders — MDs, CFOs and more — at a large private sector organisation in an industry he doesn’t know well. The brief: inspire them to do something with AI on Monday morning. Last year’s effort was apparently too technical and too salesy. Kieron’s approach: Nokia, Blockbuster, Kodak — the companies with brilliant moats that simply didn’t see what was coming. Then: how hard would it be for a new entrant with zero capital and great AI to disrupt your supplier and customer relationships tomorrow? And a use case matrix: 22 different applications mapped against feasibility and business impact. The challenge, as always, is getting people to hear a use case from a different sector and think “that applies to us” rather than “we don’t do that.” Tune in next week for the verdict.
BidWriter in bid writing season 📋
It’s peak public sector tender season. The one where the commissioners are releasing everything before summer so they can sit on a deckchair while everyone else frantically responds. Neil’s been deep in BidWriter and the Agentic version with Donald. Key discovery: GPT 5.4 mini writes answers that are too short and too succinct. Asked for 2,500 words, it produces 900 and tells you it was being efficient. GPT 5.2 works better for bid writing. The latest model isn’t always the right model for the job. Next week: a live demo for a large organisation using their own 200-question tender as the test.
The “build it themselves” customer 🔧
Kieron’s been in a five-month scoping process with a charity that wants a public-facing RAG assistant. Five months of free consultancy, steering group sessions, demos, statements of work, and every time Leading AI provides guidance, the customer takes it to other people for more opinions. They also want no vendor lock-in and the ability to lift and shift the whole platform at contract end. Kieron’s honest response: I don’t know how you’d do that. If you want something that bespoke you’d have to set up your own company. And if you want cheap (because you’re a charity with no money) then you need to accept an out-of-the-box solution. The lesson: we need to price in the complexity upfront, or both sides end up miserable.
Measuring AI by numbers of prompts is nonsense 📊
Kieron was at a university AI champion session this week where the IT director showed Copilot usage stats: 785 users. Of course the numbers look high. That’s because everyone’s automatically logged in whether they like it or not. A single prompt in a KnowledgeFlow policy assistant probably saves 20-30 minutes. A single Copilot prompt probably saves 3. Usage numbers tell you nothing. Impact is the only measure that matters.
Leading AI is now a Claude Partner 🤝
Donald announced it. He just did it and told them afterwards. What does it mean? Still working that out. But model switching between OpenAI and Anthropic inside KnowledgeFlow is becoming increasingly important. And, as Nate B. Jones (their AI guru) says the frontier model race is now a two-horse race: Anthropic vs OpenAI. Google and Meta have dropped off his radar entirely.
Trigger’s Broom and company culture 🧹
Kieron muses on Only Fools and Horses: the ancient broom that’s had four new heads and three new handles… but is still the same old trusty broom. So, when does a company stop being the company it was? What holds culture together when almost nobody original is left? Media and advertising agencies see almost 100% staff turnover every few years, and yet the culture somehow persists. Something to muse over at the weekend.
AI slop at interview 🤦
Neil interviewed someone for a finance role this week. First question out of the candidate: “How has remote working impacted your revenue and culture?” Neil’s reaction: as a finance professional, you should probably know that remote working impacts cost, but is unlikely to impact revenue. He asked if they’d used ChatGPT to write the question. They denied it. Neil’s verdict: if they didn’t use AI, they shouldn’t be in a finance role. If they did use AI and didn’t check it, they still shouldn’t be. AI slop at work… in both senses of the phrase.
Neil ends the episode by blowing smoke up Kieron’s backside. The Audient is presumably still drooling on his pillow somewhere.
Two mates. A bar. Thirty years of business between them. And all they want to talk about is AI.
Pull up a stool — we’ll get the beers in. 🍺
TRANSCRIPT:
This week in Leading AI… #16 – 9 Jun 2026
Neil Watkins
OK, then, well, let’s start, shall we? Let’s get this pantomime horse of a podcast underway for week 16, and yeah.
Kieron White
Here we are again. Here I am still in the back of the pantomime horse.
Neil Watkins
Yeah.
Kieron White
How is it up the front?
Neil Watkins
Well, as I said, but it seems fine because as I mentioned previously, sometimes I think it’s my accent actually that the transcript can’t work out. It called it panty hose again last week, which is like, oh, crikey, it’s really bad. So anyway, moving on swiftly, the Pantomime Horse of a podcast for week 16.
Kieron White
Nice.
No.
Neil Watkins
I’ve got a bunch of things I would like to talk about this week. I’ve had some really interesting customer conversations. I’ve been doing some more work with BidWriter and the latest versions of that, which are pretty spectacular.
I’ve got another customer story where basically they want to do things themselves and I’ve been trying to point out to them the challenges around that. And then I’ve got a little funny insight to finish off. So those are the things on my list for today. Kieron, what have you got for you?
Chat.
Kieron White
I have. So on my big thing that is on my mind a lot at the moment is a talk, a stage conference session I’m running next week with a large private sector business with all of their senior leaders in the room. So that’s 60 people and they want to talk about AI and how it helps them.
in an industry I don’t know very well. But so I’d love to talk a little bit about what I’ve been thinking around that and how to pitch it. It’s difficult. I also would love to talk about how we judge which customers we should be taking and not. So we’ve had, as you know, a challenge through the last couple
Neil Watkins
Mhm.
Kieron White
the last five months, in fact, of scoping something with an organisation. So I want to talk about that and the challenges with that. You’ve just prompted me that I’ve spent quite a lot of time this week on embeddings, strategies and sort of metadata that goes with those. I’ve been talking with Ibby in our
Neil Watkins
Yeah.
Kieron White
dev team about that a little bit, so I’d love to explore some of that stuff and a couple of other bits and bobs along the way. So buckle up and I shall get firmly in the back of the pantomime horse.
Neil Watkins
Well, maybe you should move up front and start with your private sector AI presentation. So that sounds like a challenge. So yeah, tell us a little bit more about it without naming the customer.
Kieron White
Haha.
It’s a huge challenge.
Yeah, indeed, and I will keep them. So the challenge is 60 people, I am told, I don’t know them, I’ve met now four or five of them, five different individuals, which is great, and it’s lovely that they’ve spent some time with me to kind of help me understand. But obviously, you’re always very aware that you’re getting their perspective on the rest of the group, and that may or may not be.
actually where everyone else is and within where they are in AI. So for me to know where to sort of start really and what to cover is really challenging because I’m trying to get that message and I’m told there’s certainly the ones I’ve spoken to are all pretty adept and are using some good tools and
Neil Watkins
Yeah.
Kieron White
kind of have a good enough knowledge, I think, of what it can do and help with. I’m told others will have none at all and barely used it, might have just played with Copilot a little bit. And so kind of trying to work that out. It’s A 90 minute session and I’ve pitched it to them as a working session.
so that they will leave with some ideas they’re going to go away and work on with their teams on Monday morning, that kind of style. And that’s what the CEO of the group wants. They had a crack at it last year, AI, and I’m told it was too technical and a bit too
advanced, the sort of positioning and a bit of a sales pitch. So obviously I’ll be keeping well off of that. So I really want to add value to them. And I think so what I’m here’s what I’m what I’m planning to do is do a little bit of the why AI, why now, why do you need to do it? And I’ve got a slide at the moment in my deck with the Nokia Blockbuster and Kodak.
Neil Watkins
Mm.
Kieron White
logos, which I quite like, because they kind of all of these had good moats, they had customers, they had products, and they just didn’t see what was happening, didn’t believe it, didn’t or whatever. And even in their businesses that with the organisation that I’m speaking with, they have customers, they have physical product,
Neil Watkins
And.
Kieron White
and they’ve got supplier relationships, all of those really important. But I’ve got a slide talking about how hard or easy is it for a new organisation to arrive tomorrow with no capital, with amazing AI systems running everything. How much is your moat enough? And you know, initially you think, well, yeah, we have to ship millions and millions of pounds worth of products around the place.
Neil Watkins
Yeah.
Kieron White
Well, okay, but your supplier relationships and customer relationships, they could be presumably quite easily disrupted by someone that had a lot more data, a lot faster data, a lot more personalised approaches. So I’m going to do that, then I’m going to spend a bunch of time on use cases in their sector, which I’ve done some AI research around
Neil Watkins
Yeah.
Kieron White
what’s going on right now. I’m going to show them a couple of things that we can do on KnowledgeFlow that will, I think, play to what they have to struggle with every day. And that’s the bit I don’t normally do in any of my talks is sort of going out there on there’s 22 different use cases on a matrix. So I’ve got that kind of
feasibility versus business impact. So you’ve got some really advanced stuff, but really some really basic things as well, like meeting summaries and that sort of world. So I want to kind of get across some of that, but I’m kind of worried about that section as well, because I want them to use it to stimulate their thoughts.
Neil Watkins
Yeah.
Kieron White
not see it as a list of things they’ve got to choose from. And I’m hoping that I can get that across because my experience of that is people aren’t very good in the space of kind of like hearing a story and thinking, oh, hang on, if you applied that to this bit of my business, that would be amazing. People generally, in my experience, tend to hear the thing and go, well, we don’t do that too.
Neil Watkins
Yeah.
Kieron White
Why would I care?
Neil Watkins
But I think one of the benefits of this group, I don’t know them, but I understand there’s lots of managing director and CFO types. So you should have a bunch of bright people in the room. And, you know, in every cohort, there’s going to be some who get it and some who don’t. There’ll be a bell curve.
and, you know, appealing to those. I think it is a difficult pitch when you’ve got something like that, especially in a, how should you say, more traditional industry, which isn’t necessarily used to that kind of disruption. But if some of the guys are already using tools and doing things, then, you know, you’re going to have some bright people.
I think you’re also going to have the challenge that some of them will think they know best and they’ll want to be pushing their own solutions in order to promote themselves. So you’ve got a really tricky balancing act with all of that stuff. But you’ve dealt with more tricky situations than that and it’ll be fine. It’ll be fine.
Kieron White
Yeah.
I have. I have definitely dealt with more tricky situations. I’m just really keen for it to add value to them. That’s the thing. I want them to leave feeling inspired to do something and I’m giving them some more of the out there examples as well for those that are already well on the journey and have got their data organised and they’ve got a load of automation in place already.
Neil Watkins
Yeah.
Kieron White
I’ve got a couple of like use cases which are really kind of out there that will make a big impact if they can get to that. So we’ll see. But yeah, so next this time next week, I’ll be able to give you the feedback or maybe I’ll just be dead somewhere in a ditch. I hope not. That would be a very bad outcome.
Neil Watkins
Excellent.
Ha ha ha.
No.
That would be a very bad outcome. I don’t think it’s going to be that bad. No, and I, well, I think there’ll be a lot of learnings from it. And actually, it’s wrestling with the tricky characters, isn’t it, that always you get the most out of, people who challenge you and make you think and actually push the boundaries. That’s where most of the… A wise man once said to me, Kieron, if something’s easy or something is hard,
Kieron White
But…
Neil Watkins
Do the thing that’s hard because you’ll get more out of it or something along that. I think that’s what you said, isn’t it?
Kieron White
So.
Frightens you. If there’s two courses of action and one’s frightens you, do that one. No, you’re right. You’re right. It will be good for me. And it is good to actually to dust off these slides literally from start again, because, you know, you fall into a bit of a trap speaking at conferences. There’s easy to, you know, I know I’m comfortable with my slides. I don’t even really need to think about it before they come up on stage and I know what I’m going to say, whereas this
Neil Watkins
Yeah, exactly, exactly. Cool.
If.
Kieron White
different so I’m going to have to get right down and prep again. So I think that’ll be a good thing in the long run. Just a bit alarming for next Wednesday.
Neil Watkins
Yeah.
Well, it’s always good to have some butterflies. I think the trouble with overconfidence is it’s always it always comes just before you trip yourself up.
Kieron White
Indeed, yes, pride and a fall and all that stuff. So you’ve been wrangling with customers this week, or at least on…
Neil Watkins
I look forward to it.
A little bit around. I’ve been having lots of really interesting conversations. I’ll share a couple with you. One is around Bid Writer. And I’ve got a couple of different Bid Writer stories because that seems to be the most popular thing right now. It’s just bid writing season.
Certainly in the public sector, it’s common for a whole load of bids to come out before the summer so that whoever’s putting out the tender can go off on holiday and sit on a deck chair somewhere with a cold drink while everybody else is slurving to get their bloody responses in for the end of the summer holidays.
Kieron White
Yeah.
Man, Dave.
Neil Watkins
So, but they all joking aside, so we’ve got a we’ve got a presentation next week for it, and it was a bid that we we used BidWriter to bid for, and I’ve mentioned it on before in the podcast. Have you used have you used AI in the way to this bit? Of course I have. It would be rubbish otherwise, wouldn’t it? So, anyway, we we’re no surprises with with three to the.
Kieron White
Yep.
Neil Watkins
to the demo stage. And what they said was, we want an hour and a half of your time, okay? We want an hour’s demo and half an hour of questions. I was like, hang on, you want to know what, an hour’s demo on BidWriter? It’s like 10 minutes. The whole point, the whole point is that it shortens the bid writing process and you press a button and it gets you 60 to 80% of the way there.
Kieron White
Haha.
Neil Watkins
within an hour and instead of two weeks. And so, you know, we’re going to, we’re literally going to load up some, we’re going to show you some files that are they want, they wanted a demo. So I’m going to, I’m going to use their exact forms that they set with the with the nearly 200 questions that they posed to us.
Kieron White
Mm.
Neil Watkins
So, I’m going to use that as an example, and then just press the button and do a short cut down version that will only take a few minutes just to prove that it works, and then and then show them the rest of KnowledgeFlow and all the other good stuff that it can do, and the you know, perpetual storage and all of that good stuff. But it’s like the bid writer piece is like, right, well, there’s your first, there’s your first version.
Kieron White
Yeah.
Okay, yeah.
Neil Watkins
So now that’s now setting that round the team to sprinkle their knowledge and all the bits that they haven’t written down, like, you know, who’s the latest customer or the case study that they’ve got on their laptop or, you know, the latest financial numbers, whatever you want to put in. But here’s here within the first hour is your, but we’re just not, it’s doing, it’s working in a way. You don’t, I mean, we can have a cup of tea while you’re doing that or the.
Kieron White
Yeah, exactly.
Neil Watkins
beverages available. So it’s really odd to me that they wanted an hour’s demo. It’s like, who’s going to do that? Bid writer. Anyway, good luck to them. But the thing that’s been really interesting is playing with the Agentic bid writer with Donald, who, as you know, is a bit of a genius in this area, but we’ve been playing with different models and
Kieron White
If the…
Well, that would be fun.
No.
Yes.
Neil Watkins
There’s a couple of pieces around this that are super important, I think. So we switched up to 5.4 mini a little while ago, and the responses we were getting weren’t just weren’t right, just kind of too short, too succinct, didn’t expand on the
Kieron White
Mhm.
Neil Watkins
didn’t expand on the answers that we needed. And so we switched it down to 5.2. And actually, we got better responses, but we’re still having problems. And turns out, the more you kind of dig into this, the more you understand the way that the different models work, they are driven by writing very short, very succinct answers.
And so for example, there was a question for 2 1/2 thousand words and the first answer I got back was like less than 900. So I’d used less than half of the word count. And I was like, why didn’t you go, I specifically told you to go up to the word count. Why didn’t you go up to the word count? He said, because long answers can be waffly and I was writing it to be succinct and clear.
Yes, but you haven’t included all these examples and things and why not? Well, I can include those if you want me to. Yes, please. Why do I have to? So you have to fight with the thing in order to get it to do the thing you actually want it to do. But the models, the difference in the models was really interesting. And if I compare that to the stuff that we’ve been doing in Claude and the differences between 4.7 and 4.8, some of the
Kieron White
Yeah.
Interesting.
Neil Watkins
stuff is brilliant. Absolutely. I mean, it only came out what last week or was it this week? I don’t know which week it came out. They change all the time, as you know, but it’s really, I mean, really, really good. But it’s only good for some things. It’s not good for everything. And the word on the on the on the AI wires is that it is not as good as some of the other things. And it was
Kieron White
Yeah, yeah, yeah.
Last Friday.
Neil Watkins
it was a release brought out to line up with their IPO announcement because they wanted to say, look, we’re moving things forward. So there’s a whole piece, I think, for people about the latest number isn’t always the best model. It’s about using the right tool for the job and thinking about
Kieron White
Yeah.
So, I heard.
Neil Watkins
how we help customers do that because their natural reaction is just to want the latest number. They just want the latest thing because it’s the latest. And actually that’s not necessarily true. Which links to another customer I spoke to earlier in the week who basically said, love you bid writer, it could really help us. But we’ve just got a new CTO and they want to do it in
per pilot. And so, you know, that’s not an unusual story for us. So I said that, all right, I’ll jot a few things down for you to take to the CIO or CTO, whatever the job title is. Just some things to think about in our experience of doing this for the last three years. And I talked a lot about some of the things that we’ve discussed in the past on here.
Kieron White
Yeah, indeed, yeah.
Neil Watkins
around things like chunking and the importance of understanding that and how you structure the model so it pulls the information back correctly, how you ingest, where you ingest from, what you’re doing, are you doing, do you need OCR for optical character recognition for those of you who don’t know what that means?
Kieron White
Mm.
Mmh.
Neil Watkins
for our audience, in case he’s still awake. And things like which cloud hosting solution you’ve got. So for example, just be aware that if you’re using AWS, it can be 10 times more expensive than Azure. So be really careful. You make these choices.
Kieron White
Yeah.
Neil Watkins
and then you repent at leisure afterwards. So there’s a whole suffer on that. Then I got into some security things, which are obvious, and any good security chap should know about them. But what I majored on was the ongoing maintenance piece. And we’ve had this with so many customers, like, you know, how do I keep my documents up to date? How do you do it now? You know, what are you doing now to keep your policies up to date, et cetera?
Kieron White
Yeah.
Yeah.
Neil Watkins
What about quality and evaluation? We talked last week and we will come back to many times, I think that whole, and how do you evaluate, have you got a standard set of questions that you can test responses against and indeed test models against, which is what we were doing with 5.2 and 5.4 mini. That whole kind of things like
Kieron White
Exactly, yeah, how do you know?
Neil Watkins
keeping permissions up to date? What happens if somebody leaves? Can they still get in? You know, what happens if you get, you know, a poison code or a poison document injection, which, you know, tells you to do that things or actually passes information out? How do you do that? And then the other thing, which I did finish on, was the whole
cost piece because we’ve seen prices starting to rise and last week we talked about Microsoft shutting down their clawed options because of the rising costs and it becoming unsustainable. So cost is a massive issue. But my summary to this chat was, it might take just a few weeks of engineering, but
It’s the kind of, if you don’t get it right in the 1st place, it will go wrong, it will die on a vine, you’ll waste a lot of time and money, but you will also, if you don’t keep it up to date, if it’s not maintained, if there isn’t a commitment to maintenance, then you’re going to be doomed as well. So really interesting, that whole kind of corporate view.
Kieron White
Yeah.
It is interesting because it really feels to me like it’s coming from a place of massive ignorance. And it’s that whole, I mean, this new technology that’s around. We’re all in this sort of, what’s it, unconsciously incompetent, isn’t it? That’s the first stage of the learning cycle. And we’re all there, but then you have a bit of a play and then you think, oh, that’s good.
Oh, look at that. That’s magic. And I think, I mean, this particularly Copilot Studio is the thing that mostly alarms me because that’s the rag model of Copilot where people are, you know, putting their document sets in and then running that and deploying that to staff and have no idea how it works or importantly, as we’ve said before, what to do when it’s wrong.
Neil Watkins
Yes.
Kieron White
And I mean, I just imagine, I can’t imagine what an IT, even a really good geeky IT fella or war lady, I can’t imagine what they’re going to do when they’re told this is incorrect. I don’t know where you’d go with that. I mean, you obviously start running it through AI and ask them what you should do. But in Co-Pilot Studio, you then can’t get at the settings that you need to
Neil Watkins
Mm.
Kieron White
Unless you’re a developer mode, you can’t get into the bits where you need to be. So I do feel it’s all, yeah, it’s kind of, I don’t know, maybe it happens every time there’s a new tech and people kind of get overconfident. Maybe it’s that whole, we’d rather, we always want to build and own it and have an enterprise and have my empire rather than outsource to people with specialists.
Who knows? It is crazy. But I’ve had on a similar conversation at one of our university clients I was with this week in their AI champion session. So they’ve established a lot of AI champions, which is great. And we’ve got KnowledgeFlow in there. And they also use Co-Pilot, which is perfectly fine because Co-Pilot lives in all of your documents now with Microsoft busy.
Making it, what should we say, easily accessible is one way of saying it, isn’t it? Or like bloody clippy everywhere you turn, you get a , anyway, they, so they’re using Copilot a lot for sort of general productivity, and their IT director introduced, he did a little update on how they co-pilot usage, the was, and that kind of thing, and then…
Neil Watkins
Omnipresent.
Kieron White
talked about the difference with KnowledgeFlow. And I thought he did a really good job of, in a very short introduction of, you know, Copilot is general productivity. KnowledgeFlow is the specialist Agentic tool that is there with a bunch of agents which know all about the right data that they need and the process, whether that’s creating a new
degree programme, whether that’s policy kind of answers for students or staff or their annual monitoring review or their APP process or a whole range of other tools that they have. Each one of those agents knows all about that, has got access to the data it needs and only the data it needs, not everything anywhere. That means you pick up all kinds of
Neil Watkins
Mm.
Kieron White
dirty old data. And so yeah, I thought it was a nice intro that he kind of did to talk about the difference, which is great. And then the big challenge, of course, is when you measure use, Copilot, his Copilot has a load of use. Of course it did, 785 users. That’s because everybody’s already logged in.
whether you like it or not, you’re logged into it. So you’re a user. Secondly, it’s all over everything, so you’re only going to touch it by mistake or whatever, and it shows me too.
Neil Watkins
That’s right.
Which I do all the time. I don’t, you try and write an e-mail or something and you click slightly the wrong place. It opens up copilot. Go away, I just wanted to type.
Kieron White
Yeah.
Yeah, exactly. So all of those probably counting. And I think, but the real point is measuring just purely on numbers, because a single, as I said to them in the conversation, a single prompt in KnowledgeFlow into, say, a policy agent, or assistant, as we would call them, and not Agentic, by the way, those things, we do have Agentic stuff, but
Neil Watkins
Yeah.
Hmm.
Kieron White
they aren’t, just to be clear. But a single prompt in there probably saves you 20 minutes, half an hour of looking around or asking people or having a waiting for your next week’s meeting or forgetting all together. Whereas a single prompt in co-pilot probably saves you about 3 minutes. So I think just the simple measuring of the
Neil Watkins
Yeah.
Kieron White
numbers of use is really not sensible. And I think the interesting thing, I think I read something about this this morning, is we need to get to the point which CFOs are already at now, I think, or getting there, of measuring the outcomes and the impact. It isn’t about
It isn’t about how much use have we got and how many tokens have we burnt and isn’t that great. It’s are our staff staying and liking their job more? Are they more productive in their job and are we delivering better outcomes for citizens and customers? That’s the real measure. And I think if and when people get to that point,
Neil Watkins
Mm.
Kieron White
that’s where KnowledgeFlow is going to show itself as the platform you need to be on. So I really think so. But on your pricing, just quickly, so I think I touched on this last week about token cost in Claude, and this is a real problem for Claude, is there is an organisation and they have remained anonymous and you’ll see why in a second.
Neil Watkins
Yeah.
Yeah.
Kieron White
They burnt $500 million of AI tokens in Claude in a month because they gave all staff access to Claude and they had no restrictions and they did not set a limit on their account. 500 million. That is unbelievable.
Neil Watkins
Ch.
Wow.
That is the ex, the ex-CTO looking for a new job. That’s incredible. Can you imagine?
Kieron White
But.
That is, isn’t it? And it and our blog, our blog post this week, this weekend is on the challenge of if you give people unlimited access, then guess what happens? People are asking what the weather’s like and getting it to do all kinds of things it needn’t bother with.
Neil Watkins
Yeah.
Kieron White
But, yeah, really interesting.
Neil Watkins
What should I get my wife for her 60th birthday? That kind of thing.
Kieron White
I was probably a really good use.
Neil Watkins
Yeah, yeah, but yeah, whoever’s in charge of the payment doesn’t want to be paying for that, do they? That’s just an incredible amount of money. At what point did the alarm bells not start going? Because actually,
Kieron White
Ohh.
Presumably when the invoice arrived.
Neil Watkins
Well, but, but, but, well, we said we said we get trigger warnings, don’t we? When we go when we go above a certain amount, it just…
Kieron White
Yeah, we get, yeah, we have, we have trigger warnings and we have a kill, a kill level. Yeah, we’ll stop a certain burn on our clients once just in case someone tries to go wild.
Neil Watkins
Quill switch. Yeah.
Did you just, I tell you what, let’s just put it, we’ll put it at a billion dollars, shall we? Just, who does that? Who does that? Crikey, that won’t, that won’t remain secret for long, will it?
Kieron White
Yeah, maybe that was it, yeah.
It is crazy, yeah, but how are you?
Well, and we’re back into the world of ignorance, where someone has said, we can do this, we can get a Claude licence and then Claude Enterprise license, and I’ll have it in, look, it’s magic. And just, you know, the first things we do is like data retention, for example, how legally required to define how long you’re going to keep your audit logs for.
Neil Watkins
N.
Kieron White
No one asks that. No one’s thinking about that. What are the budget limits that we need to do? Is it appropriate for everybody? Do we want role-based access? Loads of questions you need to really think about before you suddenly smash something into. Well, they’ve learned a very valuable lesson. I imagine they’d be very good if you ever wanted advice on it. So
Neil Watkins
Yeah.
On my phone.
In here.
That’s right, there’s some, there’s a book and a film about that somewhere.
Kieron White
Yeah.
Indeed, yeah. So you have, well, you know, you share the news of our wonderful Claude announcement.
Neil Watkins
Correctly.
Well, it’s all Donald, really, isn’t it? He just does these things and he doesn’t bloody tell us. And then he goes, ta-da, here we are. We’re now Claude Partners. That’s fabulous, Donald. A, when did you decide to do that? And B, what does it mean? We don’t know. He’s still going through the training programme. And he sent a great little image of somebody flicking through pages really quickly trying to read.
Kieron White
Good.
Yeah.
Yeah.
Neil Watkins
to catch up. So yeah, we’re not quite sure what it means yet, but I do think it is super important for us to be able to do that model switching within KnowledgeFlow. I get asked about it on a weekly basis, probably you do as well, but that whole kind of, you know, what’s the best tool for the job? And actually, if we could have a governing agent sitting above and then choosing between,
Open AI or Claude or whatever else. Interesting enough, I don’t know whether you heard any of this stuff on the AI wires this week, but our guru Nick Jones said that it’s a two-horse race. That’s really interesting. It’s not so long he was talking about Google and Meta. He dropped Meta quite a while, stopped on that Meta quite a while ago.
Kieron White
Mm.
Neil Watkins
But he loved the whole kind of nano banana and the Gemini stuff when it first came out, but he stopped talking about it. And he and he and he used the words two horse race this last week. And it made me just sit up and think, if that is the case, that’s really interesting. It’s just between anthropic and open AI. And loads of, I mean,
Kieron White
Yeah.
Neil Watkins
as much stuff as you can read as you ever want to read on this topic on the internet. But talk of mythos already being used in cyber warfare, even though the US government’s declared them a threat to national security.
Kieron White
Is that right?
Neil Watkins
Yeah, so like the kind of, gosh, double standards, double counting, I don’t know how, whatever you want, whatever you want to call it, but…
Kieron White
Yeah. Apparently, the, yeah, the Mythos thing is one of the rumours are the reason it’s not released is they’ve got the security concerns, but is that there’s not enough compute to run it, and so they’re trying to limit who can have it for now. Interesting. I expect there’s large truth in that.
Neil Watkins
Yeah.
But people will be seeing that already. I’m sure that is true. I mean, the computer on these things is massive. And, you know, I do get kicked out of Claude on a regular basis. It says, you know, you’ve run out of tokens for today, come back tomorrow. And, or upgrade to a much more expensive enterprise version.
Kieron White
Mhm.
Okay.
Neil Watkins
And actually, I think that is the bit that people have just, yeah, they’ve taken the whole, it’s kind of free or it’s nearly free too literally. And the prices are going to go massively. I’ve said that before on this podcast, but I think it’s just something that we and our customers need to be super aware of is that
you know, this stuff isn’t, it isn’t free and it’s going to get more expensive. And I think the real, the real challenge for lots of organisations, especially those customers, who should we say are a bit more challenging sometimes, and you know, the ones that are AI deniers or even AI curious, you know, they’re going to get left behind by those who invest in
in AI and the whole speed thing. You mentioned it already. But I do think, you know, the speed kills. It definitely does. And keeping up to keeping up to date is so important. And people just just ignoring it is absolutely the wrong thing to do. But then I would say that, wouldn’t I? But I believe it to be true.
Kieron White
Yeah.
Yeah, I did.
But I think you’ve got to have the strategy now. That is what I mean, the point for that talk I’m going to do next week is that you’ve got to have a strategy and your strategy might be move slowly. We’re not going to do anything this year or that’s fine if you want to say that. I don’t think it’s fine, but it’s at least a strategy. I think you’ve got to now decide where
Neil Watkins
Yeah.
Kieron White
what is the part AI can play and how are you going to go about looking at your business in a way that, frankly, a disruptor can turn up and go, right, thank you very much. We can do anybody shipping electronic stuff, anything that could be shipped by easily, by, you know, either because it’s software or just knowledge or whatever. I mean, all of that stuff is just…
Neil Watkins
Yeah.
Yeah.
Yeah.
Yeah.
Kieron White
hard to see how that survives. You just have someone turn up with a better version of it, with more personalization probably. That’s, you know, that whole thing we’ve talked about before, about being able to, instead of here’s the training, you can have here’s the training as it applies to everything you do. And that’s like, we can do that now, right this second, and it would be amazing.
Neil Watkins
Yeah.
Yeah.
Kieron White
And, you know, so what, you know, I don’t know what the future of that kind of world of trainers that are just given blanket training. Run this thing and it will know all about your business and the training will be exactly personalised to what, how you, the things you need to know.
Neil Watkins
And.
I.
Yeah.
Yeah.
Yeah, I think if we can crack that knowledge layer, the knowledge graph, the context layer we talked about, the knowledge, the ontological knowledge graph of semantics. Yeah, that’s right.
Kieron White
Yeah, contacts layer, yeah.
Semantic taxonomy. I got into trouble today at Stand Up on that to you, as you all know. I was, well, Ibby, I was talking to Ibby about it because I know I’ve flooded, I flooded Donald with all kinds of things and I was chatting to Ibby about it and they said, like, just have a little think.
Neil Watkins
What did you?
Oh, yes, yes, yes. No, you did. Yeah, yeah. Now we’re going to say now.
But.
Kieron White
You don’t you don’t need to talk about it yet, and I’m not asking you to do it, but, and then in stand up he said his number one priority today is working on the knowledge graph, and and then Donald’s eyes were like, “What? I’m going to get, no doubt, Donald will shout at me later, but, and deservedly, I deserve that.
Neil Watkins
Ha ha ha.
Indeed.
Yeah.
Kieron White
Two, I’m sure.
Neil Watkins
Don’t worry, he’s been shouting at me all day about the Agentic bid writer. I keep going. And another thing, is this, can you just do, actually, and that’s a problem, that’s a problem for us and it’s a problem for our customers, isn’t it? The customers who keep going and can it do this other thing and can it do this other thing?
Kieron White
Not sure.
Yeah.
Well, let me tell you, let me tell you about my frustrating customer. Though I always have to keep this very, what’s vague. But so back in January, I think it was when I met a customer that was wanting to do something that involved a rag AI assistant.
Neil Watkins
Secret. Yes, indeed.
Kieron White
being publicly available. And I was like, great, yeah, no worries. That all sounds pretty straightforward. And they were like, well, can you give me an idea on costs? And I was like, well, how many people will use it? Well, I don’t know how many people will use it. Well, I can’t really tell you about how much it’s going to cost you, can I? And they’re like, well, can’t you just give me an idea? And I was like, well, I can give you an idea. So I’ve given them an idea. Anyway.
Neil Watkins
That’s really quite odd.
Kieron White
Five months later, and I’ve been to various meetings with them. I’ve given them, we’ve created a demo for them. I have explained to them how they should run a steering group session and then joined that steering group session on how they could best gather the information they need for this thing. I’ve, I mean, I’ve days and days of work as part of a kind of sales cycle.
And it seems to me every time I give them something else, they then go away and talk to other people about that thing. So they’re kind of using free consultancy advice from me a lot. I don’t mind because I’m happy to help. But ultimately, we’re obviously here running a commercial business. We can’t afford to spend days and days with people. Loads of questions back and forth. And then eventually it was like, right, can you quote against this?
here’s our statement of work. I always am frustrated at that point because it’s like, well, I’ve showed you what to do, talked about it. You’ve now written it back clumsily with some other things which are really difficult. And you now want me instead of sending you a standard proposal, which would be quite simple for me to do, and now I want to rewrite a proposal in a way that answers your question. So
I’m always frustrated in that. And then anyway, so yeah, I have taken the view that we will need to charge them more. And that’s their fault, really, because I know they’re going to be challenging all the way along. I know from the conversations we’ve had, there will be things that they say, well, can it just do this? Can that button just be there? Well, we just want this little thing. We just want that.
And they all the way along are saying, you know, we’re a charity, we don’t have much money, we’re very budget conscious. And then they’re like trying to dream up. I said, as you said to them in the last meeting, that, you know, if you want to be low cost, you need to take off the shelf and KnowledgeFlow off the shelf.
Neil Watkins
Hmm.
Kieron White
is amazing and will do everything you’re talking about. But the fact that you want it slightly branded like this or a button over there that says that or an ability for this thing to happen is all stuff that adds complexity. And anyway, so we’ll see where they go with it. But it’d be nice to work with them. I’d like to help them with what they’re doing, but we need to price in.
Neil Watkins
Hmm.
Mm.
Kieron White
the extra time they’re going to cost us, because otherwise it’s going to be a horrific experience for both sides. And we’ll both feel, yeah, we’ll both have a bad experience. So let’s see, let’s see what they do. And if they choose not to, well, that’s fine too. And at one point I advised them to get a contract and build it themselves because they wanted no vendor lock in.
Neil Watkins
Yeah.
Yeah.
Kieron White
That’s in their statement of work. So they wanted to be able to build it and then at the end of whatever period of contract, lift it and take it somewhere else. And I said, I genuinely don’t know how you would do that. Because it’s like, we’re going to put all the embedding stuff across it in the way that we do all of that. Our settings.
Neil Watkins
Yeah.
Kieron White
How do you just take that and…
Neil Watkins
What you need to do is you need to create your own Leading AI company and then go and find the developers and the CTO and get them to do it. For a charity that claims to have no money, I suggest that’s not possible. Back to the your ignorance thing, you know, it’s kind of, I guess they are trying to educate themselves, but I also
Kieron White
Do I?
That’s it.
Exactly.
Neil Watkins
I am 100% with you on the kind of we need to be we need to be careful about the customers that we take on board, because if we if we deliver a bad experience, it’s the old adage and it takes takes ages to create a good reputation and then it’s to create a bad one, and when we can’t afford for.
Kieron White
Yeah.
Neil Watkins
for anyone to be saying bad things about us. So we, you know, we, I think we do need to pick carefully. And, you know, lots of the advice you get is around picking your customers carefully and not just taking on anyone. I think, I think because AI is such a tricky topic, and by that I mean, it’s potentially, it’s potentially
organisation changing and threatening for lots of people. I think it’s, you know, I still hear the kind of, is the AI going to take my job question on a regular basis. And people genuinely do feel threatened because they don’t really understand it. And so yeah, I think it is tricky. I do think you’re right to
Kieron White
Yeah.
Neil Watkins
I’m not sure the right set of words, pricing, some messing around about factor, but actually, you know, if we could get them into a good place and it’s, you know, a worthy cause and it’s all doing good stuff, then great. But, you know, we’ve got to be super careful. We can’t, it’s kind of the long tail thing, isn’t it, about, you know,
Kieron White
Yeah.
Neil Watkins
how much effort is involved in somebody who doesn’t just want, I know it’s a toaster, but can it do eggs as well? And actually, can you put the coffee on? I know it’s a toaster, but yeah, yeah, yeah.
Kieron White
Yeah, exactly.
Yeah, it’s no use to me at all if it only makes toast. No, I like eggs on my toast. It was a nightmare. So yeah, so that’s been my grappling this week and then a couple of other bits and then you should talk about the, what did you call it, large scale comms I think is the way you put it.
Neil Watkins
Very good.
Yes, right. Yeah, that’s come up a few times, so I’ll contact in a second. You go first.
Kieron White
Yeah, so I did it. Here’s an interesting thing I did today, which is pretty basic AI use, but I just come out of a meeting which we didn’t record, it was just dumb, as usual, forgot not to put the recording on. And then so I had notes and I thought, I can’t be bothered to write those up. So I used KnowledgeFlow and Voice Mode and just read the actions to KnowledgeFlow and to make that into a
meeting note and it did and then I sent it to the what would have been 20 minutes of typing was about 4 minutes of speaking, which is quite a nice way. And there is an app that does a whisper flow, they’re called, which are very good apparently at reorganising your thoughts. You can ramble at it.
Neil Watkins
Nice.
The.
Kieron White
and it will write an amazing blog post or something and it learns with you. So when you correct it with your, when it thinks you’re saying panty hose or whatever, instead of pantomime horse, you can correct it and it’s apparently very good at that. So, but KnowledgeFlow was pretty decent considering we’re not really playing in voice. It did a good job.
Neil Watkins
Yeah.
Kieron White
We definitely picked all the words out, because obviously I can speak the Queen’s English, so OK, this is the King’s English now.
Neil Watkins
Nice.
The King’s English, that’s right, yeah. And you’re not Northern, go and say it.
Kieron White
Okay.
Exactly, I’m not, I’m not, no, I’m not Northern.
Neil Watkins
Very funny. Very funny. Right. Well, let me talk about one of the other topics I wanted to discuss, which was the large scale comms piece. And we’ve had another organisation come to us and they were doing multiple parties in multiple parts of the country, all wanting to get on board with the same set of staff, same set of training, same set of information, same.
quality of communication, style, tone, all of that good stuff. And it just strikes me that it’s just becoming much, much more obvious to people that it’s a great way to go. So I’m genuinely excited about that. And we were chatting before about, you know, if we’d have had this tool
Kieron White
Yeah.
Neil Watkins
It would have been a great way to get, you know, 150 local authorities on board and share the same information, allow them to do the natural language querying and allow them to create localised plans which, you know, reference their staff, but also took the stuff from the centre. You know, it would be a great way of
of running policy. And I’m sure there are lots of very clever and bright people in and around Whitehall doing that kind of thing. But I think for us, finding organisations that want to do that is super exciting. And so yeah, I’m looking forward to the next one and hopefully being able to share some
some information about the real customers in there soon because I think it will make a difference to them.
Kieron White
Yeah, it’s really interesting, isn’t it? And that’s when I think exactly back to those days working with the Department for Education, trying to drive national change programmes. You want people to have the same kind of guidance, but be able to work with it locally. You want to share the best practices. There’s a PMO tool you can put in there. So how are we doing? What are the updates?
Neil Watkins
Yeah.
Kieron White
dates? What are the comms messages? I need to write a comms message for my teachers, staff, whatever. Here you go, use this one because it already knows the context of the… It’s an amazing potential that KnowledgeFlow could offer in that place because you can have a single organisation at the centre kind of driving the data that’s inside it. That becomes the single version of the truth.
Neil Watkins
Yeah.
Yeah.
Kieron White
for everybody that needs it. And as you say, localised plans drawn from the national kind of requirements or central requirements, but taking on board your local context as well. I think it’s just a really, really interesting angle on the world we used to live in and AI bring crash together, though. On the world we used to live in, here’s a reflection for you, right? I was musing over this. Remember old
Neil Watkins
Yeah.
And.
Yeah.
Kieron White
only fools and horses with what’s his name, Trigger’s Broom. It’s like the oldest broom. The oldest broom is, I’ve had this broom for so long and it’s only had four new heads and three new handles in all that time. I was thinking about company culture and that, and that same analogy is like,
Neil Watkins
Yeah.
Yeah.
That’s right. Yeah.
Uh-huh.
Kieron White
And I was thinking back to my media and advertising days consulting there. And the reason for that is a lot of their staff turn over quite rapidly. They generally are two years around and then move on. So you get, and I was like, when is a company no longer the previous company? When is it Triggs Broom? And I was like, and what does that tell you about culture really?
Neil Watkins
Yeah.
Ha ha ha!
Ha ha ha!
Kieron White
Because, you know, that whole kind of change the culture thing everyone wants to do the hardest thing to do. And yet somehow it grips an organisation because I don’t want to name any organisations, doesn’t really matter. But the culture is very similar 10 years later than it was, even though you’ve probably only got three people that are still there.
So, what’s going on? Anyway, there you go, something to muse over over the weekend.
Neil Watkins
Interesting.
Okay, I will. Speaking of culture, I’ve got a final little story for you, which amused me a little yesterday. So I interviewed someone for a finance job. And I’ll let you into a little secret to our audience. If anyone’s ever on this,
podcast is going to be interviewed for me. I always start with the same line of, hello, nice to meet you. I’ve got some questions for you, but it’s only polite to let you go first. So what questions have you got for me? And if they haven’t got any questions for me, that tells me all I need to know. But anyway, this person did have some questions for me. And the very first question off the bat was,
Kieron White
Yeah.
Neil Watkins
How has remote working impacted your revenue and culture? And I was like, what?
Kieron White
Yeah.
Neil Watkins
Those 2 things don’t aren’t natural bedfellows, revenue and culture, are they? Is it really? And I, as I hang on a sec, as a finance person, you should probably know that remote working is probably more linked to cost because you don’t have an office and people aren’t commuting. And so you could argue.
Kieron White
Yeah.
Neil Watkins
very clearly that it brings the cost up, but not the revenue is really interesting. And I said, I don’t really understand where you’ve got that from. Did you use ChatGPT to write that? As the person went, oh, no, no, no, I made it a poll on my own. And then I thought, if you really did make it a poll on your own, you, it’s not the right, this isn’t the right place for you. And if you did use ChatGPT,
Kieron White
Ha ha ha ha.
Neil Watkins
definitely isn’t the right place for you. But yeah, just they did have some other questions which were clearly they had thought about, but there was two in there which I just thought you’ve just made these up and you’ve just taken the first things out and you’ve just thought, yeah, that’s good. That’ll get him.
Kieron White
Right, it’s interesting, isn’t it?
Neil Watkins
It did get me, it got me in the wrong way and it didn’t get you a job, so thank you very much.
Kieron White
Yeah.
Yeah, be careful what you wish for. Yeah, that is interesting. That is AI slop at work, isn’t it? It’s like, great, I’ll ask that. That sounds important. I’ll ask that one.
Neil Watkins
An AI slop, exactly.
Correct, yeah, yeah. This will bamboozle. Yes, it will, but in the wrong kind of way. So, interesting. There you go.
Kieron White
Yeah, yeah, very good. Well, I will watch out then if I’m ever being interviewed by you. Hopefully that won’t happen.
Neil Watkins
But.
Ha.
Yeah, I don’t think, I think you’ll come with better questions. You always have very good questions. You always put effort into thinking about things beforehand and you’re very good at that. So there you go. On that, on that blowing smoke up your backside comment, I’ll let you go. Yeah. Have you got anything else for the weekend or is that as that was done for this week?
Kieron White
Well…
Oh, well, listen to you.
Yeah, I’ll go blush.
I think that’s us done. I’ve got a couple of things that we’ll run into next week and I will be able to update our audience on. I know what will be on the edge of his or her seat to find out how my talk went. Oh, yeah, it got made of his pillow.
Neil Watkins
Excellent.
Or bed, depending on where he’s sleeping, drooling onto his pillow like Homer Simpson.
Kieron White
Very good. Have a good one.
Neil Watkins
Right on that image. Cheers, fella. Bye.
