In this episode, Neil and Kieron cover a lot of ground, starting with Kieron in full rant mode over the idea that colleges could replace KnowledgeFlow with NotebookLM. They get into why that comparison falls apart fast, from data protection and governance to document limits, prompting complexity and the complete lack of audit trails.
They also dig into a string of unsettling AI and cyber stories, including agents deleting an entire TV series, hacked supermarket freezers in France, malware delivered through adverts, and what all of that says about testing, patching and basic cyber common sense.
Elsewhere, they discuss the discontinuation of custom GPTs and Gems, why shadow AI is a fragile strategy, and why enterprise-grade AI needs proper control over prompts, data, security and model changes.
The episode also features a look at a new Education, Health and Care Plan (EHCP) tool built with colleges, designed to help assess consultations more quickly and accurately, with pseudonymisation, legal context and structured decision support built in.
And running through it all is one of the podcast’s recurring themes: AI value is not about prompt volume or token counts. It is about whether the tool helps people make better decisions, avoid mistakes, and get better outcomes.
Expect practical AI debate, a few alarming cautionary tales, and the usual mix of sharp insight and Friday-afternoon humour.
Two mates. A bar. Thirty years of business between them. And all they want to talk about is AI.
Pull up a stool, we’ll get the beers in. 🍺
TRANSCRIPT
Neil Watkins
Right then, should we get this pantomime horse of a podcast underway? Stop faffing about and get on with it, shall we? Because our audience won’t put up with us messing around, Kieron. They want some serious AI insights and they want them from you.
Kieron White
Ohh.
Is that right? Were they? I’m not sure that they’re in the right podcast for that, but we can certainly try. I shall wriggle my way into the back of this pantomime horse outfit. It will be pantomime season soon, actually.
Neil Watkins
Very good.
It will be pantomime season. Yeah, very good. We probably, oh no, I can’t even bear to get an outfit. That’s just too much, too much. Anyway, what have you got on your list for this week, fella? What’s on your list? Tell our audience what you’ve got.
Kieron White
Yeah.
Yeah.
Well, I’ve got…
I’ve got a few things. One, I’m blooming furious, so I’m going to explain why.
Neil Watkins
Blooming, more blooming furious than than usual.
Kieron White
than usual for a hungover Friday. Yeah, I’m very angry about something that I will share liberally and vociferously. So I want to talk about that and then I want to talk about a couple of other bits in the news this week, an Accenture story and the demise of custom GPTs and gems.
Neil Watkins
Okie dokie.
Okay.
Kieron White
both getting discontinued, which is a bit of a blow if you’re mainly looking in our audience as teachers, because they use a lot of those in shadow AI. So I’d like to talk about the implications of that and a few other bits along the way.
Neil Watkins
Yeah.
Well…
Shadow AI is very naughty, as we as we well know. Well, I’ve got I’ve got three international stories that I wanted to touch on, and then I’ve got four or five things from the UK and largely around your manner of housing. I’ve been I’ve been I’ve been messing around in housing this week, Kieron.
Kieron White
Yeah?
Good, good to hear it.
Neil Watkins
And I wanted to I wanted to share some of the things that I’ve that I’ve come across, but but before we before we do that, should we get should we get this crossness off your chest? Should we do the should we do the furious thing first? Let’s get that over and done with.
Kieron White
That’s probably good. It’d be cathartic. So this is, try not to swear. So in college land, FE, there is an organisation that advises colleges and gives them technical advice on what systems and platforms might be useful for them. And they are
Neil Watkins
It will. Don’t swear.
Kieron White
So I’ve been hearing from college contacts. They’re out and about and they’re trying to knock us and they’re telling colleges that knowledge flows.
you can replace it with and get this. This is the thing that made me very cross, is they have told at least a college that they could replace knowledge flow with Notebook LM, the Google product. It’s actually called, I think it’s Google LM now, or Gemini LM or something, or Gemini Notebook.
But so that is a bit like Copilot Studio, although without any of the security features or very few of them. It’s a simple rag model. You load your documents to it. And the Notebook LM thing is brilliant for podcasts. It will create a kind of artificial podcast, as you know. And it’s a really fun, great thing to do. But it is entirely unsuitable.
Neil Watkins
Mhm.
It will.
Kieron White
for a college to use. Number one, the data is not stored inside the enterprise, even if you’re a Google Enterprise customer. So number one, you shouldn’t be recommending that. And you’re a technology consultancy that’s being paid to advise a college. This is what makes me really angry. Two, there are so many limits within it. You can’t add system prompts to it. You can’t have more than 50 documents in any one of them. So
Neil Watkins
Yeah.
Kieron White
All of our, I don’t know, a college hasn’t got at least 100 policies. So you’d have to have two policy assistants, would you? How’s that going to work? So you kind of. Yeah, you didn’t find the answer in that one. Yeah, maybe look in the other one. So that doesn’t work. So that doesn’t work for you. And then, I mean, there’s so many bits that made me cross.
Neil Watkins
Yeah.
I’m going to say the answer’s not in this one. I’ll have to go and cheque in that one. Dead user friendly.
Kieron White
The idea that you could, they was claiming that you could save the entirety of your Knowledgeflow fee. So they’re like finding savings, which obviously is a great thing to do if you’re consulting, say, look what we found, we can save you this much money. Totally failing to add any cost for how you would even try and set, I mean, a Notebook LM trying to do the same stuff we do with Ofsted inspection stuff. It’s just, I cannot imagine a way it could do it. You would need to load a load of your college data to a notebook LM. So now you’ve just sent your data off to wherever it is being processed in the US most likely. And then you would need to write a series of very long prompts. I mean, our Ofsted stuff’s 10,000 word prompt.
Neil Watkins
And…
Kieron White
that across the suite of them are huge and really detailed and accurate and cheque and double cheque and then look at this and you know loads of no normal person is going to be able to do that. Never mind do it with any efficiency. But all that cost of like bothering with that is totally failed to ignore. You could have a free tool instead of KnowledgeFlow. There’s no governance in Notebook and then you can’t see an audit trail.
Neil Watkins
Mhm.
Kieron White
IT manager can see what was going on through it. You don’t know who’s using what. You don’t know whether the documents are the right documents. It absolutely strikes me as amateur to, it’s like, in my imagination, I think what’s happened is whoever’s doing this review, the team, have seen Notebook LM and gone, wow, wow, look at that. Isn’t it amazing?
Well, that can, we should recommend that. And they’ve literally done no further thinking or no due diligence, as that seems to me. And it is, it is, makes me cross because it’s kind of obviously very personal that they’re having a crack at us. But it would make me cross anyway on the basis of you’re making a recommendation in a report, which is a breach of data protection laws, which
Neil Watkins
Yeah.
Kieron White
probably is not possible, I would wage, or at least not very, we take some serious advanced knowledge of prompting and what’s going on in the LLM and the rag search. Because you can’t change chunking in there, you can’t tell it about Excel spreadsheets and have it understand what the data labels mean and none of that stuff’s doable.
Neil Watkins
Mm.
Hmm.
Kieron White
So it’s just absolutely outrageous. There you go. I’m really cross about it.
Neil Watkins
It can, in some ways, it reminds me of the early days of ChatGPT, you know, three, two, three years ago when people go, oh, it’s magic. Look, let’s just do everything. And hang on a minute, let’s, where’s the data and what’s happening to it. So it does feel very kind of basic, amateurish, I think was the word that you used. Interestingly enough, I,
I, I’m not so cross about it, and I think there’s two reasons why: one is…
When you’re, when you’re, when you’re at the top, Kieron, you’re always going to, you’re always going to get people detracted. And Ben has a phrase, which is, if you’re going to swing for the king, you better not miss. And they’ve clearly missed because, you know, whoever they’ve given that report to,
Kieron White
Haha.
Neil Watkins
is clearly going to go, this is complete bobbins and nonsense and unprofessional and useless. So away you go in short jerky movement. So I think they’ve done themselves more damage than they’ve done to us, is the honest truth. And
Kieron White
I hope so. I hope people know that’s what worries me is if they’re saying it and then you’re a college and you’re not going to be that technically able, then maybe you think, oh, okay, well, this technology firm thinks it’s okay, then it maybe is.
Neil Watkins
Yeah, but if that is the case and your IT manager at your college can’t do the very first thing, which is where’s the data stored, oh, then as a college, you’re in much more trouble than you are from anything else. So any half decent college is going to just see through that and any IT manager
Kieron White
Yeah.
Well…
That is very true.
Neil Watkins
these worth their salaries just going to go. That’s nonsense. Now we might get the whole kind of, I can do it in co-pilot thing, but that’s a different thing. And we know that the.
Kieron White
At least it’s secure, but even that’s hard to, I mean, you couldn’t replicate, you couldn’t replicate most of our quality stuff in Copilot, not doable.
Neil Watkins
Hey.
Exactly, exactly. So I get why you cross. I don’t like it when you’re cross. It upsets me. But the reality is, I think it’s I think it’s I think they’ve done themselves more harm than they’ve done us. And, you know, if they if that if that’s what they are trading on,
Kieron White
Ohh, well…
Neil Watkins
then they’re in much more trouble than we are. And yeah, best of luck to them because they’re going to need all the luck they can get. So yeah, I’d just let them crack on, fella, because as I say, any college worth their salt is just going to go, that’s rubbish.
Kieron White
Well, that’s nice to hear, although I’m still cross. And I want to, but I’m also like, what do we do about it? Ignore it probably is what we’ll do, but there is a kind of, do we actually officially do something? I don’t know. Anyway, there you go. Maybe I’ve just ranted on the podcast. It’s my action.
Neil Watkins
Black.
Maybe, maybe, maybe after you’ve had your hair of the dog peroni at 4:00, you’ll be, you’ll be in a better mood.
Kieron White
Yeah, that would definitely be true. So what do you got? Tell us about your international stories. This is an interesting.
Neil Watkins
The.
Oh, yeah, there’s three international stories that struck me this week. And some of them link back, I mean, the first one definitely links back to something you said last week, really, which is the whole, do not set your agents loose without testing because you’re an idiot otherwise. And in the US, an entire TV series has been deleted by a set of agents or several 100 days work.
Kieron White
Yeah.
Neil Watkins
and no backup, no way of recovering. And yeah, somebody somewhere let the agent go. And yeah, they’ve lost their entire, all their recordings, which were due to go out on TV. So Matt.
Kieron White
No way.
Both.
Wow.
No way, that is awful. No doubt they got a, oh, you’re quite right, I shouldn’t have done that. That’s their appeasement.
Neil Watkins
Oh, yeah, yeah, yeah, yeah. Well, I’m not sure how much of these that you can get from it. At that point in time, you just switch the blooming thing off and go to the pub, that it’s just, oh my goodness, that’s just dreadful, absolutely dreadful.
Kieron White
Yeah, geez, can you imagine? That is, I mean, I’ve had those moments where you’ve worked on something for, you know, like 2000 words of document and that goes, and then I start again, that pales into insignificance compared to a multi-TV show.
Neil Watkins
Yeah.
Yeah, yeah.
Yeah, yeah.
Kieron White
Geez, that’s tough. Wowzers.
Neil Watkins
The second one is a hacking story, which is, it’s not directly AI related, but it shows the importance of patching and keeping up to date. So you might have heard this in France, hackers broke in and turned all the freezer temperatures up to 50 degrees C and
Kieron White
The thing.
Neil Watkins
Did you not hear this? Those supermarkets and wholesalers, and they lost hundreds of thousands of pounds worth of food because somebody had hacked in and turned all the freezers up. And it’s that, no, you shouldn’t, because actually it’s kind of, there’s a lot of challenge about hybrid warfare and cybersecurity. And actually,
Kieron White
No.
My word, Jeez.
That’s enough.
Neil Watkins
disrupting the food supply chain. It’s a dead easy test, right? Let’s just go and let’s go. Anyway, the patches have been out to stop the security vulnerability since August. But of course, people haven’t got around to doing it. And then the next thing you know, your rooms are flooded and all your
Kieron White
Yeah.
Right.
Neil Watkins
Or your prawns have gone a bit manky, so you can’t use them. Yeah, yeah, yeah, so…
Kieron White
They’re cooks now. They turn the fridges into cookers. I said, I heard a similar story I read in IT Pro, my daily newsletter. Who knew that I’d be interested in that? On this agent, an ad agency is serving adverts with get this zero click malware.
Neil Watkins
Yeah, listen to you.
No.
Hello!
Oh, crikey.
Kieron White
Without you saying anything, that’s what’s really alarming. You don’t even say yes, you don’t click it, you don’t do anything. You’ve literally loaded a web page or an advert, in fact. That is terrifying. Apparently it’s down, well, yeah, it actually listed some of the target organisations or where they were served. So they were on Duolingo. If you’re on, I don’t do that, but if you’re on there, then there were adverts on there.
Neil Watkins
Ohh my lord.
Which adverts?
Oh, did it?
oh my god.
Kieron White
and a few other platforms that are listed as where they appeared. But they seem to be targeting particular, whoever was behind it was trying to target particular organisations. So obviously they’re not that interested in Duolingo, but they were on there trying to hit Neil Watkins, blah, blah, blah, whatever. Unbelievable, though. I mean, that is terrifying. And apparently it’s something, it’s the ad tech.
Neil Watkins
Yeah.
Yeah, yeah.
Kieron White
that needs to catch up so that they’re checking, the company serving them should have checked and don’t. They’re just like, yeah, yeah, off you go, shove whatever you like out. And then of course, you’re on your own phones and stuff, allowing it to happen in the 1st place. But that is terrifying.
Neil Watkins
Yeah.
And.
Crikey. It feels like there’s not a lot you can do about that.
Kieron White
Well, yeah, I mean, what, yeah, nothing. Apparently, though, you should turn your phone off and on at least once a day.
Neil Watkins
Okay, that’s good advice.
Kieron White
I heard that from a security. I heard that from a security dude a long time ago, like someone, the serious player in government, and they said, yeah, you should definitely, because if it’s got a lot of stuff can get loaded and then it starts listening to you or whatever it’s doing, when you restart it, most of that stuff is gone and they’ve got to try and… Yeah, very wise.
Neil Watkins
Mmh.
Just a second, I’m just going to switch my phone off.
Kieron White
No, because I mean I don’t turn mine off for a week at a time, and basically only when it runs out of battery is the only time it really goes off. And they just gather all this dodgy stuff.
Neil Watkins
Yeah, on the same, I might say.
Might turn it off before I go into the pub so that no one’s listening to what I’m all drinking.
Kieron White
Yeah.
Another bag of pork scratching, sir.
Neil Watkins
I don’t mind if I do. Thanks very much. Why am I getting adverts for pork scratchings all the time? I don’t know.
Kieron White
Okay, don’t click them. We don’t even need to click them. They won’t. Don’t even look at them.
Neil Watkins
Don’t crunch them. I’ve just put on a stone because I looked at the pork scratchings again. My third international story was about Bank of America. And Bank of America have hired 1000 contractors to fix all of the bugs and vulnerabilities identified by Mythos.
Kieron White
So.
Yes.
Okay.
Neil Watkins
Wow, that, can you think about the cost of a contractor in that kind of organisation, in that kind of environment? The cost of, yeah.
Kieron White
No way.
On.
That’s a hell of a budget, isn’t it? But that what? How much did they obviously found? A hell of a lot of stuff then.
Neil Watkins
Yeah. So here’s my question to you. If a bank like them have got to hire 1000 contractors to fix the identified problems,
Kieron White
Yeah.
Yes.
Neil Watkins
What about housing association or a council or a government in the UK, for example? Crikey, the gaps must be hideous. You don’t need, you don’t need specialist adware spy malware because there’s blooming driver coach and horses through the things. So that was quite a terrifying, that was quite a terrifying stat that I heard this week.
Kieron White
Matt.
Mhm.
Yeah, yeah, yeah, absolutely right.
Yeah, and you think because they would have been already pretty well protected. I mean, Bank of America aren’t exactly a small outfit, are they? So they would have already had a load of good stuff and they still found enough to require 1000 contracts. That’s crazy.
Neil Watkins
No.
They did, but it’s not, I mean, this links on to some of the UK stories that I wanted to touch on this week. I don’t know if you’ve heard any of these, but there were four or five, which I thought were really, really interesting. So UK government is planning for when AI agents cause serious harm. So they’re already thinking about it and
They’re already thinking about including or considering including AI loss of control provisions and near miss reporting into the new cybersecurity and resilience bill they’re putting together. And I was just thinking about, okay, AI out of control that.
Kieron White
Right.
Neil Watkins
OK, we can, we, yeah, if you were running an organisation, you might find out about it, but you might not. Because actually, if you look at some of the frontier labs, they’re going, oh, yeah, we found out in June that, you know, the Australian thingy looks back, but we’re not reporting it till September. But the near miss reporting, it’s like, how is that going to work in practise? How do you know it’s been a near miss? What does a near miss even look like? So
Kieron White
Yeah, yeah, I hacked it, yeah.
Mm.
Neil Watkins
you know, for most organisations that, you know, we’re dealing with, you know, it’s AI is such a challenge already and actually having to report near misagentic, and who’s going to want to do that? That’s like saying we almost had a data breach. I don’t want to really report that to the ICO, but
Kieron White
Yeah.
No, indeed.
Neil Watkins
But you might have to.
Kieron White
Because we track those, we have to do those under our ISO standards. We track near misses and, yeah, but typically those are things like someone attempted but was blocked by XYZ, which is those are all counted as near miss.
Neil Watkins
What the near misses?
Yeah.
Yeah, but then I was thinking about the near miss for the agentic stuff, so if…
Kieron White
Yeah, I mean, how do you know when it’s, yeah, when it’s borrowed its way into something? Yeah, yeah.
Neil Watkins
Especially if it lies to you. Yeah, especially if it’s lying to you. It’s fabricating what it’s made, what it’s doing. No, let’s try to go over here.
Kieron White
Are you malware? No, no, no. Over there. Look over there. Jeez.
Neil Watkins
So that was, speaking of hacking and the ICO, did you know that this last week…
Kieron White
Oh yeah.
Neil Watkins
The Housing Ombudsman was hacked, and…
Kieron White
I did see, I did see that, yeah. I couldn’t get complaints through to them.
Neil Watkins
At.
Correct. Their third party complaints handling software was hacked and it was, we don’t know who it is. There’s been nothing in the press. It’s all been kept quiet who the third party supplier was. But it made me wonder about if it’s case management software that was hacked.
then, and it’s in the housing ombudsman. Is it going to be in the housing associations or is it in other ombudsman? You know, where else has been where else has been? So, I thought that was quite salutary, especially given the potential confidential nature of some of those complaints.
Kieron White
Yeah, yeah.
Neil Watkins
and there’ll be, there’s likely to be a lot of personally identifiable information in all of that. So yeah, that’s really, really tricksy and frankly a little bit embarrassing. So security, security, constant vigilance, as they say in Harry Potter, Kieron, constant vigilance.
Kieron White
Yeah, yeah, indeed, yeah.
I’m keeping aware, I’m being aware.
Neil Watkins
You keep being aware. You mentioned something about gems.
Kieron White
Gems and GPT custom GPT. So they are being discontinued. So for reasons that wasn’t clear at all in the article, in fact, even to the journalist writing, it was like, I still can’t find out why this is a decision. Apparently the closest match you can get in ChatGPT is a GPT plugin.
Neil Watkins
Yeah.
Kieron White
but they are not shareable. So any teachers, which I know I hear quite a lot about it in education, where they’ll create this little custom GPT or a gem to do this thing, you know, whatever, gone. So all of that work you’ve done on shadow AI without doing it as part of the enterprise is all going to be just vanish. And that, and it’s kind of, I mean, the part of the reason I’m interested is because
Neil Watkins
Mhm.
Kieron White
KnowledgeFlow as an AI harness, which is kind of, I think, what you would define it as, although, you know, the language in this space changes all the time, doesn’t it? Or at least, at least different people have different views of what the things are called. But as the knowledge flow is the harness, that owns the data, owns the prompts, owns the audit trail and the security, and it doesn’t matter what it’s plugged into. So if ChatGPT was to say, we’re not doing these models anymore and Microsoft pulled all their stuff,
Neil Watkins
Yeah.
Yeah.
Kieron White
Well, we just plug it into a different model. So it’s the kind of importance of enterprise grade AI tools and not just saying, yeah, good luck everybody and have fun with your own little tools and devices. So yeah, that’s.
Neil Watkins
Yeah.
Yeah.
Was it switched off quickly then? Is it is it just was it a sudden chop?
Kieron White
I don’t, I’m not sure of the timing of it. I think that is being planned in because it was, I think I read the first thing was they could not go, no new ones will be able to be created from now. So I think they’re going to probably run them out over whatever rather than just end it overnight. But just that whole, it’s kind of in a wider sense fits into the kind of control and sovereignty question of
Neil Watkins
Yeah.
Okay.
Yeah.
Kieron White
who’s actually in charge of this stuff. And if they just decide to end it and you’re relying on it, a bit of teaching GPTs and gems is hardly, you know, going to end your, this is not going to damage a pupil’s future or anything probably. But it’s, yeah, it’s just wildly inconvenient and no say. The users just have to live at it. So there you go, watch out as the watch out for everybody and make sure you’re doing things sensibly.
Neil Watkins
Ben.
Yeah.
Absolutely.
Kieron White
With knowledge flow, do it with knowledge flow, in fact, that’s the answer to all of all of these problems.
Neil Watkins
There you go. There’s your plug. Have you got a have you got a feature of the week?
Kieron White
I do have a feature of the week, which is, play your jingle in your head, the EHCP agent. So our latest creation, and we’ve been building it with colleges, which is how we like to do things in our sectors. So we either build stuff with housing, social landlords,
for housing or with colleges. So we’ll get their input and they tell us about what’s important and they give us documents which are the, you know, feed the data and stuff and then we build the tools in knowledge flow and the EHCP thing. So one of the colleges we’re building it with, they this year had 1000 EHCP consultations to respond to.
Neil Watkins
Wow.
Kieron White
So that’s what happens at the beginning of term for a college, and presumably must be the same in a school, is the local authority will write to you and say, here is a learner, here is their EHCP, and we would like you to take them, and we think you can discuss. And they have, I think it’s two weeks.
Neil Watkins
Mm.
Kieron White
page documents and not a quick cover sheet. Then think about what provision you can offer and what you can’t. Then work out whether there’s any cost implications. So if someone needs a, you know, a full-time assistant with them all day, then it’s an extra whatever that is per year. And then write all that back and reply to the
to the local authority with a yes or a no. And so of this particular college, they have to do 1000, they take about 200 on average, it sort of works out. So for those ones, obviously, then they’ve got other steps, creating learner success plans, which are the kind of adapted plan for these particular learners that you share with the teachers.
Neil Watkins
Mhm.
Kieron White
and then an annual review and a load of other stuff. So we’ve built a tool that will securely, it’s all completely anonymized stuff. So we’ve got our pseudo anonymization running on it. It is a stepped runner, so it’s very accurate. It takes every part of the EHCP and does a load of stuff with it.
takes about 3 minutes to run, but then you get your answer. Yes, we can take this one and it’s going to cost 23,000 pounds or whatever, or no. And
Neil Watkins
How long would it take? How long would it take to take one of just a human?
Kieron White
I mean, it’s hours, hours, very human. You’ve got to read it and think and look at it and cross-check to do other documents to see what’s possible. And there’s the law, we’ve got the law built in on this, the Children and Families Act has a section on the HCPs and how you have to deal with that. So it’s really impressive. And that, I mean, it’s
Neil Watkins
Okay.
Kieron White
We’ve had huge interest from colleges when they’ve heard about it and when Heather has been showing it off, they’ve been going, that’s amazing, when can we get it? So hopefully that’s going to be a real big time saver and as always, time is never the thing I’m not interested in. It’s the can it help make better decisions,
Neil Watkins
Yeah.
Yeah.
Kieron White
get the right support to the students that need it, create better learner success plans or individual learner plans. They’re sometimes called, create better ones that are going to really help rather than be this bog standard because frankly, if I’ve got 1000 or even 200 learner plans, that’s huge. So
Neil Watkins
Yeah.
Yeah.
It is, yeah. The reason I asked is because I told you I’d been dibbling in housing a lot this week and come back to something that we’ve talked about before, which is, you know, it’s not the prompts, it’s not the number of tokens you’re burning, it’s actual quality. Actually, I’ll take a non-housing story first. So
Kieron White
Yes.
Okay.
Yeah.
Neil Watkins
We had a customer this week who really interesting. They’ve been a customer for a couple of years. The person doing the bid writing wants to continue with the bid writer, although they haven’t seen the new one yet. And
someone in the organisation is blocking it because they haven’t seen it and don’t know about it or whatever. And the CEO said, why haven’t we just done it? Because we’ve won business that we wouldn’t have won otherwise without this tool. So just get on and do it. So really,
Kieron White
Yeah, yeah.
Neil Watkins
Really interesting. It’s like, because they only had a small number of prompts, because we look at the usage stats every month. And we always have a debate about, it’s not about the number, it’s about the quality. But there’s a classic example of where someone’s looked at the usage stats and gone, oh, it’s pointless not using it very much. But actually, it’s winning that organisation.
Kieron White
Mm.
Neil Watkins
contracts that they wouldn’t otherwise otherwise get. And it’s absolutely, absolutely nuts. But and similar thing in housing, really, I’ve been digging under some of the housing stats and actually looking for the impact stories. And I put one out on LinkedIn today, I’m going to put out a whole load more. We can do this every week because we
Kieron White
Yeah, exactly. It’s no brainer, isn’t it?
Right.
Neil Watkins
We’ve got so many of them, but it’s kind of, we just need to make more noise about this, save this person this much time or effort, but actually it gave a better response. It got this repair done quicker, it got this repair done better. Or, you know, that’s the real impact is not is how it helps somebody
Kieron White
Yes.
Yeah.
Neil Watkins
under pressure, not make a mistake that would have been costly or difficult to unpack. Actually, how do we get those stories out into the world better? And I’m really, I’m really keen to do that. So I was really, I’ve enjoyed my week dibbling in the housing, in the housing information.
Kieron White
Yeah.
Yeah, quite right. It’s a good sector put to be in, really interesting.
Neil Watkins
It is a good sector and actually linked to that. Ice. Oh, go on then.
Kieron White
I’ve got a point now I wanted to chuck in on your, since you’re talking about tokens and monitoring prompts, is it occurred to me that counting prompts as a kind of indicator of AI success is pretty, it was like counting keystrokes to measure someone’s productivity. I mean, it’s just like,
Neil Watkins
Oh yeah.
Kieron White
Yeah, I mean, it really is. It’s like the same thing. Oh, yeah, brilliant. Yeah, they’re typing all day. They must be really, really good. Well, maybe, maybe which keys were they pressing?
Neil Watkins
Yeah.
Do you remember? Yeah. Correct. It’s like you hear those stories about people who are at home working and they’ve done a little like a bit like you’ve done on your mouse where the button and it just presses every every 10 minutes to prove that something’s happening so their remote boss thinks they’re actually working.
Kieron White
Yes.
Yeah, yeah, yeah.
Away from keyboard.
Neil Watkins
Or the olden days, this, in the olden days, Kieron, you won’t remember this because you’re too young. But you used to get this thing where you’d get cheap life insurance if you joined a gym and you had, or you had to show however many steps on your phone and you’d see people just moving their phone up into the house, inappropriate.
Kieron White
Ha ha.
Neil Watkins
shaking their phone in order to make it look like it gets the step counts up. Absolutely useless, absolutely useless.
Kieron White
Get step cancer, brilliant.
The fittest, the fittest bloke in the in the working man’s club.
Neil Watkins
Right, yeah, that’s right in the pork pie factory.
Kieron White
Yeah.
Neil Watkins
I was going to also mention the regulator of social housing, you probably know, Kate Dodsworth. Did you see that she’s put some stuff out about AI usage in housing this week?
Kieron White
Brilliant.
Ohh, yes.
I have not seen that, no, but I saw them speaking.
Neil Watkins
Oh, crikey. Do you see this be the first time ever I’m ahead of you on the housing front?
Kieron White
I think you are, you are, yeah. I was at, when I spoke about at the conference at the Housing Community Summit last month, they had the regulator there speaking, which is really interesting. Yeah, this chap about, and he wandered into AI a little bit and he said they are
Neil Watkins
Nice.
Yeah.
Kieron White
thinking hard about it now and the data and stuff. What did they say though? Tell me what.
Neil Watkins
Well, they published some stuff this week and this lady called Kate Dodsworth, who I understand is the regulator of social housing, put out that she wants to enable secure AI innovation rather than obstruct it, but is prepared to intervene where AI contravenes or undermines the outcomes that they expect from and the standards they expect from the sector, which is all
all kind of good stuff. And then she went on to highlight some key risks, which I’ll just read out for you, because none of this will come as a surprise. Obviously, data quality, data protection, cybersecurity, bias and unfair outcomes, digital exclusion and accessibility, linking back to your complaints stuff.
Kieron White
Yeah.
Yeah, interesting.
Yeah.
Neil Watkins
and then service quality and all things that we’ve kind of touched on in these conversations and others. So yeah, really good to see that all of that is sensible, what we’re doing already and potentially what does fit with what’s going on. But
There was also a survey of 66 landlords by a website called Housing Digital. I’m not sure if you’re familiar with Housing Digital. Well, out of the 66 landlords that responded to their survey, only 11% have a formal AI strategy, which I thought was
Kieron White
Okay, yeah, yeah, I’ve seen seen them, yes.
Yeah, yeah.
Neil Watkins
really interesting. And only 14% of staff in those organisations use AI regularly, which is really interesting for some of our customers, given the kind of usage stats that we have seen go up, which is really interesting. But one of the commentators said, access does not automatically lead to adoption. So we’re back into the whole kind of
Kieron White
Bye.
Neil Watkins
Yeah, just because you’ve got co-pilot. Yeah, it doesn’t, it doesn’t make. And we’re back to the old whole kind of Disneyfication of how do we get people to actually want to use it rather than just that, here’s another tool, great, but it doesn’t fit in with my workflow and my work patterns and all of that, all of that good stuff.
Kieron White
Don’t we just know I have?
Yeah.
Yeah, exactly. I don’t feel, yeah, I don’t feel like it is interesting because adoption really ought to be driven through sensible change management. But it’s rather than saying to someone, here’s co-pilot, it can do some stuff, here it is, good luck. I mean, that works for some people. Of course it does. The curious will find ways around it.
Neil Watkins
Hmm.
Kieron White
But it’s almost like a retraining of the team is what’s needed. It’s like, this is how we do it around here now, rather than rather than use this to sort of support little bits of your job, but just actually re-engineer the process. And so this is where it goes. And this is then you do this bit. I think that’s what’s probably needed in a whole bunch of areas. But interesting on the usage, that’s quite low. I mean, I did.
Neil Watkins
Yeah.
Yeah.
Kieron White
The latest, I updated my research for the Housing Community Summit and that was 76% of housing professionals use AI. Wasn’t, there was nothing more on how much, so yours at least I did use regularly. It might be that they…
Neil Watkins
Yeah, it said something like, I think it said something like 60 odd percent had access or something like 74% had access to the tools, but only 11% were regular users.
Okay.
Wow.
Kieron White
got Claude Enterprise or something running inside the organisation, but now they’re going to limit everybody to $29 a month, which is apparently very disastrous for some people who spend $1000 a month currently. So that’s quite a big step down. But what they’re saying is, and I think this is the kind of interesting angle, I mean, firstly interesting that now the
Neil Watkins
Yeah.
Yeah.
Kieron White
throttling is coming in. It’s no longer token maxing, it’s token minimising. But I think the implication of that is what they’re saying is they’re going to, you can have more, but you have to appoint it to a cost centre. So the way management consultancies work, as you and I know very well, is that you will appoint the time to different client accounts, depending on what you’re doing. And the idea is you incentivize your staff to be
utilised as much as possible across whichever accounts that you’ve got. So time gets allocated to those cost centres. What’s interesting is that now you’re putting AI costs to them. And that’s kind of, you know, it’s a signal. It’s more than simply like, put some of your Microsoft budget against each of your clients. No client would accept that. It’s like, well, I have to have Word, so you’ve got to pay a few quid for my Word licence or whatever.
Neil Watkins
Yeah, yeah.
Kieron White
over my office licence. But what you are acknowledging is that this thing is a kind of almost another resource that we’re bringing to your client account, which is kind of interesting rather than it being in the background. So I don’t, yeah, it’s interesting to see where that goes really.
Neil Watkins
Yeah.
Yeah.
I can imagine it will be interesting to see where it goes, because let me tell you a little Accenture story. I mean, it’s over 25 years old now, but I was out on the town with two Accenture consultants. And yeah, well, and they said, right, where shall we go for dinner? Oh, let’s go there.
Kieron White
Yeah.
You too, poor thing.
Yeah.
Neil Watkins
because it’s very nice. And one turned to the other and said, which client account should we put this one to? And I think you’re going to get exactly the same thing with the customer going, hang on a jolly minute, you spent how much? And can you prove that those tokens were burned against my project?
Kieron White
Heather.
Yeah.
Yeah, exactly, yeah, yeah, exactly.
Neil Watkins
you know, where’s the evidence pack here? And if only they had an evidence pack, because actually the evidence pack would pretty clearly tell you when people were taking the Mickey. So if you’re being throttled to 29 pounds, $29 worth of tokens each month, you’ll just go, oh yeah, I can pretend that’s linked to this thing over here, just assign it. So I would
Kieron White
More evidence, the audit chance, yeah, ohh, yes.
Yeah.
Yeah, exactly. The age-old consulting thing, when it gets to time sheeting at the end of the month, complete your time sheets at the end of the month, you have to dream back to what you did.
Neil Watkins
I would be cross, if that’s right, yeah, yeah, yeah, yeah.
Yeah, yeah, putting a jacket on the back of the chair and then pretending you’re off doing a meeting somewhere and you’re actually doing some work for a different plant somewhere. It’s like…
Kieron White
That is, that is brave, I think, taking that approach, isn’t it? But yeah, there was a story of that, wasn’t there? Definitely happened. Somebody, a jacket, a jacket in one client’s office, come in, put a jacket on the chair, fire up the PC, off you go, across the road.
Neil Watkins
It was, yeah, yeah.
Yeah.
Yeah, yeah, to pick up, pick up your book, pick up your notebook and pretend you’re going to a meeting. Much more difficult these days, of course. But the token thing is the same, is a very, very similar problem. And I think, I think if I was an accent client, I’d be going, hang on a jolly minute. What exactly are your, you know, what’s the value adds of your guys then? Because this is back to the,
Kieron White
Yeah.
Yeah.
Or can, yeah, exactly. Can I have less of the people and more of the agents on mine, please?
Neil Watkins
Yeah, $29 for a million output tokens. I’ll take the $29 rather than your $2,000 a day for your individual. Thanks very much. So yeah, I think, I mean, the whole consulting model has been under some stress since the arrival of AI, hasn’t it? So
Kieron White
Ha ha.
Yeah.
Hasn’t it?
Neil Watkins
Similar with accounting and the whole kind of, you know, if you’re going to use accounting to, so if you’re going to use AI to do the auditing, then you need less people. So my bill should go down. You’re going to get the same with consulting, I’m absolutely sure.
Kieron White
Yeah.
Yeah, but it is already happening, isn’t it? So yeah, I think that’s going to be very interesting to see how all this plays out. And as I think I touched on before when we talked about lawyers, because they are totally rife for disrupting with AI because it’s all reading words on pages and arguing over what that word meant and whether or not it’s reasonable and what does reasonable mean.
and AI can do that in spades. But the interesting question, I think, for lawyers is because they’re regulated mainly, they’ve got a place for quite a long time, because you can’t buy a house in the UK without a lawyer approving it. And no amount of AI is going to persuade somebody for now at least that
Neil Watkins
Mm.
Kieron White
that you didn’t need the lawyer. And no lawyer would have signed something that an AI has done the work for until they’ve checked and done the work themselves, probably, or at least most of it. So I think they are quite protected for a while. Legal industry. OK, so maybe that’s what I should tell my kids to look at when they finish uni. And yeah, maybe that’s the way to head.
Neil Watkins
Yeah.
Yeah.
Yeah.
Yeah, yeah.
What, being lawyers?
Kieron White
Ha ha.
Neil Watkins
Is that ’cause is that ’cause they know how to bend the rules, Kieron, and they know how to skirt the edge of?
Kieron White
All right. My children are squeaky clean and lovely and wonderful human beings. I love you know, for purpose, for the purpose of this public broadcast.
Neil Watkins
An angelic, I know I…
I know them. I am fortunate enough to know them very well and I am glad to have helped at least one of them with his driving lessons. So I won’t hear a bad word said against your kids, Kieron.
Kieron White
Indeed.
Yes, you have a date.
His friend was listening to our podcast and gave us some feedback a while ago. But yeah, so they do, it’s quite weird how I occasionally will get a message back from Louis saying, oh, Galvin says your podcast is like this and boring, Paul. But yeah, so it’s quite weird that they like that they listen in, but maybe hello, Louis, if you’re listening.
Neil Watkins
Ohh, dear.
All right.
That is rubbish.
Kieron White
Nice to good to have you with us today.
Neil Watkins
Yeah, you’ve boosted our listening numbers by 100%, so thanks very much.
Kieron White
That’s right, you have, yeah, if you get two of you listening and it’s best. I am.
Neil Watkins
Brilliant. You said you weren’t going to do public maths. I see the way you avoided saying what the percentage was.
Kieron White
I did, I did keep clear of all that. I want to share a philosophical perspective with you as we, I guess, head towards the end of this podcast. I mean to pontificate as our audience is drifting off to sleep. So I was reading it, I was reading an article and I can’t remember the kind of full context of it.
Neil Watkins
The.
Oh, okay.
Okay.
He’s asleep already.
Kieron White
but it was talking about the boundaries between disciplines, so between law and accounting and psychology and geology and whatever, are totally made-up by us. They don’t really exist in reality and for an AI, for a large language model,
They don’t exist at all.
Neil Watkins
Mm.
Kieron White
because the concepts are the concepts behind the, you know, the same concepts playing out in this thing or over there or whatever. And the fact that it happens to be called law over here doesn’t really make a difference to the AI view of the world. And the kind of point was like, really, maybe our small brains just can’t get our head around.
all of that. And therefore, you know, we’re trying to corral AI to be like, we’re going to have this model that’s really focused on this area. And you know, actually, that’s because of our very poor way of having to split disciplines because we can’t keep it all in our heads otherwise, whereas AI can. Anyway, it’s just a philosophical reflection that made me
Neil Watkins
Mm.
Kieron White
think again, yet again about what the future world might look like in a super intelligence, as I believe you’ve got to call it in the US now.
Neil Watkins
Yes, right.
Do you know what your philosophical question makes pop up in my mind?
I need a beer so much.
Kieron White
I was going to say it makes you want to go to the pub. I know it. Good.
Neil Watkins
It does, yeah, yeah. I’ve got enough blathering on with you, fella. Right, I’m going to go to the pub. You should do the same. Get rid of that hangover with a little hair of the dog. This isn’t setting a good, this isn’t setting a good example for your kids and their friends’ other. So yeah, have a lemonade or something.
Kieron White
Yeah.
Ohh.
I’m going to…
Well, I’m going to set a good example because I can nip up and go and collect Fin from school because we’ve finished a few minutes early, so that’s perfect.
Neil Watkins
Excellent. In that case, go and say hi to Fin for me and I will catch you next week.
Kieron White
Alright, nice. Have a great weekend. See ya. Bye.
Neil Watkins
Take care, bye.
Neil Watkins stopped transcription
