[00:01] SPEAKER_02:
Welcome to Principal Center Radio, bringing you the best in professional practice.
[00:06] Announcer:
Here's your host, Director of the Principal Center, and Champion of High-Performance Instructional Leadership, Justin Bader. Welcome, everyone, to Principal Center Radio.
[00:15] SPEAKER_01:
I'm your host, Justin Bader, and I'm honored to be joined today by Dr. Gary Jones. Gary is a post-secondary educational leader and author. He is based in the Channel Islands, on the Isle of Jones. Thank you, looking forward to it.
[00:46] Announcer:
And now, our feature presentation.
[00:48] SPEAKER_01:
So we're here today to talk about your book, Evidence-Based School Leadership and Management. What is evidence-based school leadership, and why is it so important for our work as educational leaders?
[01:01] SPEAKER_00:
Two questions there. In terms of what is evidence-based school leadership, it's quite simply about school leaders using a diverse range of evidence, be it research evidence, their own practitioner expertise... school data, district data, uh, and the views of stakeholders, to take all that evidence together and aggregate it, and then hopefully make a decision that leads to better outcomes for their pupils, their students, their community, and the like. That's fundamentally what it's about, using multiple sources of evidence
[01:35] SPEAKER_01:
I love that phrase because it seems to me that we tend to prefer, at least in the United States, the term research-based, and we almost use that as a conversation stopper. Like, why do you want to do that? Oh, it's research-based. But you're saying multiple forms of evidence, including research, but not limited to research.
[01:52] SPEAKER_00:
Yeah, absolutely. And in a sense, that's one of the things that I really bring home in the book, which is it's about making decisions informed by a range of evidence. Research doesn't tell you what to do. It gives you suggestions about where you might want to look to help you find solutions to problems that you're facing in the context of your school or community or district. And you've then got to work out whether that research or that mechanism that you've identified might work in your school or not. And it may well be that the conditions which have actually facilitated the original research to work don't exist in your place.
[02:31]
And therefore, you're going to find something else. So it's about helping you make more informed and hopefully better professional judgments. There's a really interesting book, but challenging book, by a Norwegian professor called Tone Kavernbeck about evidence-based practice, which really outlines the role of evidence in developing an argument. And basically, it isn't the only part of an argument. It provides backing to what is known as the warrant. An argument.
[03:02]
But that's what it is. It's the backing to something. It doesn't tell you what to do.
[03:06] SPEAKER_01:
I appreciate that centering of professional judgment there, because I think often we use research as a substitute for professional judgment, or we use research as an explanation for why we don't really need professional judgment. We already know what to do, because the research tells us. And that's really not the case, right? It's a piece that informs our professional judgment, but can't take the place of it.
[03:27] SPEAKER_00:
No, because basically what you've got to try and do is understand... What, uh, and I think there's some interesting work on, there's one called the, sort of, causal cake, uh, sort of, analogy, which is, you've got the mechanism that's working, that's leading, if you make this change, this happens. But there's a whole range of support factors that need to be put in place for that mechanism to work, be it the skills and capacity of your teachers, be it time, be it buildings, be it space, be it culture. And sometimes those things just aren't there.
[03:59]
So you've got to take all these things into account when trying to work out what's the most appropriate way of trying to address I wanted to bring up the role of how topics for research are suggested, because often when educators are making professional development decisions or program decisions, you know, what to do with their students, they're asking, you know, is this research based?
[04:32] SPEAKER_01:
But it's not as if researchers just spread out over the country and study everything everyone is doing and, you know, identify the best practices and disprove all the bad practices. You know, how do researchers in the first place figure out what to study so that there is research evidence on certain practices?
[04:51] SPEAKER_00:
I think it's important to remember that researchers have a different audience than teachers, and teachers aren't their only audience. So, therefore, quite often what researchers research is what's fundable and what's interesting to them and what is most useful for them in the context of their career. That isn't necessarily the same as things which are useful for teachers. That's not a bad thing. I'm not being critical on that at the same time, but it's just they've got a different job to do. Hopefully, part of that job They'll produce stuff which is useful to teachers, which they can then think about, use, and adapt.
[05:24]
And it may well be that sometimes that's actually what the researchers are setting out to do. It's just important to recognize that it isn't just a research into education. It just isn't about research into teaching. education is much broader than that.
[05:39] SPEAKER_01:
Right, and as practitioners, particularly at the K-12 level, we're interested in, you know, the insights for practitioners. You know, often funders are looking at things like program evaluation, you know, if there's a large national program, The provider of that program wants to have research about its effectiveness and can fund research, or an independent organization might fund research. What are some of the questions that we should be asking ourselves? And how do we formulate good questions when we go out looking for evidence? You know, like, let's say a school is wanting to adopt a particular approach to solving a particular problem. What are some questions that we need to keep in mind, rather than just saying, is this research-based?
[06:20]
I feel like that's the question that we usually start with. Is this research-based? We'll do it if it is, and we won't do it if it's not. Help orient us around that question more usefully.
[06:27] SPEAKER_00:
In England, there's been a lot of emphasis on helping teachers access research. For me, that's very useful, but actually, before you actually give teachers access to research, it's helpful to work with teachers to help them develop their skills at asking good questions. Because if you can get the question, not right, but in the right ballpark, for want of a better phrase, then... that directs you, that helps you find out the best evidence.
[06:54]
If your question's wrong or not as precise as it could well be, then there's a good chance that the research that you'll look at won't be appropriate. So I think, for me, the starting point in terms of helping teachers use research is to think in terms of developing good questions. And there's all sorts of useful rubrics around to help teachers go about that particular process. So, there's something called the PICO format, which I'm not sure you may have come across, which basically says, look, in terms of a question, what is the group of pupils or problem that you're looking to deal with? Okay. It then says, okay, what things, what intervention could be put in place to address those needs?
[07:40]
Whatever intervention could be put in place to address those needs and then finding what outcomes do you want to achieve as a result of addressing that particular problem. Now, sometimes it may well be you know the problem, you know the outcomes, but you don't know what interventions or comparatives that you can use. Well, that's part of the process of going out and finding the research and evidence. But if you haven't got a sound understanding of your problem,
[08:15] SPEAKER_01:
Well, and let's talk about that definition of a problem of practice. You talk in chapter three of the book about a problem of practice. And sometimes, honestly, I think in our profession, we don't start with a problem of practice. We start with a vendor and a solution and try to figure out, would this be good for us? And we maybe skip that step of clarifying our problem of practice. So what is that and how do we clarify that?
[08:38] SPEAKER_00:
Well, I think in terms of identifying a problem of practice, what I think it's useful to do is evidence-based practice should be seen in the context of what a school is trying to achieve and its priorities and the problems that it's trying to address. And I think when I talk about a problem of practice, I emphasise the idea that it needs to be something which is important, you know, to the organisation, to pupils and to the staff. But also, in this particular context, I think it's one of those things which actually is non-urgent. And the reason why I say it's a problem which is important but non-urgent is that evidence-based practice can be a time-consuming activity. And if it's an important and urgent problem, you are probably going to have to rely upon what you already know and your instinct and your experience and the like.
[09:31]
You may need to check it out a little bit, but you won't have the opportunity all the time to go and check out the research and stuff of that nature. So, for me, a problem of practice is something which is important to the organization that needs to be addressed But it's non-urgent, but adds to the capacity and capability of the organization.
[09:49] SPEAKER_01:
To go back to the PICO acronym that you mentioned before, just reading a little bit from the book. PICO, so Problem, Intervention, Comparison, Outcome.
[09:57] SPEAKER_00:
Yeah, so basically, there's all sorts of other versions. There's something called SPICE, there's something called CIMO. And I think the thing is, I use PICO because it was the first one I came across, and it's pretty well known in the context of medicine. But there's other ways of doing it. And what matters is that you find a way to help you structure your question in a way that then helps you go about finding the right evidence and then helps you then interpret that evidence in the context of that question. Otherwise, you can spend a lot of time finding, you can find a lot of research evidence, but it doesn't help you.
[10:36]
You just get, you just get drowned in it. You need to have something that allows you to actually, yeah, this is relevant to me because it addresses that particular problem.
[10:45] SPEAKER_01:
And that role, again, of professional judgment needing to remain centered. So we're starting with what we know, we're starting with a problem that we need to solve. And we talked earlier about the idea of multiple kinds of evidence, not just, you know, is there a study on this? How do we kind of bring those different perspectives together? Because obviously, you know, we might have an instinct about something, we might have heard...
[11:08]
what someone else did in a similar situation, and then we might also have research in front of us. What can that look like when schools are really trying to be evidence-based, but with an awareness that there are multiple kinds of evidence that aren't really apples to apples? You know, it's not that we just have study A and study B. We have lots of different things.
[11:25] SPEAKER_00:
I think what you can do is actually just write it down. I mean, this may sound really bizarre, but just write it down on a table. Just saying, okay, what does my research evidence tell me? Can you write down what the research evidence is saying? If we do this, then this will happen. And what is the research to back it up and how robust it is?
[11:44]
You could then say, okay, what do we as practitioners, what are our theories of action? So, you know, what do we, what are our values and beliefs? If we do this, then we think this will happen. Write it down. What are the experiences and views of others? And then come up with, and then, and what does your organisational data tell you about whether the problem that you're trying to solve is actually worth solving?
[12:09]
Or is solvable? Or, actually, there's other things you should be solving. Write it down. And in that process of writing it down, then, one goes as far as saying it's magical. That's one way to describe it. But at least it allows you to get an overall picture And you can start seeing, yeah, that looks strong, that looks weak, you know, we're missing something here, we're missing something there.
[12:31]
So, it's actually about, I use a phrase, doing simple things well. Just see if you can capture, summarise the position on one side of A4, under four headings.
[12:43] SPEAKER_01:
A4 being a sheet of paper, right?
[12:44] SPEAKER_00:
Yeah, A4 being a sheet of paper. You know, you can do a table, write it down, a few, four or five bullet points per heading. You know, what do you know, or what do you think you know? And from that, hopefully, you'll get a sense of the direction that you need to travel in.
[13:02] SPEAKER_01:
Doing simple things well. I think that's such an antidote to the complexity that we often land ourselves in, because, again, back to the research, research is often done on very complex things, right? You know, a complex intervention, you know, for reading, a complex system for school culture or behavior. And we have all these complex things, but often when we have two or three or four or ten complex things in place at once, and maybe they were all adopted at different times, often we lose that clarity about the simple things, and we lose that ability to do those simple things well, and, you know, not at cross-purposes with one another. So I love that, of just getting a sheet of paper and saying, what is our problem? What is our, you know, what do we think will work?
[13:47]
What are our sources of evidence? Uh, you know, using some of those tools that you talk about, like Pico, to kind of articulate that, and not getting wrapped up so much in the complexity, but really in trying to explain it to ourselves as simply as possible.
[14:01] SPEAKER_00:
I mean, I came across some interesting work done recently on theory of change. I think it's called the PIP-B model of developing a theory of change, which basically, what's your problem? If we do this, then this will happen because... Can you write that down?
[14:17]
If you can, then you've got a chance. And then having written your theory of change down, can you then write out, what about phrasing a simple logic model, says, let's put these resources in place... If we put these approaches in place, we'll get these outputs, which hopefully will lead to these outcomes. If you can't work out by putting the inputs in place will lead to your outcomes, you've got a problem.
[14:41]
Because what you're trying to do, you're just hoping that you throw some resources at something, Then you're going to get these outcomes. You've actually got to try and understand what's the mechanism that gets you from A to C. And if we can't even articulate it for ourselves, how can we expect it to actually work? Yeah, and again, these are simple, and these are relatively simple things. Well, we phrase that. I think there's a phrase from Mark Twain, which is, if you want me to write 5,000 words on a topic, you can have it tomorrow.
[15:09]
If you want 500 words on a topic, you can have it in a fortnight, two weeks' time. And I say these things are simple. Actually, they take a lot of thinking. But the product isn't big. It's not about writing master's dissertations and the like. It's actually about trying to synthesize these things down to the basics.
[15:30] SPEAKER_01:
Well, let's give people an example, uh, if we could. I think one that comes to mind for me is the approach called PBIS, Positive Behavior Interventions and Supports. And I think there's probably been a ton of research on it. Honestly, I haven't read any specific research on PBIS, but I know in our world, it's one of the, you know, school climate, school culture, behavioral kind of, uh, approaches that's fairly well researched. But as a basic logic model, You know, it comes down to some pretty simple things. Like, if we know how we want students to conduct themselves, if we explicitly teach those expectations to students, if we are consistent as adults in terms of upholding those expectations and treating them as, uh, you know, skills that we need to teach, like any other content, then we will...
[16:15]
Yeah.
[16:33]
You know, upon a little bit of scrutiny. So what can that synthesis process of becoming a little bit more evidence-based in our decision-making look like? Or how would you apply some of your insights about evidence-based leadership to a school situation where leaders are striving to improve climate, striving to improve behavior? They're thinking, hey, maybe we need to do PBIS. What would some good questions be or some good ways of thinking about that problem with the evidence-based approach?
[17:00] SPEAKER_00:
One way of thinking about it is, again, to do a quite simple table, which I think I referenced in the book, which is, if you take PIBS, okay, how strong is the theory or the research on it? Okay, how strong... You need to look at the register and say, okay, the research is fairly conclusive, there's a fairly consistent view across the evidence, across the research, research findings of what about phrase confirming each other, da-da-da, That would suggest that we've got someone which is robust. Okay, so let's assume that we've got someone which is robust.
[17:31]
You then may ask yourself, what resources have I got in this school to support whatever we try and do with that intervention? In terms of time, money, spaces, the school year and how that plans out. Okay, what resources have we got? Then the question then becomes, what's the commitment have I got from my colleagues? Do my colleagues get it, understand it, and are supportive of it, or are they mainly reluctant, or they need to be won over? And also, you then may say to yourself, okay, what expertise do we have inside the organisation, inside the school, about this particular area?
[18:14]
And then, in effect, just map it where you are. So, what you may find is, actually, you've got some fairly robust research, but your resources available to support it are pretty low. There doesn't... There's some commitment on behalf of your staff to go along with it, and you've got some expertise somewhere in your organisation, or people who experience this programme elsewhere.
[18:36]
On that basis, if you introduce it into your school, you're going to probably do a very small pilot. And I think, or maybe a small pilot, with maybe three or four staff trying it out. I think there's a lot to be said for looking at the work on improvement science and BRAIC, and around lots of small... steps where you do things, you plan, you do, you study, you act, you do these cycles, and you're continually learning.
[19:03]
But you start small and gradually go big. Probably, for many people, they think the research is strong, but they forget the evidence behind them about their schools and context. And they try, and I did this in my own world, you go too big at the beginning, and the research isn't strong enough to support it, and your infrastructure's not strong enough to support it, and it falls apart. So...
[19:26]
In terms of being evidence-based, I would say it's about identifying your problem, finding the best way of tackling it, and then trying some small-scale PDSA cycles in your school, which you then use to generate your learning about how this thing will work in your context, and then how you can
[19:50] SPEAKER_01:
Very well said. I'm a big fan of Anthony Brick's work on improvement science and the Plan, Do, Study, Act cycle. And it's interesting to see how that has developed as an organizational learning approach in organizations that are not in the business of learning. You know, they're manufacturers, they're hospitals, you know, and, like, we should be the best at this, right? We're in the business of learning.
[20:12] SPEAKER_00:
Yeah, but what's interesting about... What's interesting... On my blog...
[20:15]
There's a post, which I did recently, about plan, do, study, act cycles, and teachers find them quite difficult. There's some research done on three schools, and where they try to introduce something using PDSA cycles, and these PDSA cycles were new to the school, plus the innovation was new to the school. So, effectively, you had two innovations going on at the same time, and Only in one of the three schools were they able to, in inverted commas, make it work. And a lot of it was down to the capacity and capability of the staff. If you try and put these cycles in and you impose them onto people, then they say, this is the next bit of work. Why am I doing this?
[20:57]
They don't see the value of it. Where you do it and you say, you know what? We're going to use this PDSA cycle. But in order to give you the time and space, we're going to stop doing these two other things. And so, therefore, they've then got the space to do it. And we're going to...
[21:11]
There's a phrase which a colleague in England has used. He calls it, you bake things in rather than bolting things on. You bake them in, you don't bolt them on. In other words, you make it part of the day-to-day work. Over time, it should become a fundamental part of your practice. It's a fundamental part of what you're doing.
[21:31]
It's not something you're adding on because you want to show that you're a research-orientated school.
[21:37] SPEAKER_01:
Right. Because we can have... We can have lots of research-based programs that simply don't fit together, that we aren't implementing well. And, you know, it's that challenge you describe on your blog of making what worked there work here.
[21:51]
And if we bolt it on, if we bolt on all these different things and don't really have a coherent sense of how they're baked in, you know, how they're baked together, it's not going to work.
[22:00] SPEAKER_00:
So therefore, as I say, the key thing about, and going back to leaders, if they want to create conditions for evidence-based leadership, I would argue the first thing that they would need to do is be humble, okay? And to acknowledge that they don't have all the answers and that actually whatever they come up with could be wrong. And start with that basis. Other people probably know better than you. Or if they don't know better than you, can certainly challenge what you think you know, so that actually your knowledge becomes more robust. So I think that's a critical, critical thing to do.
[22:35]
And in a sense, what leaders need to do, they need to model this stuff. So they need to show that they've got a genuine interest in research to challenge their practice, rather than confirm their existing biases. That's a really useful starting place. The starting place for me is, it's about challenging what you know, not confirming what you know, or confirming existing prejudices. And if you do that, and you've got that commitment to challenging yourself, and you demonstrate that to others, then hopefully, staff will begin to see that actually it's acceptable to challenge their own practice.
[23:13] SPEAKER_01:
Wow, so much we could go into in more detail, and I know you do in the book, but a couple of big ideas that are resurfacing for me at this point in our conversation, you know, the importance of asking the right question, of considering multiple kinds of evidence, you know, our own experience, research evidence, what other schools are doing, articulating a logic model, you know, if we do this, what do we expect to happen? You know, do we have even a coherent plan in the first place? And you talked about capacity. I think we moved kind of quickly through that, but I think that's such a critical piece to ask, you know, do we have the resources in place to actually implement this the way we want? And back to our PBIS example, I was thinking about a time when my school was in the early stages of implementing PBIS, and it was something our teachers were interested in. They said, You know, we've had, you know, this hallway behavior that we're not really happy with.
[24:02]
PBIS is working in other schools. It seems like a good solution to what we think is the problem. And looking back on it, one of the very first things we did was hire someone with expertise in PBIS. Not because we had a PBIS position to fill, but because we knew of this person and their expertise, and we thought, you know, that could be a really good direction to go, but we're not going to go very far in that direction if we don't have that expertise and some help in building that capacity, because simply knowing that PBIS is good doesn't give us what we need to make it work here. We need to bring it in, yeah.
[24:37] SPEAKER_00:
The thing is, though, that in making that appointment... Whoever does that job has then got to recognise that what worked in their previous setting may not work in this setting. And they've got to craft the solutions in a way. I think the term is, you need to show fidelity to the principles rather than fidelity to the process.
[24:56]
So, in other words, what you're trying to do, you want to make sure that what you're doing is consistent with the underlying principles of the approach. That doesn't mean that you state the process that you adopt has to be the same as it is elsewhere.
[25:09] SPEAKER_01:
Wow. Fidelity to the principles, not fidelity to the process, because the process might be different.
[25:15] SPEAKER_00:
And the resources that you've got to support the process, in all likelihood, will be different. The people that you've got will be different, be it the staff or the pupils. So, therefore, you've got to make sure, okay, we're trying to make this thing work. There are certain clues from other places about what you need to do to make it work. But, actually, to make it work here, we may need to add this. Well, I love the disciplined habits of mind and the patterns of thinking that are showing up in this conversation.
[25:55] SPEAKER_01:
And I want to encourage people to check out the book. You can find it on Amazon, Evidence-Based School Leadership and Management, A Practical Guide. And, uh, Gary, I know I found you on Twitter, at Dr. Gary Jones. And if people want to read your blog and follow that and learn more about your work, where's the best place for them to go online?
[26:14] SPEAKER_00:
If they want to go online, the best place is to go to www.garyrjones.com. And that'll take you to my website. And then each week I write a blog on some aspects of evidence-based practice. So some of the things I've mentioned here in our interview Well, Gary, thanks so much for joining me on Principal Center Radio.
[26:40] SPEAKER_02:
My pleasure.
[26:41] Announcer:
Thanks for listening to Principal Center Radio. For more great episodes, subscribe on our website at principalcenter.com slash radio.