And I’ll say a word or two about this in a moment, but I do hear a little bit of background noise, so if you wouldn’t mind putting your phones on mute during the webinar, that would be great. So, I just want to welcome everyone and thank everyone for joining us today for our fourteenth network learning webinar as part of the AHRQ Fall Prevention Program Initiative. Today, we’ll cover, you know, briefly the housekeeping information that we typically cover. I want to just spend a moment on the upcoming webinar schedule, and the large portion of our webinar today is going to be hearing from Dr. Pat Quigley, and you’re all familiar with Pat. She’s been sharing her expertise with us for close to two years that we’ve been working on this project. As you all know, she was formerly the associate director of the VISN 8 Patient Safety Center of Inquiry, which she has since gone into retirement, but she tells me that she will never give up on falls and fall prevention efforts, so we’re very happy to have her with us today. So just a brief word on housekeeping again, you know, we encourage you to contribute to the discussion when we get to that point toward the end of the webinar presentation, and you can do that by raising your hand and asking a question or using the chat panel. And as always, we ask you to keep that background noise to a minimum. We are recording this webinar and would like to post this later on the AHRQ web site to share this information with others. So please keep the background noise to a minimum. Just a brief word about our upcoming webinars. We have six more webinars after today for the remainder of this AHRQ Falls Initiative. The webinar that we have planned in October is, again we’re going to have a special guest presenter, Pat Posa who’s going to be presenting on sustainment of quality improvement initiatives which is very relevant now as all of the hospitals that are participating in this initiative are really working on sustaining their implementation efforts. The one special note that I want to say about this, this webinar is going to be conducted on Thursday at 1 p.m. eastern time, so you will be joining with the pressure ulcer prevention hospitals for this one single event in October. So I will update the calendar notice, and you know I invite you to share the invitation with others in your hospitals that may be interested in it. And then in November, we’re going to be hearing from another one of our experts, Cait Walsh who’s going to be talking about teaching critical thinking skills for falls risk assessment. And then in December, we’re going to be hearing from another special guest who’s going to be talking about Lean Six Sigma and a deep dive into reducing patient falls, an effort that they have done at their hospital in their health system. So with that, I would like to turn the control over to Pat, Dr. Pat Quigley, who’s going to be talking about measurement with us today. So Pat, I’m making you the presenter now. Oh thank you so much Michelle, Dr. Krieger, thank you for your kind introduction, all of your remarks. You know colleagues, it’s just been a pleasure to be part of this process, and implementing the AHRQ Falls Toolkit, and all of the work that you’re doing, knowing all that went into place for even developing that toolkit. So it’s just an honor and a privilege to be with you here today. And Cait will be an expert reviewer too for that, in the fall, I know has been with you all along and other people that are here on the call. So I love being able to talk with you about measurement and helping this come alive, because I love one of the quotes that’s in the AHRQ Toolkit, Chapter 5, “The basic principle of quality measurement is if you can’t measure it, you can’t improve it.” We want to help everybody to really have some precision and be able to look at the data and have it be meaningful for you. So I again, thank you, Michelle, so much for inviting me to do this talk. So these are the objectives that we have for this hour: To describe core measures of fall and injury program components for which many of them I’m sure that you’re actually collecting already. To use data to profile the population and the risk. To apply data analysis to program evaluation. Discuss elements of high reliability. And so we want to get into fidelity and reliability. This is implementation science process. I think we’ve shared a little bit of that when I’ve gotten to be on with you before. And examine a framework for dissemination for program evaluation. So we collect so much data, but when you look at the goals of data management, the users of data should be involved in collecting, analyzing and using the data. And data should be simple and understandable and presented in the same way so that it really has meaning for the staff. And it is concise and precise. And I always, when I talk about precision, and trying to help people really think about, you know, what are you doing to prevent this accidental fall, and what are you doing to prevent the anticipated physiological fall? That’s the kind of precision that I’m looking for. And then, it should be helpful in making decisions and improvement so that you can take action, and if you implement the change that you really are specific about the outcome that you’re trying to achieve, whether it’s reducing those toilet related falls or falls associated with postural hypotension, having precision. And that needs to be able to be comparable so that you do have data tables and benchmarks and targets, and used to develop evidence-based practice so that you do some projects or look at the strategy that you’re implementing and see if it is indeed evidence-based. And then it would be used from research thus being able to improve practice. So here’s some examples of Fall Program Core Outcome Measures. So these are the outcomes. So there’s multiple outcomes that you will see in the literature. Many of you are doing the fall rates, but how many of you are doing the fall rates by type of fall which is really where we want to go? And that’s where you have precision. Or how many of you are looking at repeat falls? So you could have a repeat fall rate, or you could look at the percentage of repeat fallers, and the repeat fall rates, your fall is the outcome, but the percentage of repeat fallers is at the person level. So the person is a target of interest. Or you can have the percentage of fallers. Those two with fallers, you’re looking at the person level, you’re not just looking at the outcome. So you could have one person who falls multiple times, so the percent of people that were fallers, that would be a different rate than the percent of people that are repeat fallers. So you have to look at how you actually calculate that. And the percentage of falls with injury which people aren’t doing in severity injury level. But what about fear of falling? How would you might be measuring fear of falling, and that’s at falls of efficacy, a different kind of an outcome measure. Or if you’re not tolerant on a study or project to look at falls, maybe you’re looking at improving gait and balance which is work that I’ve done in the past and one of my clinical trials as a proxy for falls, so that if you improve gait and balance, you actually are ultimately reducing a fall rate. So those are outcome measures but what are the structure measures that you’re looking at? And structures are formative measures. Outcome measures are summative measures. So the formative measures is, are you looking at staffing, are you making a change in staffing? Maybe it’s skill mix. Are you using a valid and reliable screening tool as the basis of your structure for care planning? Do you have individualized care plan so that it’s not just universal precautions? Is that part of your structure where you can actually measure the number of risk factors that a person has and whether or not they’re modified or eliminated? Your interdisciplinary team, how many of you are keeping track of your interdisciplinary team members who are coming to team meetings or participating in post fall huddles or involved in care planning. Your patient population profile, are you actually controlling for that? Are you looking at what percent of your patients in terms of structure are over the age of 85? What percent of your patients are 75 to 84? Profiling your patient population, that’s your structure. Your equipment that you have available, all of these are examples. And I know many of you can come up with other examples of structure that’s in place. And then process measures. The process measure that’s part of formative measures are really important whenever you’re evaluating a fall prevention or injury prevention program. So this is what is being done. Structure is what is in place and then what are you doing to implement a change? So this could be the timeliness of your fall risk assessment or the completeness of it. You know, not just screening, you will not see anything here for screening, but actually doing a multifactorial assessment, the completeness of it. We’re actually measuring that. Or the timeliness, how long does it take us to implement a care plan? Are we actually following that? Or the timeliness of the care plan being developed? If you’re going into using floor mats, how long does it take you to get a floor mat at the bedside for someone who needs to have it? Or if you have a patient that needs to be on a scheduling program, if we found out that there was a problem with bladder retraining that needed to be done, that we actually implemented a toileting program and we actually are measuring the timeliness of it. If it’s scheduled to be done every two hours, is it really being done? So whenever you look at a program, this is a Donabedian model, you want to look at structure, process, and outcomes. And remember as we’ve said before, you have to measure structures and processes twice as much as you do outcomes to be able to measure the effectiveness of any program. So as you’ll see on this slide, when you put it all together, to really evaluate a program and help this all come alive for effectiveness, you have to look at all the components. You have to look at program effectiveness, which usually the program elements, the team that’s in place. Then you have to look at the implementation effectiveness. How well are you implementing it in terms of reliability? Is it being implemented as it was designed to be done, which is fidelity? And then look at the outcomes and make sure they’re all aligned. When you look at this model, the plus sign means that it’s effective, but the minus sign would mean that it would be something that is ineffective. So if you have a good program, but you don’t do a good job at implementation, the outcome is going to be inconsistent, it’s not going to be sustainable, and you’re going to have poor outcomes. So, I’m sure that this October 20th session is going to be very interesting in relationship to sustainability. If you have a program that’s not well designed, if you have everything in place to implement it, then you’re going to, again, have poor outcomes. So all of it is really related when you’re looking at structure, process, and outcome. So we have to be clear about the data that we’re collecting. In the work that I’ve done, I’ve always used mixed methods, if you will, where you collect quantitative data and qualitative data. So, the quantitative data are the ones that you can actually measure. The quantitative data have attributes that are categorical measures that would be nominal data or ordinal data. Or you could have variable that are continuous measures that you would have interval and ratio level data that you would be collecting. The qualitative data in contrast is where you’re doing grounded theory research, or you’re collecting opinions or perceptions, or you’re trying to identify themes. People are coming together. Maybe you’re doing focus groups as a way of doing qualitative data. The qualitative data often can give us meaning to the things that we’re trying to evaluate or see if they’re actually going to work, or the perceptions of people that are going forward. So let’s look very briefly at these types of data. The nominal data are the ones that are actually coded and can be sorted. So, this would be ones that you assign a number to and give it to its attributes. A category that’s unordered, a category that could be exclusive or exhaustive, and nominal is the lowest of those measurements. And it’s only binomial by choices, where you actually just kind of number to something like one for yes and zero for no. That would be nominal. So, the nominal data, if you look at some examples here, like for gender here you could have gender as zero or male or a female that’s a one. You could have codes for ethnicity types, marital status. So, this is where you take an attribute and you assign it a number, so that you can run some statistics with it. That would be nominal level data, the lowest level of measurement. The next level of measurement would be ordinal data. This is data that is assigned to categorical variables and attributes that you can now start rank ordering. There are rules on how you can rank order them and their categories that are, again, exclusive and exhaustive. The quantity of the attribute can be identified but the intervals between are unequal. So this would be the example of the Morse Scale. If you look at the Morse Scale, you do have a continuous scale from zero to 125 and you’ve got the weighted variables in there. But what many people do is they reduce these categorical var- the continuous scale categorical variables by reducing the Morse Scale to one, two, or three for low, moderate, or high level of progress. So you take a continuous data and you reduce it to an ordinal level and that limits the type of statistics that you can do and the way you can use it in research. And here’s other examples of ordinal data the level of pain someone might be having, sibling order, degree of coping, satisfaction scales, acuity levels, you can see the severity of injury scales that many of us are using. It would be any type of Likert-type scale of ordinal data. So here’s another example, is a program actually meeting the personal objectives? And you could go ahead and have strongly agree to strongly disagree and each one of these would have a number that would be assigned to it. I’ve used a self-assessment tool of organizational, infrastructure, and capacity to reduce fall-related injuries. And all of those variables have an ordinal scale of being able to say from a scale from zero to three something that’s not being done at all to being fully implemented. So, that would be an example of an ordinal scale. Now, the next type of data, where you start having more- where you can start getting into some inferential statistics, the first two ones being descriptive statistics is an interval level data. And this is where you have distances in the intervals on the scale that are equal to data categories are exclusive and exhaustive. But, you have a continuum of variables and it’s continuous. But you don’t have a zero point. So in interval data, here are some examples of those maybe the number of days between preventable falls, many of us are doing that, the number of days between serious injury, we oftentimes will see these on charts that are available on unit, maybe it’s a blood pressure on admission to a nursing unit, then two hours later or four hours later, when you’re looking at postural hypotension. The last type of data, and your highest level, would be your ratio data. This is the one where you actually have a zero point on the interval scale. And it can’t be negative, but it is continuous. And this is the highest form of measurement where you can do very rigorous statistical analysis, your ANOVA, your regression analysis. Those kinds of things that can be found when you have this level of data. Much of the research that you see being done on fall or injury prevention in hospitals, we’re looking at all kinds of variables, but it’s really mostly descriptive kinds of studies with limited types of research and analysis that can be actually implemented. So here are some examples of ratio data. You have weight, and length, and volume, and blood pressure, and you see the example that was right there. But that would be the very highest level we had just mentioned. So when you look at any of the data that you’re collecting, we always want to also ensure validity and reliability. And these are characteristics that we need to have in place when we’re looking at measurement. Validity is the measure to which the extent an instrument for example holds up when you have two people implementing the instrument or you have repeat measurement of the instrument. So if you look at the Morse Scale, the Morse Scale is designed exclusively to be done for validity measuring – what it was set up to measure. The Morse Scale is designed to actually identify the likelihood of the anticipated physiological fall. That’s what really sets the Morse Scale apart. Because in terms of validity, is it measuring what it was designed to measure? So, many of the other scales that you see out there in the field were designed to measure falls. Falls as an aggregate, not a precise type of fall that you see here like the Morse Scale. And then, reliability is where I would say and mention that if you get consistency of measurement, two people measuring something the same way or two people- with someone measures something twice they get the same response. This is the internal consistency of what is actually being measured. So, the example of marking history to falls, yes, for someone who gets admitted because of a fall or had the fall within the last three months. If people are actually doing this, are they using the scale as it was designed to be done? Is interrelated reliability- you can have intra-related reliability or interrelated reliability. So, these are the characteristics of many of the instruments and the surveys and the tools that what you’re always looking for when you select an instrument to go forward and measure something is the validity and the reliability. So when you look at data and tests, there’s only certain tests that can be performed on certain types of data. And, that’s when you get into the statistics and understanding the types of data will determine the test that can be appropriate to us. And then, understanding that data type will guide decision on the strongest and most robust data that you want to be able to have included and whether it’s quality improvement or research or program evaluation, so that you can use the most sophisticated analysis and get the bang for the buck, if you will, in analyzing the data that you are collecting because it takes a lot of time to collect data. And more sophisticated analysis generally provides richer data analysis and interpretation of findings, which is why so much of the work that I have done and where I worked in my patient safety center, is we used mixed methods: quantitative and qualitative data. So in terms of evaluation programs, I think I am the only one who has ever published a prevalence study, because we were using the Morse Scale across all of our VA Medical Centers. So 99%, probably 99.9%, of our VAs were [inaudible]. My VAs in Florida and Puerto Rico, I actually conducted a prevalence study. I could profile how at-risk the in-patient population was based on the Morse Scale for the anticipated physiological fall in a given snapshot in time. Like you do prevalence studies for pressure ulcer risk. You can do prevalence studies to profile how at-risk your patient population is. I believe I published that work in around 2008, but I was actually able to show that as the patient population, as you look at the increasing age group, the level of risk on the Morse Falls Scale for the anticipated physiological fall increased. So think about it, being able to say that 80 percent of your patients are at risk for an anticipated physiological fall but maybe 20 percent of your falls are that type of fall. That’s a very powerful message. So those are the things that you can do when you start getting into evaluation methods, to look at your patient population. Formative and substantive evaluation helps you to get into the price of falls. You know, we want to be able to get past that overall aggregated fall rate and get to the types of falls, because that’s where the rubber meets the road. Are 80 percent of your falls anticipated physiological falls, or are 80 percent of your falls accidental falls? That’s a very different picture based on what type of falls are happening. And then getting to severity of injury. How are you really measuring this? What is your evaluation of injury? You just can’t measure injury at the time of rescue and think you’ve captured all the injuries that someone has had when they fell. You have to be able to look at the duration of the extent of injury of evaluation, and the extent of injury. Because oftentimes what happens with injury evaluation, is you have reported the most serious injury or severity of injury level as high, but you didn’t say it’s a hip fracture or a head injury. So people fall and hit multiple body parts and not every body part has that serious level of injury. It’s usually the highest one that gets reported. So how do you make that evaluation? And then how do you look at repeat falls? Are you looking at the repeat fall rate, the repeat number of falls. Are you at the repeat faller level, the personal level. How are you doing that? Or survival analysis, which is really moving into the number of days between those serious injuries and the number of days between preventable falls. And then I’ll give you some examples in terms of evaluation from the run charts, which are very very powerful as well. So there’s lots of ways to build your evaluation model. So let’s revisit falls as an outcome. Remember, if you’re focusing on falls, measure preventable falls. That’s really what I like to help people think about. You know, preventable falls, if you’re not getting to that, then that implies that you’re looking at all falls as if they’re the same. And we need to have more precision. You want to have your measurement, your evaluation, to have precision. So it has as much meaning as it possibly can have, so that you know the interventions that you’re implementing, are they making a difference or not. Are they being successful in their implementation process? Otherwise, you’re measuring effectiveness of interventions to — or are you also measuring effectiveness of interventions to mitigate or eliminate risk? So that was in that David Oliver article that you may remember. I’ll mention it very briefly next. But there’s two recommendations there that really help us to appreciate that we have to evaluate some of those factors and then intervene. The number and type of modifiable risk factors might be a better measure of your success, of your actual fall prevention program. So in that David Oliver article, November 2010, Clinics in Geriatric Medicine, we knew that 30 to 51 percent of the patients who fell in the literature had an injury. So we don’t know the extent of how they were evaluated in terms of their injury, but they had some injury. We knew that 80 to 90 percent of the falls were unwitnessed, and is that maybe something that you’re looking at? Or are you measuring, where are they falling from? Fifty to 70 percent of the falls are happening from bed. Are you doing something different about it? And then we also were reminded of all the fall risk factors that someone has. But in that article, again, David Oliver et al, Clinics in Geriatric Medicine, 2010, the most effective intervention should target both the point-of-care and strategic level. And these were those recommendations, just four of them, that are really the framework for best practice approach in hospitals that we’ve been implementing across the Department of Veterans Affairs for years now. This was published in 2010. But, you know, we all have a responsibility for implementing a safer environment. And when you implement a safer environment for the entire patient population, you’re reducing accidental falls. These next two recommendations, which I was mentioning on this prior slide right here, recommendations #2 and #3, this is where we have a responsibility to identify modifiable fall risk factors when we do a complete assessment. You can measure that. You can have patients who come in and they’ve got 15 fall risk factors, 10 of them are modifiable, five are not. The ones that are modifiable, what are we going to do? We’re going to help treat them, which is the implementation interventions. We have to treat those risk factors. The five risk factors that are not modifiable, what are we going to do? We’re going to help them compensate for them. We’re going to teach them about those risk factors and help them compensate. So you’ll recall that the fourth recommendation is, if someone does fall in our care, we don’t want them to get injured. That’s the injury reduction strategy. But here, colleagues, you can be very clear that if this was your theoretical framework or your strategic model, your strategic planning model — in your falls committee, if you were going after implementing a safer environment, then this would be clearly aligned to reducing accidental falls. If you were going after identifying specific modifiable fall risk factors and implementing interventions to reduce those risk factors, you would be going after the anticipated physiological falls that then brings us back to the Morse scale. This is all to be aligned. Your structure as you’re using your Morse scale to screen for risk factors to do an assessment, implementing individualized plan of care, to be able to reduce risk factors which is ultimately reducing your anticipated physiological falls. You can just see how you’ve just built that model for a very, very dynamic program evaluation study you can launch. And then, if the intervention is to be able to protect from injury, remember our separate and distinct from the fall prevention strategy. So many of you are calculating rates. You have probably a very consistent formula now for calculating rates. I know when the hospital engagement networks started a couple years ago — we’re getting ready to launch 3.0 now — people were still collecting fall numbers and they still didn’t have consistent fall rates. The NDQI, the National Database for Quality Indicators, had operational definitions for all of this rate. So people should have their rates. And then you have to look at, is your formula actually standardized? You know, what’s your formula for fall-related injury rates? And there are different rating scales for injuries severity. And so we just don’t want people reporting the number of falls because you can’t do any kind of comparison with that. But here’s some typical fall rates formula, the number of patient falls divided by bed days of patients times 1,000. So for those of you who are in NDNQI, NDNQI had five options for that denominator, to select the number of bed days of care. And so, even though you might see the fall rates that were reported by NDNQI research, it doesn’t mean that they were clear about which numerators were used in which hospitals, because hospitals, when they sign on contract to determine what formula they were using to determine the number of bed days of care. But then how many of you, again, are you measuring your fall rates by type of fall? You know, if you’re able to showcase at your unit level, your organization level, to the Board of Trustees — you know, only 10 percent of our falls are anticipated physiological falls, 40 percent of our falls are accidental falls, and the other 50 percent are those falls we can’t prevent. I mean, what a picture does that paint when you’re telling a story? So I love for data to tell a story. The fall-related injury rate, the percent of injury, this is where you have the number of injurious falls divided by the number of falls. This will be a percentage times 1,000. So this would be all injuries, and that’s a very different measure than reporting the number of patient falls with injuries divided by bed days of care, which is usually where you get your fall injury rate. So you can see, you have to decide on which combination of formulas best paint a picture of what is happening within our care. So you may be analyzing your fall injuries by severity as well. The severity levels. Again, remembering that you could have one patient who has, on a given fall, sprained a wrist, fractured an elbow, and fractured a hip. Because they had reflexive posturing and reached out before to try and break that fall. So you can have two serious injuries and one moderate injury, all with one fall, one person. How do we profile that? And then you have the multiple — we just mentioned the multiple injuries. So that’s a very different kind of measurement. So here’s a percentage example. As an example of injury rates, your facility had 80 falls in the last month. Of the 80 falls, five of those falls were minor injuries, and three resulted in major injuries such as a hip fracture. And the remainder had none, so there’s your minor injury rate and there’s your major injury rate. So even though you had the minor injury, remember, in older people, minor injuries can be grave. So you still have to carry out that extent of what’s your duration for measuring severity of injury. And ask yourself if what you’re doing is truly enough. Do you really know what’s going on? How do you truly know that your program is indeed working? How do you know people are doing what they are expected to do and that they’re doing it the same way across all people, so that you can have confidence in your outcomes. So we want to know, how do you know if your program’s working. How do you know staff are implementing a practice the same way? Like this Morse scale. I can tell you, for any scale, what it takes to make sure people are doing it as it was intended to be done. Just like the transfer measure, the mobility measure, there’s a component for if they’re not walking, is that you measure how they do the activity with the wheelchair. And then how do you compare your rates with other units, recognizing that units going to have a very different context, a very different population? Beyond fall rates, you know getting to those repeat fallers, we want to really tackle this, you know whose having those repeat falls, and maybe your goal is going to be to reduce the delay of repeat falls, a delay by a day or so, this gets into the time of the occurrence, that’s often times our measure that I have suggested for rehab units that are working with really tough patients that they are dealing with like Traumatic Brain Injuries patients who are so impulsive or right brain CV A’s. If you can delay a repeat fall from occurring by a couple of days, you’ve done a really good job and you have an opportunity to celebrate success. Or maybe you’ll start getting into diagnostic cohorts and looking at the fall rates, so I mean that’s going to stem with orthopedic patient populations or orthopedic hip fractures. You know in rehab, again the stroke population, but still stroke is to aggregated you have to look at left brain versus right brain CV A’s are very, very different. And then getting to the known patients with fall related injuries, looking at the people who come in and they are already injured or they got admitted because of a fall or they’ve fallen. So getting into different populations is really important. So this is an example of really helping the data come alive, because I can design — I have designed a program evaluation around Post Fall Huddles and this is getting into your repeat fallers. So to me, this is an example of a Special Emphasis Population, and you can pick your own, you know you could do this with new moms, you could do this at a hospice population, you can select your own. But you know we want to use safety huddles, the Post Fall Huddles and they were designed to be done within 15 minutes of the fall event, with the patient. And the guiding question [crosstalk] the guiding question in the Post Fall Huddle was always to get at the root cause of why someone fell. So your always asking 3 times, with the patient, what was different this time you were doing the activity compared to all the other times you did it and the other times you didn’t fall but this time you fell. So you’re trying to get to the when, the how, and the why of how they fell. So that you can do what? — Come up with a protective action strategy to prevent it from occurring again. So the Post Fall Huddle, right here you’ve got a time factor, you’ve got timeliness to be able to measure, you can measure who came to the huddle, which would be, was it more the nursing which gives to structure, did a team member come, like a physical therapist assigned to the unit on a given day. You want to get to the structure of it and the process of it, because if it’s not done the way it was designed, and you can’t expect to be able to achieve the outcome. The outcome is making sure someone doesn’t fall because of the same reason. So the outcome of the Post Fall Huddle is you know the reason why someone fell, the proximal cause, the immediate cause, when you know the root cause of why someone fell, then you get into the type of fall that just happened and I just had a conference call today with one of my VA organizations, that I’ve worked with before, and helping them to work through those Post Fall Huddles again on long term care. To help find out, make sure the know, why are people falling in our care, getting to the actions that can be put in place to prevent reoccurrence and then have a plan, and a change in the plan of care. So right here you’re looking at the root cause, that’s essentially the reason why someone fell, they type of fall, when you get into being able to analyze the types of falls is all based on the reliability of your Post Fall Huddle. The actions to prevent it from occurring again is going to be your follow up plan of care, and was it changed. So those things are process measures getting into identifying what needs to be done and how long did it take us to change the plan of care and get it implemented. Making sure the patient is involved, that would be part of the structure process, and then preventing that repeat fall from occurring based on the same root cause. You can have someone who is a repeat faller but in terms of the Post Fall Huddle effectiveness, the true measure of effectiveness is that a patient did not fall because of the same root cause. So you can hear the amount of precision that’s really needed to do the Post Fall Huddle. So the formative measures when you’re getting to — to differentiate between formative and summative, is the structure and it really depends on the day shift, yeah you’re going to have nursing, but there should be more than nursing that come, you’ve got the physical therapist on a unit or a hospitalist and they are there when someone comes, they could come to the huddle. There should be a change in the plan of care, if there’s no change in the plan of care we’re not done. My storyboard on a unit, I want to have these actions annotated on a run chart, on a unit, I don’t want a line of fall rates on a unit, I want a story board on a unit that says when a patient fell what was the root cause, what did we do about it. On the next mark would be, the next date that a patient fell, the root cause and what did we do about it so that the nursing staff can see, that we know why people are falling on our unit and that we’re following up to make sure that things get done to prevent it from occurring again. That’s the heartbeat of my story board. So there’s your structure, and then the process measures would be the timeliness of things, making sure that Post Fall Huddle was done within 15 minutes. The timeliness of getting that change in the plan of care done, but then following up to make sure that it actually got implemented which is making the whole Post Fall Huddle process come alive. But right there you have three measures of process and you’ve got measures of structure. As I mentioned you have the summative outcomes which is preventing that repeat fall based on the same root cause, the same type of fall. And ultimately you’re reducing the cost that are associated with falls and certainly fall related injuries, when you start reducing the burden of the repeat falls. [crosstalk] So in the collection of fall related data it does help us to answer these questions, you know what data and are your variables evidence based and are they are they reliability and validity data and are you collecting data from EMR’s or narrative notes or observation. You know the observations have a little bit more reliability as long as everyone is full trained. If you can observe make sure of similar training, but we know what the limitations are of going back and collecting data and EMR’s are so check box oriented and narrative notes, you know not everyone documents the same way. But you’ve got to have people who are trained to be able to read notes. Whenever your deciding on your data collection, you have to plan all of this out, which is really important, and when I mentioned about the mixed methods, you know when we launched the Post Fall Huddles, we actually collected data on the Post Fall Huddles about the effectiveness of being able to reduce repeat falls due to that root cause, but remember we also did qualitative research. We actually did focus groups with the nursing staff to ask if this was something that was really value added or more burden. That’s when we learned the true meaning for them, the nursing staff, and this was nights on med surge, what was so powerful about the Post Fall Huddles was that they didn’t have to do with the fall alone. So that’s the value of really getting to talk to people, get their perceptions, and rather not just collecting numbers. But when your collecting numbers, yeah you’re going to collect and drill down on data, you’re going to define the data elements, you have to have operational definitions that everybody is very clear on what does this mean. It’s just like being clear to link everybody when they are using the Mores Scale to only one type of fall, the anticipated physiological fall, is that you cannot look at the effectiveness of the Mores Scale in relationship to all falls. That’s emphasized. So you have to define your data elements, you have to determine the frequency of data collection, you know — Be realistic about it. And then develop data collection tools, and pilot test them and make sure they are going to work and that they are clear for anyone who is going to use them. That’s such important parts of the planning, and then have a data code book so that anybody who comes and starts collecting data or is going to be involved in your Quality Improvement Project or Program Evaluation or Program Research is understands what the data means. And determine where to find it so the people can go easily go back and find that data. Then you establish your reliability, so those are all the planning parts before you even get started in looking at the data that you want to collect. So for example, improving your Falls Program, your interdisciplinary care planning, you need to be able to define, what are you going to include in terms of your interdisciplinary team, core members versus others, what their involvement and assessment of patients and their implementing of the care plan. Or you can do the Post Fall Reviews and I just gave you a framework for being able to do that. Or maybe you’re going to integrate technology at the point of care beyond the [inaudible] which we all know doesn’t work with fall prevention. But look at your story boards, because the story boards are what really paints the picture, and people that’s their go to image for units to be able to know the success of our patient’s safety efforts. So this is an example of a fall rate by patient days, 1000 patient days, and you have your upper control limits and your lower control limits, those are the dotted lines around your average. You have variations that occur, on a monthly basis those are the data points that you will see, this is meta up data over a couple of years. So ultimately what you are trying to do is to run this rate down and reduce the amount of variation that’s allowed around the mean. That’s getting below the mean as you can see, the national mean. But remember, this is all fall rates, it only begins to paint the picture. It doesn’t tell much when you look at all falls as if they’re the same. When you get to fall rates by type of fall, now this is where you have a story. So yeah, you’ve still got that overall fall rate but you can clearly see that the majority of falls that are happening, when you look at types of falls, are anticipated physiological falls. If I know that, when I come to visit an organization or visit someone else’s unit if it was in my hospital, I would know exactly what’s the problem. If most of our falls are anticipated physiological falls, we know we don’t have assessment, we’re not doing assessment, we don’t have an interdisciplinary team. Because that’s all the evidence to be able to reduce anticipated physiological falls and multi-factorial. You have to have an interdisciplinary assessment and you have to have the interdisciplinary fall prevention plan of care. Most of what you find in electronic medical records in the hospitals is a nurse managed program. So, we have a lot of opportunity. This is an example of number of days between a chart and between serious injuries. This gets into your survival analysis. So, every data point is the number of days we went before we had a serious injury. So, on the Y-axis, you have days between serious injury. On the X-axis, you have time. So this paints a very different message too which is really — it shows how protective our environments are. And then this would be an example of an annotated run chart, where you are capturing innovation. And I don’t know, when we were doing our TCAB work with IHI, we actually captured all of this. I just didn’t have my drop down arrows to the point in time, but we actually kept track of all the innovations that we were implementing over time, and how we were able to successfully continue to sustain them so that they became bundled. And this is a really fun storyboard to have on your units is to be able to capture: What is that we launched? When did we come up with an idea? When did we get buy in? How long did it take us to prepare to launch this? What was our pilot testing time? And when did we perfect everything that we could actually go to full scale implementation? That’s a really fun run chart to do with the nursing staff. This is a different run chart where we are actually just looking at an outcome. You know, for reducing the amount of restraints, the number of restraints over a period of time, and some of you will remember the days in long term care or even in hospitals when you used restraints to prevent falls. So your restraints went up and your fall rates went down. In this case the restraints were going down absolutely because we went restraint free. Or if given kind of a control chart for a pilot unit where you’re looking at the fall rate before you launch something then you launch the whole program on the pilot unit. And you looked at decreases in the average monthly fall rate when you launched an intervention after implementation. So this is a phased in control chart, where you had to phase of implementation and no close fall assessment. This is a G-Chart, where you look at the number of days between serious injuries due to falls. Another example that you have seen before, they just go ahead and change the X and Y-axis on this one. But anything that you do, you want to paint a picture, because its not just a number. You want to know, Is your program indeed improving? Are your patients safe? How many of you can really say with confidence that your patients are safer? And with falls this is so important because of all the work in the hospital engagement network, falls is the only one average event that is not going down. And how many of you can say that the patients you are cognitively alert are even more confident? Because they came into you care that they are going to be better off after they leave you. So there’s lots of ways to paint the picture, and that’s where we start getting into implementation science. I love this world and I’m so glad it has so much emphasis because honestly at heart I’m a clinical nurse specialist. And then as a clinical nurse specialist you’re implementing programs and helping staff to be clinically confident. And feel good about the work that they’re doing. So, you work yourself out of a job because you’ve built clinical confidences and structures and processes so that you know things will be done. So, implementation science is the study of methods to promote the integration of research, findings, and evidence into health policy and practice. And it helps us understand behavior of healthcare professionals and stakeholders as key variables in the sustainable uptake adoption implementation of evidence. So, this is closing the gap in practice. So, these are the questions that actually get asked in implementation science. You want to be able to create generalizable knowledge so that you can say, “Why are things working?” And the questions that you ask is, why do things lose effectiveness over time? How do things fade away? They lose importance they lose relevance. There’s something that we all thought was really important to be done. Or, why do tested programs sometimes exhibit unintended effects when transferred to a new setting when people don’t even control for context? How you roll something out on a day shift is very different than a night shift. How you roll something out with a group of novice nurses is very different than a group of expert nurses. You know, another question you can ask is how can multiple interventions be effectively packaged to capture cost efficiencies and reduce the splintering in health care systems? This has a lot of promise in today’s world when you look at all the team efforts and the pax programs to look at bundled care. So, implementation really does matter. Because it helps us to identify and adopt what works and what’s not working to understand what — So that we can reduce the barriers to implementation and achieve successful outcomes. And we have evidence based programs and practices still to — that need to be delivered, but we have to assure fidelity and reliability across all settings that we’re working in and one of the strongest messages coming in from the research is fidelity – the quality to which treatment is delivered. And this is critical to our implementation for a successful outcome. So it is not to the faint of heart to change practice, you know, it can take two to four years and you look at trying to do all that needs to be done in terms of planning, installation, implementation. So there is the stages that you have on the slide. Even being able to explore “is this possible for us to do?” Then to get what we need to have in place and then to do the initial implementation and testing, the pilot testing. Be ready for any kind of push back and resistance and this is — remember the world of innovation diffusion and you’re getting ready to deal with a lag. And then get ready to implement in full scale implementation. And then innovation sustainment. So I can tell you, when we were launching the barcode medication administration years and years ago, you know when we had national implementation, all the lessons that were learned over time to be able really make it fully work and fully operational. So the key implementation support is you’ve got to have implementation teams. And many people I’ve moved into the unit-base champions for falls communities have practice where you’re building expertise. And having that data driven feedback, remember it’s not just the number of falls, it’s — you know, how is it working? How is your program going? How is this structure working for you? What are the processes? Are people coming? Is it timely? Are you getting the feedback? You know, developing the confidences. And then supporting the fidelity of assessments and formative program evaluations. And then you get your practice to policy feedback mechanisms because you just don’t want to put something out on paper and think it’s going to be done. And that involves team leadership. And leadership is so important. All the falls — the last four falls tool-kits that have come out, getting leadership on board is one of the first chapters of all of these works. So getting leadership on board to help with fall prevention is just still a major priority in today’s world. So, when you look at high reliability organizations, the high reliability organizations are those that are achieving a high level of success or reliability despite dangerous or hazardous conditions. And we’re very interested in this because we all do want to minimize errors and achieve exceptional performance in our patient’s safety and quality. And there are several studies that have indicated core processes in healthcare that are defective 50 percent of the time, so we have to reduce those. And I know you’re going to be looking at Lean Six Sigma as part of this as well and Kate’s going to be helping you with dealing with clinical confidences. But you know, patients receive only about 50 percent of the time, the care that they should expect, but in a healthcare delivery system. So we need to be able to do better than this is what we want to be able to do. So this is when you can get into fidelity assessment. So fidelity is the extent at which the delivery of an intervention adheres to. You know, it’s done as it was intended to be done. And you establish fidelity measures. You know this is critical to achieving outcomes. This is where often times observation can tell so much. You know, how well is your hand off being done? Is it being done as it was designed? As it was taught? Or that we — I think that the last time we were together we talked about purposeful rounding which I’m not a fan of, but for those who are doing that, I haven’t found anybody who is implementing purposeful rounding the way it was designed to be done. And they’ve morphed it all over the place. So making sure things are being done as it was intended to be done — Toileting! Scheduled toileting is important. So fidelity is concerned with the structure. Do we have everything in place and the processes to make sure it’s done? The whole issue of adherence is it being delivered as originally intended and designed like the floor mats are only down when the patient is resting in bed. Hip protectors are only for those patients that at risk for osteoporosis or already having hip fractures. It’s that whole issue. Is it being done as it was intended to be done? Exposure gets into the number of sessions or treatment that were supposed to be done. Are we really making sure that someone who’s supposed to start doing early mobilization is actually having these sessions implemented. Quality gets into competency, skills, and enthusiasm when you’re looking at the quality of care delivery. How passionate are people about really helping to protect people from an injury if they were to fall. Protect that 85 year old. Participant response of this is getting people engaged. This is engagement and retention that you have people that are willing to make this stay important. Then program differentiation helps us to identify unique features between programs. So those are really important components of any kind of fidelity and assessment. Realizing that you have to control for context, compliance, and competency. So there’s a lot to fidelity assessment. And these are the questions that get answered. Is if you change a structure — If you actually change structure what effect will it have on process and outcomes. You have to think through all of that. If you implement a change in practice what approach are you going to use to implement it and what did you use? This gets into being able to detail what is that you tried to do and how did you implement. What was your process? And when I come back to purposeful rounding — As people have tried to make sure this gets done. As I’ve mentioned, they’ve morphed it so much that people don’t keep track of all the changes that they made when they were trying to this one approach to patient satisfaction, improving patient satisfaction. And how many of you can say how confident you are that something you’re testing is actually being done. If you’re checking for postural hypotension, that everybody’s doing it the same way. I just had this conversation today with a group of people. And then measure reliability, the accuracy and timeliness of the implementation. That everybody’s doing it that same way. And how do you know that a change that you’re making is indeed effective. Did you really link it to the correct outcome. It’s really important when you plan this out. And reliability compliance is different because reliability is the capability of a process or procedure to perform it’s intended function over time or between people. And I know we talked about what this meant earlier. So you’ve got to quantify reliability, and then some people do the defect measure. I know you’ll do Lean Six Sigma. We’re not an operation why we’re not an assembly line. We have a lot of floating components, and we’re very complex. So this sometimes gets hard to do. Still there are ways to make sure that people are doing the same thing the same way like the orthostasis. But it can help us to standardize our approach and build some decision aids and reminders. Just like in our electronic medical records, in our post-fall note, we actually built in how to check for postural hypotension. The time frames for measurement. We had flags in there that people could click on and get information about, and if they had any questions. Same thing with the Morse Scale. So you take advantage of preexisting habits and patterns and then you help them be able to make sure the desire to act on becomes the default rather than the exception. And it helps us to reduce redundancy. And that’s when you start bundling related tasks. And it all depends on teamwork. So remember the change practice is not for the faint of heart which brings me back to the PDSA model. I love this model, and I’ve got built in to hear about Donabedian’s model where you have to control for structure and process. If you don’t you’re not going to affect outcomes. You can do all your PDSA cycles but if you don’t make sure you have a core team, keep them going. If you don’t make sure processes are in place than it doesn’t matter all these iterative cycles it’s going to be very tough to be able to implement. You’ve got to have structure and process. So we all should be planning, and that’s the most important phase of anything that we’re going to do. And in this planning you decide what you’re going to do. And then you operationalize it. And then you have measure. And then you have data collection tools. So you plan and then you’re going to do. And with every time you implement something, every small test of change, people have to keep notes. What worked, what didn’t work. You’ve got to take notes. And then you’re going to study it. And then you’re going to make a decision for what to do next. And that’s the study part. After you’ve looked at all of this, you have to look at what do we need to do. How are we going to reduce barriers and increase facilitators. And that’s when you act. So each one of these steps are really important, so you’ve got to collect data on it. Because every PDSA cycle is an iterative process to get something to work. You want it to be successful. Whatever it is you’re trying to do you want it to be successful. And you want it to improve patient outcomes. So remember to go back to the model. You’ve got to look at the program. And have to define the program. Then you have to ready for implementation. Then you have to be clear about the outcome you’re going to measure. And be precise. And don’t over collect data. So the PDSA model, remember this is a theoretical approach to change. This is planned change theory. So whenever you’re implementing change you can back to a theoretical approach. And keep track of those PDSA cycles because essentially if you can do that then you’re going to build a toolkit for what worked. It doesn’t mean because it worked on a rehab unit it’s going to work on a hospice unit. Or it’s going to work in psychiatry. But you can maybe have generalization to another rehab unit. But you have to give it a try. And that’s why making sure what it is measuring what it is intending to measure so that we can know whether or not we’re are making an improvement. So remember that colleagues, that principal and guidelines for testing is when a test of change — Be specific. What is the question that you’re trying to answer. What test of change. It requires a theory and prediction. Can you make a difference in reducing accidental falls? Can you make a difference in reducing anticipated physiological falls associated with toileting? I can have a very specific intervention for that or practice change. And then testing on a small scale and collect data. And then get ready to build upon that so that you can do the iterative cycles. And then plan for changes as you go forward. Everyone one of these is built on a hunch. Something we want to do to be able to make improvements before we make wide-scale changes. Because if you don’t it will fade away. And that becomes the very first premise of why are we doing implementation science. We want to know why do things fade away. Why can we not get things to change and become acculturated and know with confidence that people are going to do this because it’s so important. And we know it makes a difference in patient outcomes. So implementation science and strategies they do help to trend and analyze PDSA cycles. They are empirically designed to give us insights and tools for high implementation improvement with diverse and complex real-word settings. And unit-based champions are a winner. And then you’ve go to build your guidelines and your models. If you don’t have them already, your manuals. And make sure that you’re holding people to the fire, that you’re having the meetings. Just like I said in terms of post-fall huddles, we made huge successes with post-fall huddles but this took us time. So we went over the formative measures and summative measures, and we were able to show a reduction in repeat falls. And reducing the amount of people who were falling more than once. So this remains the top report at-risk event even though in the VA our primary outcome was to keep making sure that people don’t injured as a result of a fall. So Michelle, as I close, I always encourage everybody to approach everything they’re doing with data collection and quality improvement with rigor. And I encourage people to go back to the SQUIRE guidelines because the SQUIRE guidelines were designed to help you to be able to publish your work. But to publish quality improvement and program evaluation you have to have rigor, and you have to have data. You have to have questions, you have to have measurement tools, and you have to be able to explain your implementation. And I’ve published multi-scale, multi-sized quality improvement projects, and I know you can too. So Michelle, I want to thank you so much for inviting me to present. I know it was a lot of information. And with that I’m going to go ahead and then pass the ball back over to you Michelle. I hope because I’m not seeing you here now on the — Let me just see where you are so I can pass the ball back to you. Thank you so much Pat. So I know we are close to the top of the hour but I did want to give participants an opportunity to ask any questions that they might have of Pat, while we have her here. Especially the measurement related questions. So feel free to chime in if you do have a question that you’d like to ask. Or raise your hand. And always remember too, I’m always available to you after the call. I’m not always a phone call away but I’m always an email away. Thanks so much Pat. We know and we appreciate that. So I don’t see any hands at this point. But I just want to thank everyone for joining us today, and I hope that this has been helpful information. I think this slide deck is going to be very valuable for people to go back to and look at. Unless there are no questions, just want to thank everyone and remind you that our next webinar is on Thursday, October 20th. Excuse me. And thank you again Pat for — Was there a comment. Yes, I had a question. Okay, go ahead Terry. Is that Terry? Yes, that’s correct. Okay, go ahead. Terry Hastert from McDonough District Hospital in Macomb, Illinois. One thing that my experience in quality is the whole Plan-Do-Study-Act is all associated towards run line charts, time ordered data. And that has very — Rules, you have control chart rules to identify that a special cause has occurred. And this type of science is very different from clinical trails, which take time out of it. And so it seems to me in medicine, in hospitals lets say, where you’re providing processes. And taking care of folks. There’s this disjointedness from the people who are in charge and giving orders. The physicians that are very much use to and comfort with these clinical trials that don’t have time in it. And so I think they see this kind of study where it’s time-ordered information as at a different plane than clinical trials. And I don’t know how to beak that cultural difference. Yeah, and thanks so much Terry for that question and the comment. I think it just depends on what you’re implementing. It’s just like the post-fall huddle. When we launched the post-fall huddle, it wasn’t a clinical trial. We didn’t have a control group or an intervention group. So we modeled it after the after action reviews in the military where you bring people as close to the event together as quickly as possible to try and figure out what just happened. That still done in the aviation industry. That’s how we built it into a process. The time measure there that it should be done within 15 minutes. So for the PDSA cycle, that’s one of the things that you want to measure. Then you would be one the process measures that you have in place. Did people come together within 15 minutes. And a lot of times when I hear people talk about post-fall huddles, they’re completing their post-fall huddles at the end of the shift or two hours later. Well not that’s as it was designed. So that’s why even in a quality improvement initiative you can still have a process measure that’s a time measure. But it’s just very different. It just depends on what you want to do. Just like floor mats. If you have someone who gets admitted because of a hip fracture or has a hip fracture already it’s not okay to wait 48 hours to get them floor mats for the bedside. It still comes back to what it is you’re trying to do because if it doesn’t work if it’s not timely it’s not going to be effectively for us. And being able to protect someone, if you will, in terms of the floor mats. So I hope that that just kind of sets the stage. You have to decide for every intervention that you’re implementing — If this is the kind of rigor you want to do. If you’re going to really launch something, an intervention, we have to have commitment to it because it takes a lot of staff time to make something work. To get up and running for it as you’ll hear from Kate. And then to be able to launch it. You don’t want to just role things out without having some expected outcomes. So that I think is where the PDSA cycle doesn’t have to be linear but it has to help with looking at how do we go for a small scale pilot testing to being able to do a spread. Thank you Pat for your response. Oh, you’re so welcome. Thank you. Any other questions that anyone would like to ask? And I will say still Terry where I think we got so much traction with the PDSA cycle is from IHI. Because IHI was a TCAB project transforming care at the bedside. Really wanted to get action-oriented, and for people not to wait until clinical trials got published. To be able to get people involved in change. And to not have people be implementing change after they just got the policy that was going to be implemented hospital-wide or unit-wide. That we had to have a say, and people had to come together to see what works and what doesn’t work so that you have more probability for success. I think that they really get credited for giving us a lot of traction at the point of care about doing the PDSA cycle so that you can be successful. But if something isn’t value-added, just like the post-fall huddles, if the nursing staff on nights had of told us a very different story who knows if we would of continued them. When you ask people is this really worth it to bring people together in the post-fall huddle. And you hear something like, yeah we didn’t have to deal with it alone. That was the most important thing. Well that just sets the whole stage for this is so important we’re going to go forward. So I think whenever you look at whatever we’re going to do if they’re really a commitment to it, we want it to be successful, otherwise it just fades away. And we’ve all been doing it. That’s why we all end up drifting up to universal fall precautions but doesn’t work and the score. And it’s like well how did this happen? How can we make it different? How can we make it different? All right. Thank you Michelle. Yeah, thank you so much Pat. Of course. And thank you for sharing all of that valuable information. And thank you everyone for joining us today. So, again, we will meet next time on October, 20th. It’s a Thursday at 1pm. And we hope you can all join. It’s going to be another great presentation. So thank you Pat so much. You’re so welcome. Thank you. Everybody have a great afternoon. Bye-bye. Bye.