About the Webinar

Join us as we explore the competitive landscape of clinical trials and how feasibility and participant recruitment and retention play a pivotal role in the success of a study.

In this webinar, our experts discuss:

• Developing site selection and recruitment strategies that are not one-and-done and are continuously modified and updated over the course of a clinical trial.
• Leveraging data and insights to have a better projection of site enrollment.
• Instituting the best ways to have high-impact improvements on site selection and recruitment.

Transcript 

Cristin MacDonald:  

Thanks for joining us today. I’m Crissy MacDonald, the Vice President of Client Delivery. I oversee the consulting and research services here at WCG. Part of that is the site ID and feasibility component, and I’m here with my colleague Seth. 

Seth Halvorson: 

Hello, everybody. Seth Halvorson, General Manager of the Site Solutions business in WCG, which has our recruitment and retention business, as well as our study startup businesses, network, and our CTMS, so all pertinent to the topic. So, let’s start out with the facts that we all know and live with every day, and you’ll see variations of these numbers, but they’re all going to be directionally the same. It’s recruitment and retention is a challenge. It’s a challenge for studies. It’s a challenge in the industry, and it has been for a long time. We’ve seen different ways in why it becomes a challenge, different pressure points, and different stressors causing at different times. But the end results are pretty consistent. 37% of study sites do not meet their enrollment targets. 15% of sites fail to enroll a single participant. 80% of studies fail to meet their enrollment targets. 85% of studies fail to recruit enough patients. So you can see across, and you’ll get to different people with different numbers at different measurements, all very much giving the same message. This is a problem in the industry today, and it has been for quite some time. 

Cristin MacDonald:  

And I think the second set of numbers here really talks about why. Why is this happening? And the reality is that in the current landscape, we know that only 3% of investigators are participating in clinical research. And while that number may fluctuate, we’re talking it’s fluctuating by one to 2% Nothing drastic here. And of those that are participating in clinical research, when we did our WCG site industry survey last year, we found that 63% of respondents said that site staffing is really the reason why they’re not meeting those targets, and then 48% of sites said that really recruitment and retention are the biggest reasons for not meeting those targets, which kind of makes sense, right? We know that enrollment is closely tied to a site’s resourcing levels, so ultimately those two things being the top two detractors to recruitment and retention make a lot of sense. But what does that mean, sort of in the downstream effect? Obviously, we’re not on time, but overall, things like poor site selection, the inability of sponsors and sites to kind of predict that rate of enrollment and subsequent need for steady rescues that results in these kind of lack of recruitment targets being met really leads to at least or excuse me can lead to an increase in the cost of clinical trials by about 20% or more. With that, before we get into some things today, we just want to understand a little bit more about all of you and why you’re here, and how ultimately your organizations are selecting sites today. So, are you predominantly using relationships that you have in house via your medical monitors, your clinical operations teams, in house data, subscription data, historical experience for vendors. All right. So sharing results here. This is pretty amazing, and is sort of going to alter the narrative that I tell a little bit here in the context of the first number one way that organizations are selecting sites today, according to all of you here, is based on relationships. Second was in-house data. Third was historical experience. Fourth was vendors, and then lastly was subscription data. So really interesting here. But what we’re going to talk about is kind of what the traditional site identification process looks like today, and traditionally it’s usually a data-driven process with two separate mechanisms of looking at that data. Right, so we have that first column there, that site identification, and that’s where we’re using data or relationships. In the case of everybody here on this phone, to develop lists of sites that are potential fits for the study, and then through that feasibility process, which is on the bottom there, that’s done where we collect data from sites to account for kind of that final identification process. But as I said, the first step of that site identification process is usually leveraging data to determine how many sites we actually think we even need to complete this study on time. Right. So typically, sponsors are looking at clinical trials that have similar indications to determine what that average enrollment rate is, and then they’re going to use that number to back into the number of sites that they need based on the timeline in which that they want to complete this clinical trial. So the challenge with that is it’s often very high level, and it’s completed at a stage when usually there’s no real protocol that’s final. We’re working with a study design, a potential draft protocol, but even that is probably way far down the path from where we are right now. So as a result, there’s many contributing factors that aren’t considered. Right? For example, what’s the competitive landscape? Are there more trials in this indication now than there were historically, where you were basing those enrollment rates on? Right? Is the protocol unduly burdensome to patients or sites in comparison to other active trials in the same indication, or is the eligibility criteria more cumbersome than competitive trials on the market today? And again, it’s hard to do those active comparisons without being able to have a full final protocol to look at. So all that’s to say that those enrollment rates that are leveraged are really key to determining the number of sites needed for a trial, and at that point in time, they’re kind of educated guesses that have to be validated, right? 

Cristin MacDonald: 

But ultimately, once we determine the number of sites that are need to be identified, we work on evaluating that data. So we’ll create lists of sites with a bunch of different columns that we look at independently. Things like what’s the historical performance, how many clinical trials have they participated in in that indication, what are their startup timelines, what do historical enrollment performances look like, do they have regulatory compliance issues or quality indications like protocol deviations, data timelines, claims data, etc. What we don’t do is look at all of those together in a way that select the sites that are best combination of all those attributes to align with the recruitment and retention strategies that we’ve developed along with those enrollment timelines that are being planned. So something that we do at WCG in our site selection services is that we can have an investigator match score that really uses a proprietary algorithm to weave together each of those components in a way that takes into account these types of data points to come up with an ordered list of how to best approach that site selection to successfully implement that study, those scores with our services help to identify some of the key challenges of competitive trials. So whether there are quality or compliance issues, in the conjunction with the ability to provide hard data to a sponsor who are often having to buck the trend of site selection that’s made exclusively on relationships. So it’s important to note, though, that once we use all of that data and we come up with this list, that list that we come up with is actually drastically larger than the number of sites that are actually needed to participate in the clinical trial, so we make that list and that site identification step before we move into that feasibility questionnaire and gathering that information from the site. We often work under the assumption that it’s going to take three to four times the number of sites on that list to get to that required number of sites that’s necessary, and that three to four times is going to vary based on the complexity of the trial, if it’s a rare disease, etc. And the why is if you go back to that data that we presented on the first slide, there’s not a lot of sites that we’re going out to, right? So at WCG, when we look at historical trends over a five-year period, the number of sites declining. Study participation has increased 27% over the last two years, and the overall response rate has decreased by 15% And resoundingly, the rationale given by sites is often the result of the site’s inability to take on more research at the administrative level, so a decision to complete those feasibility surveys for new trial considerations is kind of largely dependent on whether they have the resources to effectively conduct those clinical trials. But again, now we have this list of four times or three times what we’re looking for, and we reach out to those sites to get feedback. Right, so we’re trying to get data collection in that feasibility segment, where we’re looking to understand the facilities and the resources at the site, as well as practical consideration like the site’s potential patient population, demographics, any referral geography or relationships, and this is where we hear feedback from the sites on the limited information that the sponsor has provided on the protocol. So again, at this point in time, we’re reaching out to sites sometimes not with a fully final protocol, and so we’re getting feedback from a site on really what is an incomplete set of information that they’re going to have to execute on later down the road, so that becomes important a little bit further down in our conversation here. But I think the biggest question that we have is what role do these feasibility responses play in our planning, and do they really hold a larger role than the internal and external data sources, or even the relationships, as you all identified as how you’re doing it, that we’ve discussed in site identification. So again, just to kind of reach out to all of you, what percentage of sites think sponsors trust their feasibility results? And I’ll make sure that I actually show you the potential responses here. So when we ask sites, do sponsors trust your feasibility results? 

What do you suppose the response was? I think it’s slowing down. So there’s the poll results. It’s a relatively even split between like 20 and 60 percent, with 61 and 80 being next, so we’ll go somewhere between 21 and 60 is sort of the response there. Now I’m going to ask you the reverse question, so read it carefully. On the sponsor side, when we ask sponsors, what percentage of sponsors have trust in a site’s feasibility answer? So do sponsors trust the responses that they’re getting from their sites in feasibility questionnaires? 

Seth Halvorson: 

I’ll note too that the first answer was zero to 20. You saw that jump off as the whoever was the fastest on the trigger right to zero. 

Cristin MacDonald:  

Yeah, so it’s interesting to see this because again, that 21 to 60 is in the lead again, and I don’t think you guys are too far off though. If I see the distribution here, but this was a presentation from Scope just a couple of weeks ago, and it’s really interesting to look at, right? Because what you can see here, and this was a large audience for those of you who have been to Scope, when those same questions were presented, these were the results. So, 72% of sites think that sponsors actually trust and take into consideration those feasibility responses, but only 13% of sponsors have significant trust in these answers. It’s a huge disparity. It’s insane. It frankly, kind of confirms what we used to do. So back in my day, when I was running clinical trials, we would get the feasibility response from the site, and we would divide it in half, and that was what we used to do our sort of enrollment projections moving forward. But frankly, it’s pretty unfair because what we see here is the site providing feedback at a single point? So we talked again. It’s at a single point, which sometimes is at a single point that’s not even a final protocol, and things have changed. So why is this happening? Why is there such a lack of trust there? And this happens because sponsors have seen the data of what a site can enroll from either their own internal, their subscription data sources, data provided by their CRO or other vendors, as well as what the site submitted on their feasibility questionnaires, and they can’t unsee it. So what you see on the right there is it’s a really great quote. It’s by Adam Grant from a book called Think Again. I highly recommend it to anyone in our industry as we’re trying to sort of evolve and move forward and undo all of the ways of working that we do just because we’ve done them that way in the past. But ultimately, we treat the site identification and feasibility activities and recruitment and retention activities as one and done, particularly in our mind. When the reality is that that data from both internal and external sources, as well as from the sites, is constantly evolving. Any of the data we have on historical studies is arguably irrelevant because of the changing landscape that’s happening. Any data that they provided on what patients they can enroll have potentially changed if those patients are now enrolled in a different clinical trial, or have moved to a different location, we really need to have a dynamic approach. So, at any given time, as I said, the number of competing trials at a site can change. Delayed site selection could have been the reason why they accepted a competing trial, or communication issues. But ultimately, there could even be a similar study with a different investigator that you’re just not aware of because we usually look at this from an investigator-centric approach. But really, that enrollment and planning and target number of sites is based on that protocol design, which I can’t reiterate over and over again is not often final. And in terms of facilities and resources, and those recruitment and retention strategies, do we know that the circumstances in which they answered that feasibility questionnaire is the same as it is three months into recruitment when they’re struggling? So as we said, enrollment is closely tied to resourcing levels. So imagine that tipping block if they started out and answered a feasibility questionnaire with three dedicated study coordinators, and now when they’re actually executing on that study, they only have one. You can look at other things too, like warning letters of investigators, and that’s true. It’s important from a regulatory compliance standpoint, but there are other things that we need to look at as sponsors and oversight as well. Things that are kind of quality indicators of the success of a site, like protocol deviations or data queries at sites that are again closely tied to resourcing at times. 

Cristin MacDonald: 

Sometimes even training can be the challenge, but ultimately, what happens is with every change in circumstance, be it a protocol amendment, a changing landscape at the site, a change in landscape in the external environment, we need to readjust our planning and our thinking, and we need to kind of unlearn that story we had in our head and the expectations that we set forth, and begin to listen to the site when they say what their challenges are and what their experience is while trying to execute. 

Seth Halvorson: 

So, following up on talking about the  overall and the set of assumptions, as Chrissy indicated, that you cannot overestimate or over I would say overestimate the impact of a set of assumptions being built up, and we’ll go into detail on that. But before we talk about that extensively, it’s really important to understand that recruitment and retention, although being a pain point in the industry for a long time, it’s actually really simple in its components. The components are a universal process, and if you start by breaking it down, you can reevaluate the situation and the and the context of how we’re looking at this to get to different sets of solutions. So if you look at the participant journey, and we’ll be careful not to use participant, patient, subject all across the board. Understand, there’s a lot of different positions on which words you we try to use participant just to be universal on that. There’s an identification stage, which is identifying that individual to see if they can fit the IE criteria in the first place. There’s the enrollment stage, which is bringing that person into the study or otherwise further reviewing them and seeing if their qualifications of study, and then ultimately getting them into the consent. Then there’s the retention stage, completing the actual visits and doing all of the things necessary for each visit for the protocol. And then the documentation, which happens interconnected with retention, which is getting all of the documentation for that individual that is that are the representative of the study visits, so that you can submit, you don’t submit people to the FDA. You submit the data, and so if you don’t look at the overall recruitment and retention process as interconnected from the beginning of the identification of the individual through the last bit of data about that individual, you’re missing part of the picture. And so when we look at it, we talk about it, and we talk about how site selection impacts the recruitment plan or the retention plan. You really have to look at what the sites are able to do with that study at that moment in time across the entire participant journey to result in getting that study in and getting the results you want within the time schedule and  a practical plan that’s set up. So again, keeping it very simple intentionally, most site selection or a lot of it, although relationships being number one does is an interesting conversation to this because a lot of times if data is the number one driver, a lot of one of those data points might be a survey or some other information going out or some other source indicating that the patients within the EMR of that site are sufficient in order to hit an enrollment goal or an enrollment target. We call that the internal contribution. So that’s within the virtual four walls of that site: their EMR, their medical records. Those are the patients that are there, and that’s usually a chart review or some other process that the site is going to identify those individuals and then begin the enrollment process. The internal review referral number is a knowable number. Knowing it and getting to the actual number is different, but it’s a it is a number. It exists in a moment in time because those individuals are or are not in the records. When you look at what an example like this, if we’re going to assume that that the site’s goal is 12 enrolled participants, and the internal population, once you do the chart review, once you do all of the steps for what would be inside those four walls, gets you to seven enrolled participants, intentionally making it simple, you can see there’s a gap of five for the study’s goals. That might not be a gap of what the site’s able to contribute. That might not be a gap of what the site’s able to contribute to the study. That might just be what that level is. So what it does is it tells you once you know that level, external referrals are going to be needed in order to hit that five or what other decision points you make. Now, it’s true that an individual in certain indications, asthma, atopic derm, might walk into the site. It could be a clinic that that’s a care provider. 

Seth Halvorson: 

They have they’re doing things other than clinical trials, and individuals do walk in and they do present. And so it’s absolutely possible that will happen, but you have to know if that indication is one that it will. Rare disease not as likely. Certain types of oncology different pathways. So I’m just understanding this overall. If you have if you have a gap, you need to go into your external sources and you need to understand whether that’s going to be referral provider networking, media outreach, or community outreach, you then look at the gap to figure out what the contribution is going to be to get to your goal. Now, why is this relevant into site selection? Referral provider networking is only going to be relevant or applicable to what is near and around that selected site. You need to have provider networks that are available for that indication in a geographic proximity that is that is reasonable, or whatever the indication is. Media is the site able to process the media referrals that are generated. If you’re running an outreach campaign, is that site structured? Does that team have the capacity, and is the study structured properly in order to manage that community outreach. Are there events near and around that the site that you’ve selected that you can go to for a community outreach campaign? Is the indication one that makes sense at the same time? So you’re looking at all of this as part of what Chrissy was talking about on that overall site selection, and then as that comes into the recruitment plan, you need to not only know what that internal contribution potential is going to be-that’s kind of the low fruit if you think about it-but you need to know what the capacity is of the site and the location of the site to be able to understand if these other factors are either going to be relevant. Now, if they’re not, then you have an option. You can again wait until that individual walks in. Now you’re on your clock is the universe’s clock. It’s whatever timing happens as the event of things as they unfold. You can scale up one of the external sources that it makes sense. If you’re in a heavily populated area with a lot of hospitals and clinics nearby, referral provider networking might be fantastic. Works very well in Europe and other areas. If you’re in an indication where there’s the community outreach could have health fairs, whatnot, great, bring it up. But you need to get to that point where you understand what that is overall, based on all of the decisions that have been made to know what to do. Otherwise, that site’s ability to contribute to your study targets aren’t going to be able to be met. Now, what does it look like in practice? It looks like this: you’re going to have a lot of different referral sources from a lot of different for different sites at different levels, because not all sites are the same, and not all referral sources contributions are going to be the same at each site. So, really understanding what Chrissy was talking about how the sites are structured and understanding that deeper level of knowledge about how those sites are going to be able to be able to meet your study at that moment in time is critical in order to getting to an understanding of what to do when with your recruitment and retention plan, because as everything we’re going to talk about, nothing can be set it and forget it. There, you can’t put something in place and then expect it’s going to have an outcome. It won’t work, and the reason why is there’s a whole set of assumptions that all of this is built on, and this entire process is predicated on a set of assumptions because we don’t have always either complete information or we don’t have final information. So as Chrissy was talking about the protocol, protocols the key step, the key, the initial aspect of this. It has your IE criteria, it has your schedule of visits, it has everything set up. 

Seth Halvorson: 

Your IE criteria is going to tell you what that patient population is, and then roughly how many of those are going to be out there is going to be based on understanding what how it’s restrictive or how inclusive it is. Your schedule of visits is going to speak to what people are willing to do or what sites are willing to do when it comes to the more complex how many procedures. Looking at if there’s colonoscopy required on multiple visits. That might not be an individual might not want to participate, or a site might not want to participate. So, what is the impact of that study design on your ability to recruit and on your ability to roll and retain your site selection? Your sites may or may not have the capacity to be able to deliver that protocol at that time, there might be easier protocols to work with, and it’s a key factor overall in which sites are willing to able to accept what and when. And then, of course, training you got to you got to bring it together. Training is often overlooked as something that is an afterthought because it’s a requirement, it’s a must-have. Good training is really hard to beat. Making sure that people properly understand what needs to be done so they can know how to work? Because you’re dealing with people and you’re dealing with constant changing events, you need to have a really good, clear, crisp understanding of it. And so, what does this set of assumptions and these dependent assumptions along the line do? They show up in reality your recruitment and retention. So there’s an assumption in your protocol that that IE criteria exists, that those individuals matching that IE criteria exist, it may or may not be true. I can tell you that we worked on protocols where amendments were required because the protocol itself did not have anybody that met the enrollment and exclusion criteria. So an amendment was required to go back in, but the assumption at the beginning was that this person existed, and then when it went into practice, the information back on the DEQs was it was not site selection. Again, there is a there is an assumption on your recruitment that your site selected have either the internal participants or have the ability to manage the external participants at that moment in time, we can’t stress that enough because past performance is not an indicator of future performance. And if you look at all of the criteria around a site and how they operate, and all the complexities and all of the different factors that sites are dealing with, you can’t take a snapshot and say, “Oh, got it. I got the picture. They did this on Tuesday. They’re going to do it again on Thursday. It doesn’t work that way. So, looking at the assumptions and constantly testing those because they will change. They will change during the course of your study. So it doesn’t mean if a site in a in a site is selected and the results from the site aren’t what were initially expected, that doesn’t make it necessarily wrong. Things change. Circumstances change over time. 

Cristin MacDonald: 

I think it’s a good quick segue to one of the questions that we had, which was asking: Is there data on why sponsors mistrust the feasibility question response? And I think this could help you hypothesize as to why there is some mistrust there. But ultimately, as Seth was saying, this is the impact of continuously taking that static approach to study planning, site identification, and recruitment. Right. So, a study conducted by IQVIA found that 83% of investigators overestimated enrollment rates at the feasibility stage, and when they did overestimate, they overestimated by 50% or more. There were only 5% of investigators that estimated enrollment within 10% of the actual study performance. Right, and if stop talking about it in the context of clinical research, I’m in the middle of a construction project right now. I start off, I get an estimate, and they say, yeah, sure, that’s great. Except then they open the walls, and I have knob and tube wiring, and I have condensation pipes that need to get moved, and all of these things. There were all of these unknowns when they came to start that drastically impacted that assessment. And as the days went on, and we opened more walls, there was more fees added to sort of adjust that. It’s a very similar scenario here, right? They’re overestimating not because they want you necessarily to pick them over someone else. It’s just a very inexact science with an incomplete set of details at the start. So that’s where we’re moving. And the reality is that at any given moment, as we said, both sponsors and sites are evaluating their circumstance with every action or choice that comes their way, just like we do with a risk-benefit analysis. Right? Am I going to change the knob and tube wiring? Yes. I don’t want my house to set on fire. Am I going to move the condensation pipe? Maybe I’ll just keep the wall there and not bother, right? That’s what it goes. It gets even more complicated though for us because we’re showing here this sponsor and site lens of risk assessment. But you have to remember, a lot of times, this is a third party involved, right? There are vendors that are involved with that site identification and selection that are adding their own set of risk assessments in there that aren’t always aligned with the sponsor and/or the site. They may be aligning to their own sort of financial or efficiency objectives, right? But if I look at this from the site perspective, they start to evaluate things when that protocol comes to them and that feasibility questionnaire comes to them. Do we want to participate in this protocol right now? Yes. Okay. And then they start planning on how to execute. They plan their submissions, but ultimately, as Seth was sort of alluding to earlier, if that protocol gets more complex, and they reevaluate their ability to enroll and recruit, and then another protocol comes that suits the needs of their patients and them because it’s more efficient for both the patient and the site. They’re going to take that on, and they’re going to enroll to that trial instead of the other. Right. So again, that’s information that they’re doing, but they don’t necessarily always provide that output to the sponsor unless they’re being asked for it, and it could be things like I said, complications, patient burden. It could be financial. There could be another sponsor that they know pays really well, and they’re going to prioritize that study over another. But they’re looking at all of those things through a risk assessment lens, and it really behooves sponsors to be able to know what that is because every shift in environment that occurs, they’re making new decisions that prioritize their success, their success, and that doesn’t always align, as I said with the sponsors’ definitions of success for their particular study, and that’s a key point. And your contractor analogy is perfect because what is all of our knee-jerk reaction when we go to mechanics and contractors? There’s always going to be something else. And what do they tell you when they start a project? Well, I don’t know what I’m going to find, and we all begrudgingly accept it, but look at it from the other lens. They don’t know what they’re going to find. They’re called in to replace a pipe. They have no idea what the rest of the structure is until they open the wall. They don’t have X-ray vision. They can’t see through the wall. They have to actually physically open up and cut open the space, making it apparently worse in order to make it better. 

Seth Halvorson: 

That’s a perfect analogy for a lot of the time we set this set of assumptions up going in, and we don’t know what we’re going to find. We believe, and we’re using the scientific method on the protocol to both, and we believe that these assumptions are stacked up correctly, and the dependencies are correct. 

We, if we don’t reevaluate that decision or that that let’s even call it an interim decision, as the note indicated from Adam Grant. If we stop thinking about it there and we don’t rethink about it continually all the time, we’re gonna miss the mark. We’re gonna be late, and you saw the stats on the first slide talking about what that is, because too often it’s set it and forget it, and set it and forget it won’t work. These are people, both enrolling into the study, at the study sites working on the studies. These are people. People add infinite complexity to every equation right off the bat. Not to mention the fact that they’re the people that were maybe initially creating the structure. 

They may have had a different set of priorities than the ones who are executing the study. Now we all want the study endpoints to be met, and we all want the study to be successful. But along the line and how we get there, how many procedures are there? How many of those procedures are necessary for that study’s endpoints? How many of those are secondary or tertiary? That all impacts all of those decisions. There could be super good reasons to include that, great reasons. 

But if you don’t go back and assess what the impact of that is, then you’re not going to know how to accurately measure what you’re seeing, and it’s just going to become another statistic on the top of the board. So when looking at this and the continual adjustment and the cycles here, this is across your site selection does across your recruitment and retention. This is across your study conduct overall to make sure that these are constantly aligned. 

No strategy survives contact with the enemy. It’s something we just we preach every time we go to work on with anything. Is you start with the best assumption you can based on the information you have, and you’re often dealing with incomplete or non-final information, so go back and evaluate it again and again and again. And as Chrissy was talking about, if the communication pathway isn’t open for the site to go back to the sponsor and say, “Hey, this is what I’m seeing, this is what I’m experiencing, and the sponsor to take that in and evaluate it and actually hear it. We’re just going to be in this loop. We’re just going to be continually to execute this loop. We need a reset on that. We need to be able to say that it’s okay if bad information early is okay. Get there. That that that can be a good thing, and we need to encourage that as part of it overall. Because again, endpoints successful, efficient, safe. That’s got to be just keep it real simple. 

And on this point, I think it’s time for a reset. I’ll put it out there. I think that you know the bottom image you see. You’ve got your imperial, and you’ve got your kilometers. 

If we’ve got a world today where a survey or a questionnaire or an initial goal is going to be set, and there’s that much disparity between the belief of the information and the confidence in the information, there’s something inherently wrong with the process. If you have that much disconnect between what somebody says they can do and what somebody believes is going to actually happen-it’s not working. So let’s go back into a different approach on how we view this. Let’s make it clear that from the beginning, the goal on everything that we should all be doing should be same: an effective, safe, fast trial, like that, taking care of the participant first, taking care of the study first. We can figure out we’re all going to have different motivations, and we’re all going to have different stressors and different influences at different times. But if we keep that north star on approaching the site selection enrollment by saying that study and that participant are going to be treated first, and we’re going to make sure that it’s run the right way. Everything else gets kind of easy. It does get kind of easy, and bad news early becomes good news because you find a way to put a mitigation or an intervention in place to adjust for whatever assumption was done in the past. And don’t assume that you’re going to put it on the track and it’s going to make its way to the destination without any extra effort. Nothing works that well. Nothing works that way. It takes constant management and constant evaluation. 

Additionally, open clear channels with sites and sponsors and everybody in the process that allows for accurate information to happen in real time or as close to real time as you can. I’m not suggesting everybody has everybody on speed dial, but we do need to get to the point where you can have a very quick, efficient conversation that talks about what is being experienced. As Christy indicated, sometimes the information coming back up from the sites, it just isn’t it isn’t viewed for what it really is, which is vital information talking about the construct of the study and how that’s being impacted and how that’s meeting reality and how that’s meeting whether it’s the study team, whether it’s the participants or whatever it is, there can be lots of different variables there. But allow for that information to come in and allow for that information to get processed and then look at across your study because you’ve got a whole bunch of different sites experiencing your study in either similar or different ways, that’s vital information. That is key information to come back up to make sure that your study structure and your study design is on track. 

Then the last one, and this is just key for us on the recruitment side of the story, because again, if you’ve got such disparity between what people are believing is accurate compared to what’s being reported, and there’s just that level of difference, we need to have a reset where we can say this is what we believe we can contribute, and everybody has to understand and trust that people are doing what they can do and the best they can do to get there, and they’re all acting in the same goal. See the top bullet point. We’ve got to have that. There can’t be villains in the story. It can’t be a gotcha game. It can’t be an aha. It can’t be a finger pointing. It can’t be any of that. It just doesn’t work. We have people that we’re looking at to enroll in the study to figure out if a therapy is going to work, and we’ve got to figure out if we’ve got to make sure that we take care of all of those individuals, and that’s got to be key for all of us. In doing that. There’s no room for any of this. It’s just what can, what is the best output at any individual site? What is the best to deliver that study? How can we pull that all together? How can we focus on getting everything in in time? 

Accept that circumstances are going to change. Accept the fact that the initial assumptions are going to change, except the fact that circumstances, from time to time during the study, will change, even if the assumptions were right, because it’s all going to happen. We’ve all been there. It’s a very fluid industry. It’s a very fluid process. 

Change is inevitable. Disruption is inevitable. So then the question is, what do you do when you get there? And if we’re in a place where we’ve got open communication and we’ve got a north star of study participant first, we can get to the place where we can put implementations in place that are the right implementations at that time without any repercussions, without any negativity, without anything there. Just focus on getting it done. Focus on making it happen. It takes constant vigilance, constant effort. It can’t never be said and forget. 

Cristin MacDonald: 

Yeah, I think that’s a great point. I just want to add too about the no villains in the story, because again, going back to the numbers, my question to you know sponsors who want to hold sites accountable, and sites who want to hold sponsors accountable, is really, to me, that fundamentally comes down to a communication concern, right? And for me, you know, what would the big question that I would ask to a site if they’re struggling to enroll. 

First and foremost, is are you basing it on their estimate that they gave you in feasibility, and what were they working with at the time? And then I’d ask you again when you provided them the final protocol or an amended protocol. Did you then again ask them if that updated their projection of what they thought that they can enroll? And I don’t know that that happens. I can certainly say that we have not been approached by many sponsors to be able to do that here at WCG. 

And whether that’s happening at monitoring visits, that’s possible. But once again, I think that is a tendency to be a lagging indicator. And again, we’re dealing with third parties that maybe have different goals and objectives, where a monitor is more focused in ensuring data is entered, it’s clean, queries are addressed, then they are getting to the deep root cause of recruitment issues, other than saying, “Do you have anybody? Have you screened anybody? Right, and so I think for me, I think those touch points of asking what they think is a reasonable number is not happening enough to really be able to hold anybody accountable for anything at this point in time, because you’re holding them accountable to something that, again, was based on what was potentially a completely false set of assumptions at that point in time. But yeah, back to the no villains of the story. I think that that open line of communication helps build that teamwork. So it’s not a finger-pointing villain scenario. It becomes more of that teamwork to get to what everyone wants, which is a completed trial where everyone exits safely, and the scientific question can be answered with you know patient safety, data integrity, and data interpretability at play. 

Seth Halvorson:  

So for our third polling question, how often do you reevaluate? 

Oh, of course, it just popped up on my screen so I couldn’t get it. How often do you reevaluate your recruitment and retention plan? 

Cristin MacDonald: 

A lot of seldom quarterly. I’m  actually impressed to see how many say monthly. If I’m being honest, Seth, I think that’s it’s a larger percentage than I would have expected. 

I’d love anybody to enter in the Q and A just feedback on kind of what triggers. If it is truly a timeline that’s kind of built into your SOPs of when you’re reevaluating that, or if there are triggers with things like protocol amendments or adding new sites, anything like that. I’d love to understand. So feel free to put that in the chat to provide us some more information. 

Seth Halvorson 

And then our next polling question: When will your next study planning and recruitment strategy begin. So, when will you start looking for your next study? 

Cristin MacDonald: 

This seems like some real just-in-time training we’re providing here. If 65% of you are doing this in the next zero to three months. 

Seth Halvorson 

Okay, so now for the next question. 

No, when will you start? There we go. Good, seeing some consistency. 

Cristin MacDonald: 

Love that. 

Seth Halvorson 

So that’s good. So for those who have pretty decent distribution, for those who are starting soon, you’ve already done the work. Congratulations, kudos. And honestly, for those who’ve got no plans, unless those no plans coincide with nobody, no study starting, appreciate the honesty on that one. Something else too that’s critical. Just you can’t really plan early enough, and you can’t plan. You can never plan completely enough because it’s always going to change. So plan early, plan often, continually change, continually adjust. 

Cristin MacDonald: 

Yeah, maybe to the dismay of marketing here, but I’m going to ask one more question, and feel free to write it in the Q and A there. 

But for those of you who have already started your site identification and recruitment process, I’d love to understand how many of you are doing it with a final approved protocol versus how many of you are doing it with just a study plan. So just pop in there, kind of what you’re doing that site identification and recruitment process with. I’m curious. 

I won’t I won’t wait to see what pops up in the chat, but just curious. Yeah, I see a lot of study plan and synopsis, which I think is a testament, and I’d encourage all of you who are writing that to just kind of consider what we’ve talked about today and and what that means on the results of those feasibility questionnaires and those site ID lists and how that data aligns. 

But with that, I think we’re going to take some time to do some Q and A there. So Seth, I don’t know if you see any that you want to.  

Seth Halvorson 

I’m jumping through them right now. The initial question: Do you include budget and study and cost per patient in the consideration? In the consideration for the impact it has on recruitment and retention? Absolutely. Yeah, it’s something that we take into account on the impact for what sites might answer because I guess the answer is yes there again too it’s a it’s a key factor it’s a key motivator and so if you look at the number of study visits the number of procedures and the cost on the side of the site it does tend to trigger their focus so absolutely it’s critical yeah I.  

Cristin MacDonald: 

I think to your point, though, in my experience, that is so far down the timeline of the site identification and feasibility space. You usually don’t see a ton of questions regarding financial info in the feasibility questionnaires in terms of you know what’s your desired per subject cost or something that you’re looking to ask, because usually the site has been confirmed to participate before they’re seeing a study budget. So it is something that is probably worth consideration in terms of providing some of that info to a site in advance before they choose to participate or not. Again, it’s you might not get an answer you want in terms of if you’re not paying what they’re expecting to see. They might say no, we don’t want to participate. But I would argue that that gets back to that early failure point that we were talking about earlier. You’d rather that now than later. 

Seth Halvorson: 

Absolutely, and it does. If you have sites and you, it’s often the case that are working on competing studies. The study that’s easier to manage with a higher per patient cost is typically the study that gets first focused. Now, ultimately, it’s always a medical decision. I’m not meaning to suggest it’s not otherwise, but if two things are equal, a less scheduled, less scheduled visits and higher budget costs is definitely going to win the day. 

Cristin MacDonald: 

I’m going to ask this one to you, Seth, and I can give a preliminary answer, but I think you have more experience across the board. But in terms of what specialties are currently facing challenges with recruitment and retention, I know for certain and for a fact that oncology is a huge issue, there was a recent article that I read, and I don’t want to quote the details because I don’t want to get them incorrect. But basically said that inherently there are not enough newly diagnosed oncology patients to adequately fill all of the open clinical trial spots that are available for such clinical trials right now, so that obviously is a big issue. But I don’t know what else you’ve seen sort of across the industry there, Seth.  

Seth Halvorson: 

So I would definitely oncology the, and I’m sure most on this call under have this experience. Oncology, the care pathway is going to take somebody into the next decision on trial, and that’s often the case. And so, the identification side is a pretty low focus. It has to be done, and it has to be done for every visit. But as far as overall, what we see in oncology, and we see in some of the others, is that the number of processes and procedures, and just the sheer amount of work that needs to be done-that’s being done for the complexity of the study-actually is a rate limiter on enrollment in some cases because that’s just a-it’s a lot of extra work that can’t be that isn’t always factored in upfront, and that impact goes back to that self-regulation. And so a site might enroll two to three when they could potentially maybe have four or five. I’m not suggesting that they’re not they’re taking people out. It’s just they get to a point when you’ve exceeded your work capacity and you’ve gotten to the point where. So I’d say that I would say that the other aspect is the and I can’t remember if the term is the herding when there are still groups of indications. The herding when there are still groups of indications that are popping up at a time, and the same sites are being asked to look for the same patient population in the same time window. That is still very relevant and definitely has a substantial impact. 

Cristin MacDonald: 

It’s a good one. This one here, I’m going to ask answer about DCT and hybrid trials and what changed in site selection. We actually saw a lot of sites decline participation in some certain hybrid clinical trials. I think a lot of the issues stemmed from sponsors or providers selecting third-party vendors to do things like home health care that ultimately the PI felt responsibility for per GCP guidance and outlines. And as a matter of fact, ICH E6(R3) sort of directly addressed some of those challenges in that it is now saying that the investigator should be approving all vendors that are participating in clinical research that they’ll be using. So I think we’re seeing that the upcoming guidance is addressing some of those site identification challenges that you were asking about in that space, because there were many investigators that were just not willing to take on the risk of healthcare providers that weren’t directly under their supervision or were being trained by them. There’s a good one for you here, I think, Seth, asking about, and I appeared to have lost it. Ah, asking about any sort of strategies that we have in considering or selecting research naive sites or PIs. 

Seth Halvorson: 

So, not to show for our overall company, but we do have an actual product and service specifically on that, so we have a site launcher program that will help bring either research naive or research and experience sites and be able to help them. And really, what the core aspects of that is, which are the core components to answer the question otherwise, is you need to make sure that the PI, the HCP converted to PI understands the difference of what is different between research and care because they’re different, and making sure that they have those at the whether it’s in the form of SOPs, processes, procedures, support in order to be able to differentiate between understanding how to manage the trial aspect of it compared to the care aspect of it because they’re component parts, but they are definitely different, and so when we focus in on that and help with sites that are either less experienced or sites that have no experience, that’s where we’re focusing. We’re going to make sure they’ve got the SOPs, they’ve got the processes, they understand that aspect, understand when and where the when and where, and the procedures, procedural aspect of it is the care and the resource side of it, and making sure that they don’t underestimate the amount of time it takes. That’s another key factor in underestimating, just overall, it might look a lot more appealing. There’s a lot of work that needs to be done in conducting a trial, a lot, and that’s often overlooked. So we will also focus on making sure that support is there to make sure that those individuals have the resources they need-a champion, a study champion, or some other level-so they can get that help if needed. 

Cristin MacDonald: 

I have one more for you, Seth. Here, it’s sort of a combo question, talking about external recruitment: whether sponsors can reach out directly to the subject population, or if the sponsor should be partnering with the sites to recruit. 

Seth Halvorson: 

So, not going to answer the one from the regulatory side of the world. Let the regulatory side talk about best practices. Best practices. The site should be handling the conversations with the individuals. Sponsor can always put an outreach out there and have it funneled some other way. But at the end of the day, the sponsor, the pathway for the individual is best to go from whatever outreach you have or or referral source through to the site or some company working with the site or some there you can be of different interviews other than the sponsor. So best practices, not the sponsor directly, but through the site, and then whatever support is being provided to the site. 

Cristin MacDonald: 

Awesome. There’s one that I’ll take here, asking about how country selections align with site selection. It’s a tricky question because it really you have to understand what the marketing objectives are for that compound to really answer it completely, but they’re obviously very closely linked. So a lot of times in our space, we will look first at selecting the countries that we want to go to by figuring out who has experience in those areas, and then moving to select sites based on you know what countries we select to move forward with based on like I said a variety of reasons startup timelines whatever your commercial objectives are at the end and then a lot of times what comes into play with country selection and site selection in the feasibility space really gets into the space of comparator drugs. So obviously, if you are developing a protocol that’s looking at a particular comparator drug, you have to make sure that you’re going to countries that actually have that drug and is a comparator. It happens in the oncology space a lot, where it’s not standard of care, or they don’t prescribe that drug, or it’s unavailable in that country because it’s not approved. So those two things are obviously very closely aligned, and it’s hard to kind of outline all of the scenarios here. But it’s where you know looking at data and talking with experts, and even conducting country level feasibility is sometimes important to put into play there. 

Seth Halvorson: 

I would add on to that the recruitment strategy is going to be heavily influenced by the country selection, and what you can do in each country and what’s effective in each country varies country to country. So once you start getting into the actual recruitment retention, how you can contact who you can contact, when you can contact what ways work, what ways don’t work. Heavily country impacted. 

Cristin MacDonald: 

Yep, agreed. There’s one more question that I think we’ll do here, but Seth, the good news, it’s not for you and it’s not for me. We have someone from the audience that is actually asking a question for the whole audience about whether anyone in the audience is actually re-evaluating those feasibility results at the SIV. I would argue, if you’re re-evaluating that feasibility at any other time in the SIV, please feel free to put that in the Q&A as well, so that we can see all of that information. But as you’re putting those answers in to help our colleague who put that question in there, I think we can just say our thank yous. Thank you for spending your lunch time with us. We hope it was educational for the 67% of you who are about to embark on your own site ID feasibility and recruitment and retention strategy development. We hope this is beneficial to you, and we will do our best again to send this recording out to everyone. And if there are questions that we have not answered, we will do our best to get back to you on those as well, because we ran out of time today. But thanks for joining us.  

Seth Halvorson: 

Thank you, everybody.