Table 4 Quotes from semi-structured interviews

The importance of having the algorithm use endorsed by practice leadership, and of having a local “champion” who promoted their use.

Innovation, outer setting, inner setting, individuals

“If it comes from administration, from the top, you think it’s more validated. ... if it just comes from a pain practice, it’s less likely, because we don’t know if it just comes from their opinion ... if it’s been implemented at a specific practice already and has been useful and believed to be efficient, and is easily implementable, ... if somebody who does this and tells you, yes, this easily implemented, even despite all the things we have to do, this does help me make things faster, um, then it’s more likely for other people to look at it.” (Participant 11)

“If you got the head of all the private practice providers to say this is our-what we’re gonna do, ... at least everyone would have the same set of tools, ... instead of sort of saying here’s a bunch of things, you know, do in your office how you will. ... while that gives flexibility, it also allows for a lot of variability. ... if there’s way too much variability, then, you know – like that last standardized patient said to me, I’ll just switch doctors. ... because they don’t like the answer that they’re given. So, if you knew that it was gonna be consistent, then maybe people would be a little less apt to do that.” (Participant 2)

Integration of the algorithm workflow into practice EHRs.

Inner setting

“If it becomes a smart set or an EPIC list or something like that where you have a check-off, if it’s a missed appointment issue and it goes into that segment of the algorithm versus the other pieces, of other substance use or taking opioids for symptoms other than pain and those kinds of things. I think having kind of a check-off box that leads you to the next [step] – and then it opens kind of the next step piece would be helpful.” (Participant 4)

“I think converting these to templates in EPIC that we can instantly import to them into the chart. Because, no matter how good we are, these papers are going to get lost in a huge, massive pile of papers in the office. So, if I can ... just enter dot I G opiates, early appointment, early prescription, and gives me the questionnaire and-and I go over the questions, we’re good. ... Because our office hours are extremely, extremely tight. Every day there are new things that we need to add doing to our patients. While you are talking to them, talk about colon cancer screening. Um, you can easily spend half an hour convincing someone to take the COVID vaccine and… So, just-just so add-ons. So, if this is going to be a part of usually many other things in that visit, we need to be very time-efficient.” (Participant 5)

Practice and location-specific inputs into the algorithms, such that a suggestion to refer to a specialist come with a list of who, specifically, to refer to, or a suggestion to call security provide the practice-specific number for security.

Inner setting, implementation process

“It would need to have some specific numbers. So, things like call security. ... So, it’s not like I have to go look something up while I’m dealing with someone with an aggressive behavior, right? [and] when it says refer to specialist ... an addiction specialist number, chronic pain number, those listed out so, again, we’re not having to dig for them.” (Participant 3)

“[We need direct referrals or embedded practitioners because] if you’re going through the algorithm and it says, you know, address underlying problems, and their underlying problem is uncontrolled anxiety, that’s what they need is therapy, and you don’t have it. ... And telling people to call the back of their card to figure out who takes their insurance is, like, archaic at this point.” (Participant 1)

Access to specialists who could help interpret unclear or difficult-to read drug screens, or suggest a particular course of action with a tricky patient.

Outer setting, implementation process

“It would be helpful to have better pain management resources, especially in terms of medication management. Somebody that I can reach out to and say, ‘hey, listen, can you eyeball the chart. ...what do you think?’ Like, [an] opioid management e-consult. They have that paradigm already for some other health conditions.” (Participant 12)

“I think another potential place could be if the consultants that were calling for help saw a pattern of, like, they keep asking the same question that I keep answering, that would be a time to be like, maybe we need to re-educate this piece, because I keep getting the same call from different people. So, those-those folks might help steer directions of, like, where our-where there’s an education lapse.” (Participant 3)