Table 4 Main themes and example data

 Perceived need

Product needed

Team members and stakeholders were able to identify a need for the product being developed.

E.g. one stakeholder discussing that there was a need for a product from cases A and B and comparing the two ‘in terms of population health, the need [for intervention on this topic] is probably greater for [Case B] over [Case A]’ (AB_S2).

Product not needed

Absence of evidence for a perceived need for the product by team members or stakeholders.

‘you’d get quite a lot of discussion about what is this topic about; they often contest the term in the first place’ (D_PI_2^nd).

‘[type of product] – always fun, the infrastructure is very helpful, but I’m not always convinced that the data feedback loops work brilliantly. I’d love to be proved wrong [laughs]’ (B_S4).

 User

Clear

Team members and stakeholders were clear about who the intended user is for the product, with consistent reporting across individuals.

Observation of cases A—primary care clinicians are clearly the intended audience.

Observation of case B—primary care clinicians are clearly the intended audience, with less clarity around the role of prescribing advisors.

Unclear

Team members and stakeholders were either unable to clearly identify the intended end-user or lack of consistency between individuals.

‘One of the challenges we’re dealing with is, who is the audience for this [product]? And I guess the more audiences we have the more challenging it is to ensure it’s relevant’ (D_PI_1^st)

 Aim

Clear

Team members and stakeholders were clear about the product aim, with consistency between individuals.

Case A: consistent description of product aims across multiple interviews and observations, e.g. clearer discharge summaries, better coding of clinical encounters, ensuring that medications are restarted.

Unclear

Team members and stakeholders unclear about the product aim or different aims described by different team members/stakeholders.

‘it’s been difficult to have a clear kind of vision of what we were aiming for’ (B_R).

 Stakeholder engagement

Embedded

Core team members had implicit knowledge of the subject and requirements of end-users, some of whom were end-users themselves.

‘I think one of the major differences and where project [B] maybe falls down a bit is that actually the project [A] working group are the end-users. Whereas the project [B] didn’t really have that much end-user engagement’ (AB_S2).

Tokenistic

Engagement with stakeholders’ erratic and feedback not consistently assimilated into product development.

‘I think that the learning from [Case B] is that there’s been a tendency to agree on something and then for the project to disappear off and create something…and bring back a product, rather than have engagement’ (B_S4).

Co-designed

Product co-designed with end-user groups.

Case C stakeholder engagement and co-design observed (meetings) and documented (final project report) as central to team philosophy, with each site having slightly different versions of each tool.

Consultative

Stakeholders invited to comment on iterations of the product, with feedback incorporated in a step-wise fashion.

Observation of case D stakeholder engagement, written summaries of stakeholder input and audit trail of incorporated elements.

 Motivates product development

Perception that product proposal increases the likelihood of grant award

Products not being developed because of a specific need for them, but rather because of a perceived need for products to be developed as part of normal expectations from applied health services research.

E.g. case D: ‘They’re not going to be interested just in academic outputs, they’re interested in [products] as well… the main thing when you’re writing a research grant is you’re thinking the whole time, what’s the most likely to satisfy the funder?’ (D_PI_1^st).

E.g. case C: four of the eight projects funded under the same call, some of which ‘overlap’ (C_PI_1^st), planned to produce toolkits arising from their research, although toolkits were not referenced explicitly in the funding brief.

Product proposal made with aim to mobilise knowledge

 Products being proposed or developed with the express aim of mobilising knowledge

E.g. case A: whole project [product development] based around mobilising knowledge regarding syndrome A

E.g. case C: ‘we wanted to produce a product, or a set of products, that would be useful in these service settings’ (C_PI_1^st).