September 7, 2023 • 27 mins
Dr. Justin Knox describes a 'practical' approach to integrate observation and implementation science studies to increase the public health impact of observational research. “Proposing the observational-implementation hybrid approach: designing observational research for rapid translation” can be found in Annals of Epidemiology’s Special Issue on Implementation Science in Epidemiology.
Read the full article here:
https://www.sciencedirect.com/science/article/pii/S1047279723000571
Call for papers on Implementation Science in Epidemiology:
https://www.sciencedirect.com/journal/annals-of-epidemiology/about/call-for-papers
Episode Credits:
- Executive Producer: Sabrina Debas
- Technical Producer: Paula Burrows
- Annals of Epidemiology is published by Elsevier.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Patrick Sullivan (00:10):
Hi, you're listening to EPITalk
Paper, a monthly podcast fromthe Annals of Epidemiology.
I'm Patrick Sullivan,Editor-in-Chief of the journal,
and in this series we take youbehind the scenes of some of the
latest publications featured inour journal.
Today, we're talking with Dr.
(00:35):
Justin Knox about his article"Proposing the observational-
implementation hybrid approach: designing observational research (00:38):
undefined
for rapid translation.
You can find the full articlein the journal's Special Issue
on Implementation Science at www.
annalsofepidemiology.
org.
Dr.
Justin Knox is an AssistantProfessor of Clinical
Implementation Science andIntervention at Columbia
(00:59):
University.
His research centers on HIV andsubstance use, with a focus on
vulnerable and marginalizedpopulations, including racial,
ethnic and sexual genderminorities in both domestic and
global settings.
Dr.
Knox, thank you for being heretoday.
Justin Knox (01:15):
Yeah, thanks so much for having me.
I'm excited to be here.
Patrick Sullivan (01:18):
So I'm excited about your paper because it
really brings together this ideaof hybrid approach
implementation studies, whichhas been an area that's of a lot
of interest in implementationscience world, with the
observational study design whichis sort of a staple of
epidemiology.
So you're really bringingtogether these two worlds.
It seems like a lot of theimplementation scientists are
(01:42):
maybe epidemiologists, but I'mnot sure that a lot of
epidemiologists are so familiarwith implementation science.
So I wonder if you could startout just by telling us a little
bit about what implementationscience is about, and then we
can talk about the hybrid.
Justin Knox (01:56):
Yeah, absolutely.
So that journey, I think, wasmy motivation for writing the
paper.
It's like I'm trained as anepidemiologist and how come I
haven't heard about thisimplementation science stuff,
which?
Implementation science is thestudy of methods and strategies
that facilitate the uptake ofevidence-based strategies to
improve public health orclinical practice and
(02:18):
incorporate them into everydaypractice.
So it's all about getting thethings out there that work,
getting people to use them andoffer them and deliver them to
patients and people populations.
So, yeah, I was thinking, whyisn't it an epidemiologist?
Had I not heard of this field?
And one thing I came across wasthese hybrid approaches in the
(02:38):
context of conducting clinicaltrials, but thinking that, as
you noted, many epidemiologistsmostly conduct observational
research.
So if we had our own hybridapproach in that context, could
that make people more aware ofthe potential of incorporating
implementation science intotheir work?
Patrick Sullivan (02:56):
So, as I understand this, I think there's
a piece of beginning with theend in mind, meaning that at the
same time you're collectingdata in an observational study,
you could also be trying to pickup some preliminary information
about if this association pansout and if we take the next step
of deciding that it's causal,then could we be collecting some
(03:20):
preliminary information abouthow you would bring a public
health strategy to the world,even as we're asking the
question about this association.
Is that right?
Justin Knox (03:29):
Exactly, yeah, I think, as we're trying to find
out how prevalent are theseissues?
How, what are the incidencerates?
Are there causal associations?
Could we make our research moreactionable by, at that same
time, starting to collectinformation about what we could
do to address these issues,which in many cases there
already exists evidence-basedinterventions and treatments.
Patrick Sullivan (03:52):
Yeah, that's a great point.
So we're like you have maybe ahealth condition and you're
thinking about evaluating someassociations and an
observational kind of design.
But you may know at the timeyou're designing that
observational study that thereare treatments or interventions
available.
So can you just give us acouple examples of what kind of
information you might collectduring the observational study
(04:14):
and how that would relate to animplementation strategy or how
we improve health part afterit's done and again, it's
conditional on whether theassociation is true.
Can you just give us an exampleof what some of those nuggets
of information might be in thecontext of an observational
study?
Justin Knox (04:33):
Yeah, absolutely so .
In my bio I mentioned I do alot of work on HIV and substance
use.
We're conducting a study tolook at whether or not alcohol
use is associated with HIVrelated outcomes.
So we know there are a numberof evidence-based interventions
that can reduce people'sdrinking behaviors and some
implementation data that wecould collect around.
(04:55):
That would be what would beparticipants' preferences
regarding how they would wantthose interventions delivered,
what they want them incorporatedinto their primary care
settings or would they prefer togo to specialists for those
services?
Do they want them offered bypeers or by trained
professionals?
So preference data is one.
(05:16):
We also know thattransportability is a major
issue, right?
Certain things work better incertain populations based on
their underlying characteristics, right?
It might be more effectiveamong older people or people who
are willing to engage intreatment, you know, who have
identified that they want toreduce their drinking.
So those are all things that wecould collect information about
, while we're also studying howprevalent these issues are and
(05:39):
whether or not they're costlyassociated with HIV related
outcomes.
Patrick Sullivan (05:44):
So it seems like this is really just sort of
an opportunity to think at thebeginning of an observational
trial and maybe, with noadditional cost, get some more
leverage out of the results.
But, like in this situation, ifI were doing the study that you
described, I know some of theobservational methods and
enrolling participants andmaintaining them in a cohort,
but I'm not an expert at allabout interventions for alcohol
(06:08):
use.
So what, like how would thatplay out?
Pre-study, or what would yousuggest people do if they have
these ideas?
Justin Knox (06:16):
Yeah, absolutely.
That's a great point andsomething I thought about too as
I got into this field ofimplementation sciences.
Yeah, we don't reallynecessarily know what the
interventions, treatments are.
I mean, a lot of times as youwork in an area or field, you
come to be more familiar withthose, but in certain cases
people might not be, and maybewe should think about expanding
our study teams to be able toinclude that type of expertise.
(06:39):
You know there have beenseparate calls for increased
levels of funding for thesestudies.
You know, as we're expected todo more and more, you know we
need more resources in order tobring these comprehensive teams
together because ultimately, Ithink it can make things more
efficient.
As you noted.
You know we collect informationsooner in the pipeline as to
you know, in terms of when itcould ultimately be used to
(07:03):
inform what we can do to addressthe situation.
If you're investing all thoseresources to conduct a big
observational study and you havethose skills on your team to
recruit and retain participants,you know perhaps we should
expand those to also be thinkingabout what are we going to do
with this information andincluding expertise in
intervention and implementationscience?
Patrick Sullivan (07:25):
I think this would be good.
We'll have to like send aspecial invitation to this
version of the podcast to ourNIH friends and colleagues just
to say, like seems like it's anopportunity maybe for
supplements to people who may beasking these kinds of like
associative questions.
To say, like there's a marginal, like you could apply for a
supplement to get some marginalfunds to pay for some time a
(07:47):
colleague and I think one of thestrategies is trying to find
somebody who has expertise inthe intervention side and just
say, like we're going to betalking to a thousand
adolescents living with HIV orat risk for HIV, or like
whatever the underlying cohortis.
What information would help youdo a better job of
interventions if thisassociation turns out to be true
(08:09):
.
So it may be relatively lowcost but it might save NIH
funding.
One study to describe theassociation.
And then maybe that colleaguethat we don't know yet comes,
reads your paper and says likeoh well, we should see if this
kind of like what thepreferences are on this.
The other thing that like toyour point earlier is that some
(08:30):
of these things are notnecessarily you don't have to be
an implementation scientist toadd some questions about
preferences for format orpotential, like theoretical
acceptability of different kindsof interventions.
So they may fit quite well intothe kinds of surveys that we're
already doing or the kind ofquestion formats that we're
comfortable using.
Justin Knox (08:49):
Yeah, definitely.
I think this could be anopportunity to build bridges
between people doingobservational research that
aren't familiar withimplementation science and those
more integrated into thosespaces.
But also, like you mentioned,it's not doesn't necessarily
have to be overly complicatedyou could start to think ahead
about.
I think it's something thatwould align with what
(09:10):
epidemiologists already want todo, which is conduct actionable
research that translates topublic health impact.
So totally agree.
And then the supplement modelis actually how we've had our
hybrid work funded thus fargetting supplements to
incorporate discrete choiceexperiments, preference
(09:30):
elicitation methods.
But ideally you could propose astudy that is an observational
implementation hybrid approachand get all those resources
together in one application.
I think it would be a littlemore seamless.
Patrick Sullivan (09:45):
Yeah, and having the observational
implementation hybrid approachnamed and published and
reference of all, I think, makesit more likely that you could
pitch this to a funder and say,like this is a real thing,
because it's already a realthing.
Justin Knox (09:57):
Yeah, that was totally.
Our motivation too is, I thinkpeople are doing this work and
we tried to acknowledge that inthe paper.
But sometimes having somediscourse around this, having a
discussion about its merits orits limitations, and having a
shared understanding of whatsomething means, I think can
hopefully help it bedisseminated and utilized.
So that's definitely ourmotivation and I think we're
(10:19):
hoping to follow up on thispaper with some further guidance
on how you might actually carryout, in a hybrid observational
implementation study, someexamples of where we think it
could be particularly useful,particularly when conducting
research among populationsexperiencing longstanding health
inequities, where there mightbe more urgency than usual to
(10:41):
really conduct research in a waythat it's actionable.
Patrick Sullivan (10:45):
Yeah, I'm glad you raised this piece about
health equity because this isone of the things I think we
need to be asking ourselvesabout.
Everything we do is like wouldthis kind of approach advance
health equity?
And so how do we optimize that?
So I see two things.
One is that it may be that youwould identify different, for
example, preferences foreventual provision of a service
(11:09):
in different groups, by race,ethnicity or by age groups, for
example, that may bedisproportionately impacted, and
so then you could prioritizeand say there's actually a
stronger preference for cellphone based methods in the
groups that are actuallyinequitably impacted by this
health condition, and so we'regonna choose that as a first
(11:30):
approach and if it's a littleless preferred by the people who
are less impacted from anequity point of view, like we
can take that.
So that's one thing is we canactually look within subgroups
to choose the options that willhave the best shot at reducing
inequities.
But the other is this idea ofshortening the timeline, because
health equity and healthjustice, they say justice
(11:52):
delayed is justice denied.
I mean there is a particularurgency in groups that have gaps
in health equity to shortenthis timeline and not have the
books lay sort of end to end,but have them overlap a little
bit so that we build that stackfaster.
Justin Knox (12:10):
Yeah, especially, you know, in these areas,
communities that have, you know,had less investment.
It's important, I think, thatwe're as efficient as possible,
as we've acknowledged this, andstart to invest more.
And certainly, I thinkprioritizing those contexts and
doing, you know, conductingresearch in a way that it can
(12:31):
immediately inform what to do,is a way to hopefully, as you
said, move towards justice.
Patrick Sullivan (12:38):
All right, now we're going to turn to the part
of the podcast we call Behindthe Paper, and we'd like to have
on the podcast sort of abalance between understanding
your science but also talkingabout your process.
And one of the things that Iwant to start with for this
manuscript is that you reallysort of developed and socialized
this idea and then you broughtin a number of other people who
(13:02):
work in implementation scienceto participate as authors.
So talk a little bit about justlike what the value is.
You're an implementationscientist yourself, but you
developed this idea and then youbrought in some other folks,
and so how do they contributedifferently?
Maybe there may be some authorsthat you sit and brainstorm the
(13:23):
concept with, but there may beanother group who meet the
authorship criteria, meaninglike they actually contribute to
the development of the ideas.
They read and approve themanuscript like they're real
authors, but they come in at adifferent stage and add
something different to the finalproduct.
So can you say a little bitabout how you use that those
colleagues strategically inwriting this paper?
Justin Knox (13:45):
Yeah, definitely, it was a great paper in that
regard.
I guess I'll just provide alittle context.
This is the first purely sortof theoretical paper that I've
written.
I'm used to writing empiricalpapers that ultimately can end
up being rather formulaic.
You have a gap, you set up astudy to address that gap.
You analyze data, you interpretthe results, whereas in this
(14:08):
case it was sort of.
I started learning more aboutimplementation science, coming
at it from the epidemiology side, talking with mentors.
What is implementation science?
How could it improve?
Why is it important to our work?
How can we make contributionsthere?
And in thinking through that andcoming with this idea, I had a
(14:28):
great senior author, Calvin Gang, who really helped me formulate
this idea and talk through it,and I was able to start writing
things up and getting hisfeedback.
And then, yeah, it's such agreat opportunity when you come
up with an idea and take thatinitiative to draft something.
Then you can gauge othermentors and say how does this
sound to you?
You have your own take on thisfield and how this could align
(14:52):
and how we could use thisapproach.
What might its limitations be,where gaps that you feel need to
be addressed and discussed, andI really felt so supported and
positive in the way the papercame together and the way that
you read drafts and helped meedit it down and respond to
reviewer comments.
So it's really a great learningexperience and a great
(15:16):
mentoring experience for me asan early stage investigator, to
bring all of my mentors andcolleagues and collaborators
together in an idea that I hopepeople think is a good idea and
they decide to start using it,and it's a way to make research
more impactful and useful.
Patrick Sullivan (15:33):
Yeah, I want to pick up on this idea of
mentorship because it's presentat every stage of your career.
It's present when you'reearlier in your career because
you need it, and it's presentwhen you're in your mid-career
because you start havingdoctoral students who are
looking for mentorship.
So mentorship is an issue thatcuts across all of our careers
and so I wonder for you and I'lljust say, as you mentioned, we
(15:56):
know each other and I've workedtogether outside of the context
of this paper, so I think you'resomeone who's been pretty
intentional about reaching outand asking for mentorship from
folks at different points inyour career.
So can you just say, maybe alittle bit of advice for people
who are here in every way acolleague I wouldn't call you an
earlier career, I just call youa colleague.
But putting yourself back inthat time, what advice would you
(16:18):
have for earlier careercolleagues about seeking
mentorship, like, how do youstart that conversation?
How do you ask?
How do you know when it's overfrom your perspective?
Justin Knox (16:29):
Yeah, yeah, totally .
I mean it's a great question.
It's something I feel.
I've been so fortunate to havegreat mentors throughout my
training and you get sort ofused to it in a way that you're
learning and I think a lot oftimes you're working with
mentors who've received greatmentorship themselves and have
always been incredibly generousand willing to pay it forward.
(16:51):
You know and you realize howyou work together and there can
be growing pains at times.
But I mean for me, as you, asyou noted, it's always, you know
, being willing and wanting totake on that initiative of
saying like I want to sound thisidea off you, I want to get
your input on this.
How can I cut this down?
How can I say this moreefficiently?
What do you think about this?
(17:12):
How can we refine it?
So yeah, and I think the wayacademia is set up is we're
constantly being put intosituations where we can be
mentored, whether you're on atraining program or you're on an
independent, you know, mentor,career development award.
But you know, I feel like you'realways being forced to think of
yourself in the context of ateam, whether it's a team of
(17:33):
mentors or a team ofcollaborators, to get a project
done.
So, you know, as I startedthinking about the idea this for
this paper I had my Kmentorship team in place.
I had people who wereepidemiologists and who were
engaging in implementationscience who I would naturally be
, you know, attending their labmeetings, talking with them
about my ideas, working onpapers together.
(17:56):
So, you know, it came quitenaturally to me, but I imagine,
if you weren't in that context,maybe you didn't.
You know, you're an institutionwhere there's less resources.
In that sense, my experience hasalways been you know, don't be
afraid to.
You know, reach out to anyone.
Like you're at Emory, I'm atColumbia, you know, there's
people on that paper from allover the country, right, and I
(18:18):
feel like, in a way, it's reallylike there are no borders in
academia.
You know, people are alwayshappy to talk about good ideas
and collaborate across Settingsand projects and topics.
Even so, you just have toengage them and I find you'll be
rewarded.
Patrick Sullivan (18:34):
So I'm going to just take a small moment of
personal privilege, because youtalked about, like, how we've
all benefited from mentorshipand just in the last month or so
my major professor from my PhD,Dr.
Ted McDonald, passed away andhe's just a great loss to those
of us who cared about him.
He passed away after a very longcareer and even 20 years after
(18:56):
I graduated he would email meand say I read with interest
your paper on and I did a benchscience PhD.
I went on to do HIV prevention.
You know research andepidemiology, so it's not at all
his field, but he still readthe work of his mentees and
reached out to us and he alwayshad a like, a good question,
(19:17):
like, even though it wasn't hisarea.
And just recently he passedaway and I had a chance to visit
the folks in my lab who hadsupported me during my training.
So I'll just say like I thinkthese mentorship relationships,
when they're done right, affectus our whole careers and I
remember Ted was such gratitudeand along those lines I wonder
(19:40):
you know you have been veryintentional and you sort of talk
about this how do you identifysomeone you think you might want
to be a mentor and how do youapproach them?
I just think it's reallypractical for earlier career
people to know, like is it weirdto just reach out to somebody?
Or, you know, do you look for acommon connection before you do
it?
Like, how have you assembledyour mentor team, how do you
(20:02):
find the people you want tomentor you and how do you engage
with them?
Justin Knox (20:05):
Yeah, I mean, I think everyone has their own
style right.
Some people are completelycomfortable reaching out, some
people like to havecollaborations going on all over
the place and be a little bitinvolved in a lot, whereas some
people really like to dig in andwork very closely with one
person or one team, you know,and people can be productive
across settings in a differentways, and in no way do I want
(20:29):
you know my experience to belike.
This is the way you do it, butfor myself, of course, I can
speak to that and I feel likethe most successful way has been
to work those networks in a way, you know, like, ultimately
we're not a small community ofpeople doing, you know, pretty
intensive research in certainareas and in this case, like,
(20:50):
people bring expertise fromdifferent areas, but a lot of
people know each other, they'veworked together, they can make a
reference for you, they canmake a connection, you can meet
up when you're at a conferenceor you can, you know, jump on a
zoom and shout about an ideathat you have or a paper
proposed to work with their data, and I think for me that's been
the most successful.
(21:11):
I mean, there are some peoplethat have, you know, just
reached out to cold.
I really enjoy your research.
I'd love to work with you on aproject, but for the most part
it's really been.
You know, I think Teo Samfordintroduced us.
Patrick Sullivan (21:23):
Yeah, and when you reach out to people call,
do they respond?
Justin Knox (21:29):
Not all the time.
Patrick Sullivan (21:30):
Yeah, I'm sure .
Justin Knox (21:30):
Like I only remember the people.
Yeah, I'd be like oh yeah, Inever wrote me back and it's
fine, right, like, you move on,they move on and maybe you
reconnect, you know.
So, yeah, you know everyone'ssuper busy, everyone's doing,
you know, things that I thinkexcite them and motivate them
(21:52):
and sometimes there'll be aconnection and other times there
won't.
But I think if you just keeppushing forward with your ideas
and your agenda and what youthink is important and what
you're passionate about, youknow, I do think it ultimately
works out.
Patrick Sullivan (22:04):
Yeah, I want to close just by asking you to
think about.
Maybe you know, like people whoare their PhDs in an
epidemiology program, you knowpeople who are earlier in their
academic careers or CDC orgovernmental careers or whatever
, who are epidemiologists.
But and for implementation,science is like a little bit of
a black box, seems complicated.
What are just a couple steps?
(22:25):
You might suggest that, like ifsomeone says I want to, I just
want to understand more aboutthis.
To think about, do I want tolike develop expertise?
Do I need to know just to beable to talk about it?
So one thing they should readyour paper of course, but beyond
that.
Are there any like trainingprograms or webinars or
textbooks or anything that you?
How would you suggest someonelike dip their toe in and see if
(22:47):
there's a relevant piece forthem?
Justin Knox (22:49):
Yeah, I mean, that's a great question.
Again, I think it probablydepends largely on, like, what
type of learner you are and howyou prefer to engage.
I'm a person who's not afraid tosay something stupid and like
try something new, but meaningthat I, you know, I won't know
as much in that context.
So for me, what was mostengaging and useful was to sign
(23:14):
in and apply for these trainingprograms and implementation
science.
You know, where you getinvolved in a group of people
who are learning this, thinkingabout it, they connect you with
experts and you really have a,you know, a chance to engage in
discussion and react to thingsand talk through things, and
that's the way that I learnedbest.
You know, other people mightprefer to sit down with a couple
(23:36):
articles or an introductorytextbook and really immerse
themselves in the literature andthen maybe be a little more,
you know, refined in theirthinking before they reach out,
and that's probably totally finetoo.
But for me it's really aboutthe collegiality, about the
discussions, about, you know,working in teams and learning
together and, you know, becomingpart of a cohort and
(23:58):
collaborating with mentors and,yeah, so for me it's much more
social.
But I think any way that worksfor you can get the job done.
Patrick Sullivan (24:07):
One thing I'll add to that is, I think, by its
nature, unless you're already aprovider like if you're already
a medical care provider, he'sworking in an opioid
substitution clinic or you know,like you already have the
clinic the knowledge about theimplementation environments.
Implementation science by itsnature requires some networking
of folks who aren't our usualsuspects but who have knowledge
(24:29):
about the implementationsettings.
Because if you want to askthese questions about, I mean,
you really almost need to go tothe sleep apnea clinic or the HV
prevention clinic or the TV,like whatever the thing is, and
say, like if we brought thiskind of intervention, like what
would determine whether youcould scale it up or not, like
how would it look in yoursetting?
(24:50):
And so there is this piece ofconnection with how and where
services are delivered that wethat I usually know nothing
about, and so that's also thislike part of networking.
So the methods get you to acertain distance, but by nature
I think if you dive into this,you have to be willing to maybe
cold call or maybe network alittle bit and figure out how do
(25:13):
we understand more about thesettings in which this would be
implemented, because there's apiece of humility here as
implementation sciencescientists that we can
understand, like the frameworksand collect the data, but in
most cases we're not theproviders or the service you
know providers, and so that's animportant perspective as well.
(25:33):
And thanks for mentioning thatthere's some training programs,
some like online websites, thatreally describe some of these
frameworks and, I think, talkabout them in relatable ways,
and we will put the links to thetraining programs and some of
those major websites that youcould check out in the show
notes.
So if you want to learn alittle bit more, an easy way is
(25:54):
just to visit one of you knowre-aim or like one of the you
know websites for a frameworkwhere they'll really walk
through what the elements ofimplementation science
frameworks are and give someexamples, and that would be
another great way to getoriented.
That brings us to the end ofthis episode.
Thanks again, Dr.
Justin Knox, for joining us totalk about your article on the
observational- implementationhybrid approach and especially
(26:17):
for sharing a little bit moreabout your personal and
professional background and howthis paper came about.
So pleasure to have you on theshow.
Justin Knox (26:24):
Yeah, thanks so much.
I really enjoyed the experience.
Patrick Sullivan (26:32):
I'm your host, Patrick Sullivan.
Thanks for tuning in to thisepisode and see you next time on
EPI Talk (26:37):
Behind the Paper.
EPI Talk is brought to you byAnnals of Epidemiology, the
official journal of the AmericanCollege of Epidemiology.
For a transcript of thispodcast or to read the article
featured on this episode andmore from the journal, you can
visit us online at www.
annals ofepidemiology.
org.
Hi, you're listening to EPITalk
Paper, a monthly podcast fromthe Annals of Epidemiology.
I'm Patrick Sullivan,Editor-in-Chief of the journal,
and in this series we take youbehind the scenes of some of the
latest publications featured inour journal.
Today, we're talking with Dr.
(00:35):
Justin Knox about his article"Proposing the observational-
implementation hybrid approach: designing observational research (00:38):
undefined
for rapid translation.
You can find the full articlein the journal's Special Issue
on Implementation Science at www.
annalsofepidemiology.
org.
Dr.
Justin Knox is an AssistantProfessor of Clinical
Implementation Science andIntervention at Columbia
(00:59):
University.
His research centers on HIV andsubstance use, with a focus on
vulnerable and marginalizedpopulations, including racial,
ethnic and sexual genderminorities in both domestic and
global settings.
Dr.
Knox, thank you for being heretoday.
Justin Knox (01:15):
Yeah, thanks so much for having me.
I'm excited to be here.
Patrick Sullivan (01:18):
So I'm excited about your paper because it
really brings together this ideaof hybrid approach
implementation studies, whichhas been an area that's of a lot
of interest in implementationscience world, with the
observational study design whichis sort of a staple of
epidemiology.
So you're really bringingtogether these two worlds.
It seems like a lot of theimplementation scientists are
(01:42):
maybe epidemiologists, but I'mnot sure that a lot of
epidemiologists are so familiarwith implementation science.
So I wonder if you could startout just by telling us a little
bit about what implementationscience is about, and then we
can talk about the hybrid.
Justin Knox (01:56):
Yeah, absolutely.
So that journey, I think, wasmy motivation for writing the
paper.
It's like I'm trained as anepidemiologist and how come I
haven't heard about thisimplementation science stuff,
which?
Implementation science is thestudy of methods and strategies
that facilitate the uptake ofevidence-based strategies to
improve public health orclinical practice and
(02:18):
incorporate them into everydaypractice.
So it's all about getting thethings out there that work,
getting people to use them andoffer them and deliver them to
patients and people populations.
So, yeah, I was thinking, whyisn't it an epidemiologist?
Had I not heard of this field?
And one thing I came across wasthese hybrid approaches in the
(02:38):
context of conducting clinicaltrials, but thinking that, as
you noted, many epidemiologistsmostly conduct observational
research.
So if we had our own hybridapproach in that context, could
that make people more aware ofthe potential of incorporating
implementation science intotheir work?
Patrick Sullivan (02:56):
So, as I understand this, I think there's
a piece of beginning with theend in mind, meaning that at the
same time you're collectingdata in an observational study,
you could also be trying to pickup some preliminary information
about if this association pansout and if we take the next step
of deciding that it's causal,then could we be collecting some
(03:20):
preliminary information abouthow you would bring a public
health strategy to the world,even as we're asking the
question about this association.
Is that right?
Justin Knox (03:29):
Exactly, yeah, I think, as we're trying to find
out how prevalent are theseissues?
How, what are the incidencerates?
Are there causal associations?
Could we make our research moreactionable by, at that same
time, starting to collectinformation about what we could
do to address these issues,which in many cases there
already exists evidence-basedinterventions and treatments.
Patrick Sullivan (03:52):
Yeah, that's a great point.
So we're like you have maybe ahealth condition and you're
thinking about evaluating someassociations and an
observational kind of design.
But you may know at the timeyou're designing that
observational study that thereare treatments or interventions
available.
So can you just give us acouple examples of what kind of
information you might collectduring the observational study
(04:14):
and how that would relate to animplementation strategy or how
we improve health part afterit's done and again, it's
conditional on whether theassociation is true.
Can you just give us an exampleof what some of those nuggets
of information might be in thecontext of an observational
study?
Justin Knox (04:33):
Yeah, absolutely so .
In my bio I mentioned I do alot of work on HIV and substance
use.
We're conducting a study tolook at whether or not alcohol
use is associated with HIVrelated outcomes.
So we know there are a numberof evidence-based interventions
that can reduce people'sdrinking behaviors and some
implementation data that wecould collect around.
(04:55):
That would be what would beparticipants' preferences
regarding how they would wantthose interventions delivered,
what they want them incorporatedinto their primary care
settings or would they prefer togo to specialists for those
services?
Do they want them offered bypeers or by trained
professionals?
So preference data is one.
(05:16):
We also know thattransportability is a major
issue, right?
Certain things work better incertain populations based on
their underlying characteristics, right?
It might be more effectiveamong older people or people who
are willing to engage intreatment, you know, who have
identified that they want toreduce their drinking.
So those are all things that wecould collect information about
, while we're also studying howprevalent these issues are and
(05:39):
whether or not they're costlyassociated with HIV related
outcomes.
Patrick Sullivan (05:44):
So it seems like this is really just sort of
an opportunity to think at thebeginning of an observational
trial and maybe, with noadditional cost, get some more
leverage out of the results.
But, like in this situation, ifI were doing the study that you
described, I know some of theobservational methods and
enrolling participants andmaintaining them in a cohort,
but I'm not an expert at allabout interventions for alcohol
(06:08):
use.
So what, like how would thatplay out?
Pre-study, or what would yousuggest people do if they have
these ideas?
Justin Knox (06:16):
Yeah, absolutely.
That's a great point andsomething I thought about too as
I got into this field ofimplementation sciences.
Yeah, we don't reallynecessarily know what the
interventions, treatments are.
I mean, a lot of times as youwork in an area or field, you
come to be more familiar withthose, but in certain cases
people might not be, and maybewe should think about expanding
our study teams to be able toinclude that type of expertise.
(06:39):
You know there have beenseparate calls for increased
levels of funding for thesestudies.
You know, as we're expected todo more and more, you know we
need more resources in order tobring these comprehensive teams
together because ultimately, Ithink it can make things more
efficient.
As you noted.
You know we collect informationsooner in the pipeline as to
you know, in terms of when itcould ultimately be used to
(07:03):
inform what we can do to addressthe situation.
If you're investing all thoseresources to conduct a big
observational study and you havethose skills on your team to
recruit and retain participants,you know perhaps we should
expand those to also be thinkingabout what are we going to do
with this information andincluding expertise in
intervention and implementationscience?
Patrick Sullivan (07:25):
I think this would be good.
We'll have to like send aspecial invitation to this
version of the podcast to ourNIH friends and colleagues just
to say, like seems like it's anopportunity maybe for
supplements to people who may beasking these kinds of like
associative questions.
To say, like there's a marginal, like you could apply for a
supplement to get some marginalfunds to pay for some time a
(07:47):
colleague and I think one of thestrategies is trying to find
somebody who has expertise inthe intervention side and just
say, like we're going to betalking to a thousand
adolescents living with HIV orat risk for HIV, or like
whatever the underlying cohortis.
What information would help youdo a better job of
interventions if thisassociation turns out to be true
(08:09):
.
So it may be relatively lowcost but it might save NIH
funding.
One study to describe theassociation.
And then maybe that colleaguethat we don't know yet comes,
reads your paper and says likeoh well, we should see if this
kind of like what thepreferences are on this.
The other thing that like toyour point earlier is that some
(08:30):
of these things are notnecessarily you don't have to be
an implementation scientist toadd some questions about
preferences for format orpotential, like theoretical
acceptability of different kindsof interventions.
So they may fit quite well intothe kinds of surveys that we're
already doing or the kind ofquestion formats that we're
comfortable using.
Justin Knox (08:49):
Yeah, definitely.
I think this could be anopportunity to build bridges
between people doingobservational research that
aren't familiar withimplementation science and those
more integrated into thosespaces.
But also, like you mentioned,it's not doesn't necessarily
have to be overly complicatedyou could start to think ahead
about.
I think it's something thatwould align with what
(09:10):
epidemiologists already want todo, which is conduct actionable
research that translates topublic health impact.
So totally agree.
And then the supplement modelis actually how we've had our
hybrid work funded thus fargetting supplements to
incorporate discrete choiceexperiments, preference
(09:30):
elicitation methods.
But ideally you could propose astudy that is an observational
implementation hybrid approachand get all those resources
together in one application.
I think it would be a littlemore seamless.
Patrick Sullivan (09:45):
Yeah, and having the observational
implementation hybrid approachnamed and published and
reference of all, I think, makesit more likely that you could
pitch this to a funder and say,like this is a real thing,
because it's already a realthing.
Justin Knox (09:57):
Yeah, that was totally.
Our motivation too is, I thinkpeople are doing this work and
we tried to acknowledge that inthe paper.
But sometimes having somediscourse around this, having a
discussion about its merits orits limitations, and having a
shared understanding of whatsomething means, I think can
hopefully help it bedisseminated and utilized.
So that's definitely ourmotivation and I think we're
(10:19):
hoping to follow up on thispaper with some further guidance
on how you might actually carryout, in a hybrid observational
implementation study, someexamples of where we think it
could be particularly useful,particularly when conducting
research among populationsexperiencing longstanding health
inequities, where there mightbe more urgency than usual to
(10:41):
really conduct research in a waythat it's actionable.
Patrick Sullivan (10:45):
Yeah, I'm glad you raised this piece about
health equity because this isone of the things I think we
need to be asking ourselvesabout.
Everything we do is like wouldthis kind of approach advance
health equity?
And so how do we optimize that?
So I see two things.
One is that it may be that youwould identify different, for
example, preferences foreventual provision of a service
(11:09):
in different groups, by race,ethnicity or by age groups, for
example, that may bedisproportionately impacted, and
so then you could prioritizeand say there's actually a
stronger preference for cellphone based methods in the
groups that are actuallyinequitably impacted by this
health condition, and so we'regonna choose that as a first
(11:30):
approach and if it's a littleless preferred by the people who
are less impacted from anequity point of view, like we
can take that.
So that's one thing is we canactually look within subgroups
to choose the options that willhave the best shot at reducing
inequities.
But the other is this idea ofshortening the timeline, because
health equity and healthjustice, they say justice
(11:52):
delayed is justice denied.
I mean there is a particularurgency in groups that have gaps
in health equity to shortenthis timeline and not have the
books lay sort of end to end,but have them overlap a little
bit so that we build that stackfaster.
Justin Knox (12:10):
Yeah, especially, you know, in these areas,
communities that have, you know,had less investment.
It's important, I think, thatwe're as efficient as possible,
as we've acknowledged this, andstart to invest more.
And certainly, I thinkprioritizing those contexts and
doing, you know, conductingresearch in a way that it can
(12:31):
immediately inform what to do,is a way to hopefully, as you
said, move towards justice.
Patrick Sullivan (12:38):
All right, now we're going to turn to the part
of the podcast we call Behindthe Paper, and we'd like to have
on the podcast sort of abalance between understanding
your science but also talkingabout your process.
And one of the things that Iwant to start with for this
manuscript is that you reallysort of developed and socialized
this idea and then you broughtin a number of other people who
(13:02):
work in implementation scienceto participate as authors.
So talk a little bit about justlike what the value is.
You're an implementationscientist yourself, but you
developed this idea and then youbrought in some other folks,
and so how do they contributedifferently?
Maybe there may be some authorsthat you sit and brainstorm the
(13:23):
concept with, but there may beanother group who meet the
authorship criteria, meaninglike they actually contribute to
the development of the ideas.
They read and approve themanuscript like they're real
authors, but they come in at adifferent stage and add
something different to the finalproduct.
So can you say a little bitabout how you use that those
colleagues strategically inwriting this paper?
Justin Knox (13:45):
Yeah, definitely, it was a great paper in that
regard.
I guess I'll just provide alittle context.
This is the first purely sortof theoretical paper that I've
written.
I'm used to writing empiricalpapers that ultimately can end
up being rather formulaic.
You have a gap, you set up astudy to address that gap.
You analyze data, you interpretthe results, whereas in this
(14:08):
case it was sort of.
I started learning more aboutimplementation science, coming
at it from the epidemiology side, talking with mentors.
What is implementation science?
How could it improve?
Why is it important to our work?
How can we make contributionsthere?
And in thinking through that andcoming with this idea, I had a
(14:28):
great senior author, Calvin Gang, who really helped me formulate
this idea and talk through it,and I was able to start writing
things up and getting hisfeedback.
And then, yeah, it's such agreat opportunity when you come
up with an idea and take thatinitiative to draft something.
Then you can gauge othermentors and say how does this
sound to you?
You have your own take on thisfield and how this could align
(14:52):
and how we could use thisapproach.
What might its limitations be,where gaps that you feel need to
be addressed and discussed, andI really felt so supported and
positive in the way the papercame together and the way that
you read drafts and helped meedit it down and respond to
reviewer comments.
So it's really a great learningexperience and a great
(15:16):
mentoring experience for me asan early stage investigator, to
bring all of my mentors andcolleagues and collaborators
together in an idea that I hopepeople think is a good idea and
they decide to start using it,and it's a way to make research
more impactful and useful.
Patrick Sullivan (15:33):
Yeah, I want to pick up on this idea of
mentorship because it's presentat every stage of your career.
It's present when you'reearlier in your career because
you need it, and it's presentwhen you're in your mid-career
because you start havingdoctoral students who are
looking for mentorship.
So mentorship is an issue thatcuts across all of our careers
and so I wonder for you and I'lljust say, as you mentioned, we
(15:56):
know each other and I've workedtogether outside of the context
of this paper, so I think you'resomeone who's been pretty
intentional about reaching outand asking for mentorship from
folks at different points inyour career.
So can you just say, maybe alittle bit of advice for people
who are here in every way acolleague I wouldn't call you an
earlier career, I just call youa colleague.
But putting yourself back inthat time, what advice would you
(16:18):
have for earlier careercolleagues about seeking
mentorship, like, how do youstart that conversation?
How do you ask?
How do you know when it's overfrom your perspective?
Justin Knox (16:29):
Yeah, yeah, totally .
I mean it's a great question.
It's something I feel.
I've been so fortunate to havegreat mentors throughout my
training and you get sort ofused to it in a way that you're
learning and I think a lot oftimes you're working with
mentors who've received greatmentorship themselves and have
always been incredibly generousand willing to pay it forward.
(16:51):
You know and you realize howyou work together and there can
be growing pains at times.
But I mean for me, as you, asyou noted, it's always, you know
, being willing and wanting totake on that initiative of
saying like I want to sound thisidea off you, I want to get
your input on this.
How can I cut this down?
How can I say this moreefficiently?
What do you think about this?
(17:12):
How can we refine it?
So yeah, and I think the wayacademia is set up is we're
constantly being put intosituations where we can be
mentored, whether you're on atraining program or you're on an
independent, you know, mentor,career development award.
But you know, I feel like you'realways being forced to think of
yourself in the context of ateam, whether it's a team of
(17:33):
mentors or a team ofcollaborators, to get a project
done.
So, you know, as I startedthinking about the idea this for
this paper I had my Kmentorship team in place.
I had people who wereepidemiologists and who were
engaging in implementationscience who I would naturally be
, you know, attending their labmeetings, talking with them
about my ideas, working onpapers together.
(17:56):
So, you know, it came quitenaturally to me, but I imagine,
if you weren't in that context,maybe you didn't.
You know, you're an institutionwhere there's less resources.
In that sense, my experience hasalways been you know, don't be
afraid to.
You know, reach out to anyone.
Like you're at Emory, I'm atColumbia, you know, there's
people on that paper from allover the country, right, and I
(18:18):
feel like, in a way, it's reallylike there are no borders in
academia.
You know, people are alwayshappy to talk about good ideas
and collaborate across Settingsand projects and topics.
Even so, you just have toengage them and I find you'll be
rewarded.
Patrick Sullivan (18:34):
So I'm going to just take a small moment of
personal privilege, because youtalked about, like, how we've
all benefited from mentorshipand just in the last month or so
my major professor from my PhD,Dr.
Ted McDonald, passed away andhe's just a great loss to those
of us who cared about him.
He passed away after a very longcareer and even 20 years after
(18:56):
I graduated he would email meand say I read with interest
your paper on and I did a benchscience PhD.
I went on to do HIV prevention.
You know research andepidemiology, so it's not at all
his field, but he still readthe work of his mentees and
reached out to us and he alwayshad a like, a good question,
(19:17):
like, even though it wasn't hisarea.
And just recently he passedaway and I had a chance to visit
the folks in my lab who hadsupported me during my training.
So I'll just say like I thinkthese mentorship relationships,
when they're done right, affectus our whole careers and I
remember Ted was such gratitudeand along those lines I wonder
(19:40):
you know you have been veryintentional and you sort of talk
about this how do you identifysomeone you think you might want
to be a mentor and how do youapproach them?
I just think it's reallypractical for earlier career
people to know, like is it weirdto just reach out to somebody?
Or, you know, do you look for acommon connection before you do
it?
Like, how have you assembledyour mentor team, how do you
(20:02):
find the people you want tomentor you and how do you engage
with them?
Justin Knox (20:05):
Yeah, I mean, I think everyone has their own
style right.
Some people are completelycomfortable reaching out, some
people like to havecollaborations going on all over
the place and be a little bitinvolved in a lot, whereas some
people really like to dig in andwork very closely with one
person or one team, you know,and people can be productive
across settings in a differentways, and in no way do I want
(20:29):
you know my experience to belike.
This is the way you do it, butfor myself, of course, I can
speak to that and I feel likethe most successful way has been
to work those networks in a way, you know, like, ultimately
we're not a small community ofpeople doing, you know, pretty
intensive research in certainareas and in this case, like,
(20:50):
people bring expertise fromdifferent areas, but a lot of
people know each other, they'veworked together, they can make a
reference for you, they canmake a connection, you can meet
up when you're at a conferenceor you can, you know, jump on a
zoom and shout about an ideathat you have or a paper
proposed to work with their data, and I think for me that's been
the most successful.
(21:11):
I mean, there are some peoplethat have, you know, just
reached out to cold.
I really enjoy your research.
I'd love to work with you on aproject, but for the most part
it's really been.
You know, I think Teo Samfordintroduced us.
Patrick Sullivan (21:23):
Yeah, and when you reach out to people call,
do they respond?
Justin Knox (21:29):
Not all the time.
Patrick Sullivan (21:30):
Yeah, I'm sure .
Justin Knox (21:30):
Like I only remember the people.
Yeah, I'd be like oh yeah, Inever wrote me back and it's
fine, right, like, you move on,they move on and maybe you
reconnect, you know.
So, yeah, you know everyone'ssuper busy, everyone's doing,
you know, things that I thinkexcite them and motivate them
(21:52):
and sometimes there'll be aconnection and other times there
won't.
But I think if you just keeppushing forward with your ideas
and your agenda and what youthink is important and what
you're passionate about, youknow, I do think it ultimately
works out.
Patrick Sullivan (22:04):
Yeah, I want to close just by asking you to
think about.
Maybe you know, like people whoare their PhDs in an
epidemiology program, you knowpeople who are earlier in their
academic careers or CDC orgovernmental careers or whatever
, who are epidemiologists.
But and for implementation,science is like a little bit of
a black box, seems complicated.
What are just a couple steps?
(22:25):
You might suggest that, like ifsomeone says I want to, I just
want to understand more aboutthis.
To think about, do I want tolike develop expertise?
Do I need to know just to beable to talk about it?
So one thing they should readyour paper of course, but beyond
that.
Are there any like trainingprograms or webinars or
textbooks or anything that you?
How would you suggest someonelike dip their toe in and see if
(22:47):
there's a relevant piece forthem?
Justin Knox (22:49):
Yeah, I mean, that's a great question.
Again, I think it probablydepends largely on, like, what
type of learner you are and howyou prefer to engage.
I'm a person who's not afraid tosay something stupid and like
try something new, but meaningthat I, you know, I won't know
as much in that context.
So for me, what was mostengaging and useful was to sign
(23:14):
in and apply for these trainingprograms and implementation
science.
You know, where you getinvolved in a group of people
who are learning this, thinkingabout it, they connect you with
experts and you really have a,you know, a chance to engage in
discussion and react to thingsand talk through things, and
that's the way that I learnedbest.
You know, other people mightprefer to sit down with a couple
(23:36):
articles or an introductorytextbook and really immerse
themselves in the literature andthen maybe be a little more,
you know, refined in theirthinking before they reach out,
and that's probably totally finetoo.
But for me it's really aboutthe collegiality, about the
discussions, about, you know,working in teams and learning
together and, you know, becomingpart of a cohort and
(23:58):
collaborating with mentors and,yeah, so for me it's much more
social.
But I think any way that worksfor you can get the job done.
Patrick Sullivan (24:07):
One thing I'll add to that is, I think, by its
nature, unless you're already aprovider like if you're already
a medical care provider, he'sworking in an opioid
substitution clinic or you know,like you already have the
clinic the knowledge about theimplementation environments.
Implementation science by itsnature requires some networking
of folks who aren't our usualsuspects but who have knowledge
(24:29):
about the implementationsettings.
Because if you want to askthese questions about, I mean,
you really almost need to go tothe sleep apnea clinic or the HV
prevention clinic or the TV,like whatever the thing is, and
say, like if we brought thiskind of intervention, like what
would determine whether youcould scale it up or not, like
how would it look in yoursetting?
(24:50):
And so there is this piece ofconnection with how and where
services are delivered that wethat I usually know nothing
about, and so that's also thislike part of networking.
So the methods get you to acertain distance, but by nature
I think if you dive into this,you have to be willing to maybe
cold call or maybe network alittle bit and figure out how do
(25:13):
we understand more about thesettings in which this would be
implemented, because there's apiece of humility here as
implementation sciencescientists that we can
understand, like the frameworksand collect the data, but in
most cases we're not theproviders or the service you
know providers, and so that's animportant perspective as well.
(25:33):
And thanks for mentioning thatthere's some training programs,
some like online websites, thatreally describe some of these
frameworks and, I think, talkabout them in relatable ways,
and we will put the links to thetraining programs and some of
those major websites that youcould check out in the show
notes.
So if you want to learn alittle bit more, an easy way is
(25:54):
just to visit one of you knowre-aim or like one of the you
know websites for a frameworkwhere they'll really walk
through what the elements ofimplementation science
frameworks are and give someexamples, and that would be
another great way to getoriented.
That brings us to the end ofthis episode.
Thanks again, Dr.
Justin Knox, for joining us totalk about your article on the
observational- implementationhybrid approach and especially
(26:17):
for sharing a little bit moreabout your personal and
professional background and howthis paper came about.
So pleasure to have you on theshow.
Justin Knox (26:24):
Yeah, thanks so much.
I really enjoyed the experience.
Patrick Sullivan (26:32):
I'm your host, Patrick Sullivan.
Thanks for tuning in to thisepisode and see you next time on
EPI Talk (26:37):
Behind the Paper.
EPI Talk is brought to you byAnnals of Epidemiology, the
official journal of the AmericanCollege of Epidemiology.
For a transcript of thispodcast or to read the article
featured on this episode andmore from the journal, you can
visit us online at www.
annals ofepidemiology.
org.
Popular Podcasts
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Decisions, Decisions
Welcome to "Decisions, Decisions," the podcast where boundaries are pushed, and conversations get candid! Join your favorite hosts, Mandii B and WeezyWTF, as they dive deep into the world of non-traditional relationships and explore the often-taboo topics surrounding dating, sex, and love. Every Monday, Mandii and Weezy invite you to unlearn the outdated narratives dictated by traditional patriarchal norms. With a blend of humor, vulnerability, and authenticity, they share their personal journeys navigating their 30s, tackling the complexities of modern relationships, and engaging in thought-provoking discussions that challenge societal expectations. From groundbreaking interviews with diverse guests to relatable stories that resonate with your experiences, "Decisions, Decisions" is your go-to source for open dialogue about what it truly means to love and connect in today's world. Get ready to reshape your understanding of relationships and embrace the freedom of authentic connections—tune in and join the conversation!