#33 Narrative fields and signal assessors, an exploratory study – Joana Félix and Alem Zekarias - Drug Safety Matters

Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Fredrik Brounéus (00:10):
Welcome to Drug Safety Matters, a podcast
by Uppsala Monitoring Center,where we explore current issues
in pharmacovigilance and patientsafety.
My name is Fredrik Brouneus andI will be your host for today's
episode.
The narrative fields inindividual case safety reports
can carry information that iscrucial for the discovery of new
adverse drug reactions, butthey also require significant

(00:33):
amounts of manual labor, both onthe side of the reporter and of
the assessor.
To explore howpharmacovigilance professionals
interact with these narrativefields in their work, my Uppsala
Monitoring Centre colleagues,Alem Zekarias, Senior
pharmacovigilance scientist, andJoana Felix, Medical
informatician, performed aninterview study with assessors

(00:53):
from different parts of theworld.
Today, Joana and Alem arejoining me in the studio to tell
me and you about the resultsand give us some tips on how to
write the perfect narrative.

(01:13):
Welcome to the Drug SafetyMatters studio, Alem.

Alem Zekarias (01:16):
Hi, thank you .

.
.
and Joanna

Joana Félix (01:18):
Hi, thank you so much.

I've really been looking forward to talking
to you about this study of yours, where you have looked at how
pharmacovigilance assessorsinteract with narrative fields
in individual case safetyreports.
Now I'd like to start off witha very basic question, just so
we're all on the same page here.
What do we mean when we saynarrative fields?

So, yes, so narrative fields are parts of
the individual case safetyreports and this is where you
would find relevant clinicaldetails about a patient
experience with a suspected ADR.
So here you could see it as astandalone medical story where
you bring everything together tounderstand what happened

Right.
.
And then how much text are wetalking about?

So normally because you want to include patient
characteristics, details ontheir therapy, medical history
and all of the clinical courseright now, in the current
standard, you can go until100,000 characters, which it
would mean around 24 pages A4 oftext.

That's a lot of text.

It is, yeah.

And what would be a normal?
Because I would suspect thatnot everyone would write 24
pages

Definitely, definitely no.
It's I normally, Definitely no.
It normally would be maybe lessthan a page.
I would assume around that.

?
Do you have to add a narrativeto a report to be able to submit
it?

No, actually.
So the minimum information fora valid safety report should be
at least one identifiablepatient, one ADR, one reporter,
and there should be a drug.
So on the question, if allreports do have them,
unfortunately not.
So, free text clinicalinformation does have an
important role during the signalassessment, as Joana just

(03:17):
provided us, and it differsactually.
It's quite hard to quantifywhat reports, you know the
amount of data, but for somereports they are quite lengthy
but also repetitive maybe, andfor others there are just a
couple of sentences more shortand sweet on the point.
So it differs actually, butalso between stakeholders.
We can see some stakeholdersreally like to write more

(03:38):
lengthy report, which can begood and complement the
structure information, whileother reporters do keep it very,
very short.

Could you tell me why you decided to carry out
this study?
What were the main reasonsbehind it?

So, having worked both of us with signal
detection and assessment foryears, we acknowledged, of
course, the importance ofnarratives in signal management
and in pharmacovigilance ingeneral.
And pharmacovigilance assessorsreview ICSR narratives to
determine causality, severity,but also this expectance of
adverse events.
And they may use like differenttools, like coding systems or

(04:14):
algorithms guided by, likeregulatory templates or their
expertise.
But despite this generalguidelines on the narrative
structure, we know quite littlehow assessors interact with the
narratives during the signalmanagement and what drives this
interaction.
So we wanted to study and tosee like so the study aimed to
identify needs and opportunitiesto support assessors in their

(04:36):
work.
So we wanted to focus on thechallenges and the needs in
order to be able to advance thepharmacovigilance science and
achieve meaningful impact.

Right.
So is this the first study ofits kind?

To the best of our knowledge, yes.

That's very exciting, then, I think, because
it's groundbreaking work, thisexploratory study of yours.
So how did you perform thisstudy?

So, as Alem was saying, this is exploratory
because we believe that nothinghas been done on this topic
before.
So with that sense, we use thequalitative framework and we
designed some interviewguidelines with some structure
around it and we then justselected participants based on

(05:26):
our network and based on, also,their hands-on experience in PV.
So we wanted experts in thepharmacovigilance area.
So basically, through thosecontacts and networks, we
arranged these in-depthinterviews.
They were mainly online and sowe used the interview guide and

(05:47):
after that we just transcribedit, did the dirty work and
recorded it, transcribed it andthen we did an analysis based on
the themes we would find.

Right, and how many participants were there in
all?

There were five participants from four different
countries.

I know that you cannot go into exactly which
countries these were, but couldyou say which regions they
covered?

Yeah, we can say so.
All five participants dorepresent almost all regions
around the world and both lowand middle income as well as
high income settings and, asJoana said, they have a wide,
broad experience in hands-onpharmacovigilance, which was the
key here, because the defaultwe really didn't want to have

(06:33):
the textbook answer.
We really want to have thetruth, like how do you actually
work with the interaction withthe case narratives, and to be
able to capture that real ortrue information, we wanted to
have experienced assessors.

So did you see any cultural differences there?

No, actually.
So, as I said, like they haddifferent backgrounds, you know,
different culture, regions, andwe had pharmacists as well as
medical doctors, and I think themain difference here, you know,
they were more related to theexperience of working with ICSR
rather than related to wherethey come from, whereas, of
course, what resources a centerhas also plays an important role

(07:12):
, rather than what part of theworld the assessors came from.

I would definitely agree with the resource part.
I think this where becausewe're analyzing or describing
interactions is really, reallydependent on the resources that
you have.

And if we jump forward into the results of
your study, your interviewquestions focused on three areas
when, why and how assessorsinteract with the case
narratives, and when analyzingthese answers, you identified
four main themes.
The first is to do with theassessor's view of the

(07:48):
narrative's content and value.
The second is about theprocesses where the interaction
with narratives takes place.
And then the third theme isabout the challenges that the
assessors experience wheninteracting with the narratives.
And the fourth and final theme,then, is about the needs, or
their needs, how they think thattheir interaction with

(08:15):
narratives could be improved orenhanced in different ways.
So if we start with the firsttheme here content and value
could you describe what youfound there?

Yeah, for sure.
So I think, although they camefrom different regions and
backgrounds, I think overall allof the interviews emphasize
really how absolutelyfundamental the narratives are
for their signal managementtasks.
So they really see it as thecore, the heart of the report.

(08:42):
However, like not all reportsor in this case, narratives, are
complete and some lack keydetails that are really
essential for them to reallyunderstand the adverse events.
So, at the same time that it'sseen as a vital source of
clinical information that theycan't get from any other field,
it's also an obstacle and achallenge to their work.

Considering how important the narrative
fields are coming from, a morewell, perhaps a more naive
question would be wouldn't itmake sense for it to be a
compulsory part of the reporting, just as the other fields are?

That's a good point, but again, you know it is
also, we have the structurepart of the report as well as
the free text information andyou do capture quite valuable
information with the structurefield as well.
You know drug and you know whatkind of indication, patient
characteristics and so on.
But it is also time consumingand challenging to report an ADR

(09:44):
and to make that part alsomandatory for making a report
valid can be a challenge, ofcourse, but as we saw in this
study that yes, they do see thevalue narratives, as we can
confirm both Joana and I wereworking with this as well but
also examples of informationthat could contain in these

(10:06):
narratives could be acomprehensive, as you said,
clinical picture thatcomplements the structured
information, but alsoinformation regarding risk
factors.
Can you identify some riskfactors for this patient which,
of course, may increase thelikelihood for developing a
specific ADR?
Temporal information, do we knowwhat, when, how you know the

(10:27):
patient took the drug, forexample?
But also motive to report wassomething one of the
interviewees really highlighted,which we also can understand
and respect, because someonewants to say something with a
report and trying to understandwhat the reporter wants to
convey with the report isimportant and you can't really

(10:47):
always, you know, collect thatinformation in the structured
field, and that's why thenarrative information is so
crucial, since it can, you know,have the possibility to
complement the structuredinformation.
Then you understand why didthis reporter actually take the
time to report this ADR.

So then, as I would interpret your answer also
, it's sort of a balancing acthere too: we want to have as
much information as possible,but at the same time we want to
make it as easy for thereporters as possible to report,
and so if we demand too muchinformation, we risk not getting
any information at all.

Yeah, and we live in a field that is really marred
by underreporting, sodefinitely that balance is key,
like quality, information andquantity and motivating people
to provide rich narratives, butalso facilitating that process,
I think, is key.

And I also think, like if the reporter
understands the importance ofincluding case narratives and
also one, you know, oneimportant aspect, because, like
we are all humans, of courseyou've been driving if you have
this motivation, if youunderstand the impact of
providing this case narratives,maybe you feel even more like
engaged and motivated to reportthis information.

(12:02):
But if you don't reallyunderstand or see the value of
reporting, of course it makes iteven harder to expect the
reporters to also share casenarratives.

From a communications point of view, I
would say that sounds like avery important aim, with your
study as well, to actuallyconvey to reporters how
important these narrative fieldsare for the assessors.
So if we move on to the secondtheme here, the processes, this
is to do with the when when theassessors interact with the

(12:35):
narratives, but also how they doit.
Is that right?

Yes, that's correct.
So most interviews highlightthe importance of narratives in
signal detection, but a caseanalysis, but also during code
validation, emphasizing themanner of interaction with the
narratives fields withoutdigital support.
So the narrative plays acrucial role in coding specific
fields.
There's meaning like drugs,adverse events and validating
pre-processed codes, with someparticipants using automated

(13:01):
coding models, for example.
So during prioritization,narratives are also reviewed to
identify series on unexpectedcases.
This is also a manual processand this is because, you know,
during the pandemic we saw asteep increase of number of
suspected adverse drug reports,But that's like in general when
you do work with signaldetection assessment.

(13:23):
We talked about underreportingbefore, but for some combination
, we can say almost that we havean overreporting and to do that
manually, also to know, likewhich of these you know, in this
amount of data in front of me,which reports are the most
informative ones here and how doI know where to start in this
detective work which is signaldetection and assessment is a

(13:44):
detective work.
So that's also something thatwe could identify in this study,
but also in case assessments.
You know, narratives provideessential information, with some
assessors using tools likeChatGPT for assistance in
analyzing free text andadditionally assessors often
reach out to reporters toclarify or complete missing

(14:06):
details in the narratives whichwe also could identify.
That having that possibility togo back to the primary reporter
for asking them follow-up orconfirmatory questions to what
they have shared before.

I think this is very interesting, because a
lot of what you describe is it'sheavy on the manual labor.
This and here we have oneparticipant describing how they
use chat GPT.
I would expect that that issomething we will be seeing a
lot more of in the future.
What are your reflections onthis use here of chat GPT?

(14:45):
Are there quality concerns orare there?

Yeah, yeah, definitely.
I think it's a matter ofhighlight and maybe concern as
well, but I think the mostimportant thing in this case
specifically highlights howassessors have a task and
sometimes they will come up withtheir own solution to fill
those gaps that they know theyhave.
So, for example, in this casespecifically, the participant

(15:11):
felt that they lacked clinicalexpertise and they had low
resources around them so not alot of resources that could
complement their clinicalexpertise so they turned to
ChatGPT as a tool to supporttheir work, in a way to
complement their work.
They would always say that theywould not be making the

(15:31):
decisions for them, but it wouldbe as a friend, almost as a
colleague, and I think that'sreally interesting, because the
gaps are there and if we don'tactively engage with them and
look for them, they will befilled by a solution.

I agree in this specific case, I think there was
a very innovative solutionactually to do that.
We did the study some time agonow and to be able to know that
the strengths but also thelimitations which this
interviewee knew with it, and touse more Chat GPT for a
sounding board was very wise inthis that this interview didn't
make any decision based on that,but more like a sounding board

(16:12):
due to limited resources.

Then that brings us to the third theme,
which is also very interestingthe challenges that the
assessors experience in theirinteraction with the narratives.
Could you tell us a bit moreabout your findings here?

So some narratives are time consuming
due to the length and repetitivenature, and extracting relevant
data from these narratives,such as timing, risk factors and
lab information, for example,is a manual and labour intensive
process.
So three interviewees noted thechallenge of identifying which
narratives are informative, as Isaid before, as the relevance

(16:50):
of information varies dependingon the clinical situation.
Additionally, two intervieweeshighlighted the difficulty
reporters face in writingcomplete and relevant narratives
, which often lead to irrelevantor duplicate information.
As I said before, like sonarratives, there are certain
standards and guidelines how youshould write them.
If you just repeat what'salready been captured in the

(17:13):
structure information, they'renot really the purpose with the
narratives.
It's not really.
It's pointless, useless, if Ican say it, you know, using
those words.
The narratives do have animportant role in reporting
adverse drug reactions and Ithink, like that's, there's like
in this amount of data for somecombinations, of course, it can
be hard to really find the mostinteresting, helpful case in

(17:36):
this specific assessment if theyare hidden in this amount of
information.

Thinking about this identified challenge here.
You say the sort of the lack ofknowledge among reporters on
how to write a completenarrative, and you touched upon
this already.
Could you give some more tipson how to write the complete
narratives or the perfectnarrative, so to say?

So, I have a medical background, and when you
have medical studies,narratives, the medical
background, is key.
I think one thing that is veryclear is that things that before
maybe we take for granted, thatare given to you during
education in the field, could beemphasized here, maybe to some
education and training about.
There are these guidelines, solet's show them and expose them

(18:30):
to the actual users.
So these guidelines like theCIOMS Working Group 5, for
example, just describes a littlebit what Alem has already said.
Like, in order to have acomplete narrative, you want all
the relevant details.
You don't want to just repeat,but you want to make sure that
everything you need to assessthat case is in the narrative.

(18:52):
And especially, the mostimportant thing is to have an
idea, because you have thispiece of paper or this digital
form where it's a moment in time.
It's not longitudinal.
So the idea is that thenarrative can provide you more
this sequence of time, thislogical progression and clinical

(19:13):
progression of the adverseevent.
You can add information therethat you can add in other fields
, obviously, but they should beconnected to the story.
So if you have laboratory work,if you have risk factors, if
you have other causes, theyshould be written as the story
of the patient and the adverseevent and in that way, when you

(19:33):
read it, it's seamless, andeverything that is relevant is
integrated with the chronologyand that's basically it.
The point is that sometimes, inorder for you to write a
clinical narrative, you have tounderstand what is important at
each step, and when you have aform, everything is important.
Everything has the same fieldsbesides the mandatory

(19:56):
information.
It's hard to place each fieldon a specific timeline.
So that is the key, I think, ismaking sure that, especially
that you don't providecontradictory information, that
you just provide in thestructured field and that the
information that you have in thenarrative follows this logic,

(20:19):
this sequence.

Are these tips present in the reporting form
in some way?
Do they get a pop-up with withtips or?

Currently, I believe no.
No, not at least you know, inour forms.
No, I think what has beenwritten in the .
.
.
obviously, in terms terms ofmedical narratives, a lot has
been written before.
There's a lot of guidelines andit's teaching material for
medical doctors, but I think interms of pharmacovigilance it's

(20:55):
very sparse and sometimes it'sintegrated in regulatory
documents.
So you have to really dig.
That's literally what I did.
You have to read regulatoryguidance.
That will lead you to someguideline that has been written
and there you can find it.
But I guess, as a reporter, youmight find it difficult,
especially because thisregulatory material it's not as

(21:18):
global as PV is.

Do we know anything about how reporters
interact with the narratives?

I think all reporters would provide this
information if they knew andunderstood the importance of
providing this.
As I said before, it's aboutunderstanding why it is
important to report if youexperience an adverse drug
reaction.
So understanding the importancewill hopefully motivate the
reporter to report thisinformation as well, even though
a report is considered validwithout it.
But also, as Joana said, thereare guidelines how to write a

(21:47):
report.
Reporters may also think thatthe information they have may
not be relevant for thatspecific report.
And I think like, try to reporteverything you have, and
because that information may bean important piece in the
assessment as a whole, togetherwith other types of reports,
maybe we see that that specificdrug is taken together in five

(22:10):
other reports from other partsof the world.
So I think it is important totry to stick to the truth, of
course, but write down allinformation you have, since it
can be important in the end.

Yeah, I think that's really key.
I think the fact that even ifthere are guidelines on how to
write a report, there aren'treally guidelines on how to
write a report for pregnancy,for drug-induced liver injury
Each case can be different.
So I think the idea of rightnow providing everything that
you have, it could be a goodstarting point, because there's

(22:45):
a lot of nuances to this.

And I think that's a crucial aspect that you
mentioned here, that actuallythe report in itself is not the
full story, but the report willbe a piece of a much bigger
puzzle, and for the reporter youdon't know which part of this
piece of the puzzle that you areproviding that will be

(23:08):
interesting.
Like you said, it may beaspects there that will tie
together with other aspects inother A nd I I would guess as a
reporter, it's hard to havethat kind of bird's eye view
exactly.

Yeah.
So to your point, it's likewhen you're reporting, it's
really hard to know the globalimpact that this may have or not
, and how will assessors pickthis?
So I think that sometimes Ithink although again, we haven't
done a study on this, but I canassume that a reporter might
already write something with anidea, with a lens, of what this

(23:49):
could be, like or what this orwhy did this happen.
But I agree that most of thetimes you just don't know.
You're hoping to provide someinsights but you're not really
sure which ones are moreimportant.
Yeah, that's a great point.

And then if you don't really hear anything back,
then you start to wonder, whatdid my report or reports – do
they have any impact or value,or what's the point for me to
report in the end?
So that's also very importantfor the national centers to, you
know, provide the reporterswith feedback, but also,
depending on what happens tothose reports, to give the

(24:25):
providers a feedback which canmotivate them to share even more
reports in the future.

Talking about that feedback, how often do you
get feedback on your individualreports?
Because it would be impossiblefor a center to provide feedback
to everyone.

So I only know of one case, at least that has
reached my knowledge, that forexample the Netherlands center,
Lareb, I don't know if theystill do it.
I think it stopped or at leastwas a little bit challenging
during the pandemic.
They would write feedback foreach reporter.
So in a sense exactly meetingwhat you just said.

(25:04):
I don't know if they still doit.
I don't know if other nationalcenters do it, but definitely I
think it's something that hasbeen thought about.

There are actually some national centers
that have this automation, youknow, thank you, for
acknowledging the report andthen, in some cases, giving them
feedback what happened to thereport itself.
And that's also something thatwe try to educate.
You know, when we have contactwith the national centers to try
to keep the reporters engaged,you know, involved and at least
acknowledge them for taking thetime to share the reports.

(25:34):
So there are, but I know somecountries that reward reporters,
but they have shared reportsand they have this yearly event
where they reward medicaldoctors or pharmacists, you know
, reporters, depending on howmany reports they have shared
over the year.
And I think that's a very goodway to really keep motivating
the reporters but also toacknowledge that they are doing

(25:55):
a very important role.

And that it's seen.

It's seen – exactly, because these reports
are crucial for us to be able todrive pharmacovigilance forward
.

So if we then move on to the final theme here,
the needs of the assessors asthey interact with the
narratives, in this theme youhave gathered the assessors' own
suggestions and ideas for howtheir interaction with
narratives could be made, so tosay, easier and also more
fruitful, and we've talked alittle about this already as

(26:26):
well, but it is a lot to do withdifferent kinds of machine
assistance.
Was this expected?

Yes, I think the the suggestions align pretty
well with what we might expect,given the challenges that these
assessors face.
They're looking mainly fortools that can streamline their
work and improve efficiency.
As you said, they're verymanual tasks and they do this
case by case normally, sothey're very repetitive and

(26:56):
sometimes in areas they lackexpertise because it's a very
diverse field and a lot ofclinical details.
So, for example, there was onemention, I think, of this that
goes, also aligned with what wejust said, automated guidance.
So it was a lot aboutautomation, but like automated

(27:18):
guidance for when writingnarratives or for when you were
reporting a little bit whatshould be in it and what is
relevant or not.

So just looking at how fast things are
moving in these areas withautomated assistance of
different kinds of machinelearning, AI assistance and so
on, is it possible to sayanything about where you think
we will be in, say, two years'time, or five years' time?

I would say that I think if you would ask this
question two years ago, theanswer might be different.
I think there's this Gen I waveand the fact that it's
available for everyone to useand it's free.
So, as you see here, someassessors already use it and
it's not even something theyhave developed in-house.
So definitely, I think it'smoving faster now.

(28:07):
I think it might reach aplateau.
I don't know if we'll getbetter and better models every
year, but I think, with a lotthat it's out there already and
some of them are open source, alot of this may have been
impossible tasks from the pastor that required a lot of

(28:28):
efforts, especially technicalefforts might be facilitated.
So I think there's a lot ofdoors being opened.
I don't know if we will revampthe PV system I don't believe so
in two years but I think it'spart of the picture now.
So we have to be aligned withit and I think it's moving.

(28:51):
I think it's moving there thatway, whether we like it or
oppose.
So, yes, I think, as hope and alot of things, this large
language model especially, andthis idea that we can start
trusting them, should bereflected.
I think this is, I think, thebiggest reflection that we

(29:12):
should have, that we believethat maybe they can accomplish
the tasks.
But should they?
So those are the interestingquestions and I think, if we're
moving forward in that area,those are the questions that we
need from our regulators and ourexperts.
Really, if we are going thereand I believe we currently are,

(29:36):
we need to know some thingsbefore.

I agree with that , because there is a big need,
to automize things in the workwe're doing at the moment and to
automize the things that youcan do, with respect of the risk
of data protection, of course,but other things you have to
consider, have in mind.
We need to find ways to how tohave the human in the loop,
always involved, of course, tovalidate and make sure that

(30:08):
everything is in good standard.
But I think, as Joanna justsaid, like we are moving there,
even if we like it or not.
We just need to find ways howto do it and to be aware about
the risk with using them withoutquality, assuring them, but
also to do this in collaborationwith stakeholders, as the
regulators, for example

And include everyone, ideally everyone.
This should be an opportunityfor everyone, so it should be
regulated, but also equal.
I think that's for us, I thinkour global vision is exactly
that.
So, it's not that some shouldmove faster than others.
That's at least what we believe.

Exactly, and some of the manual tasks that we do
within signal assessment, signaldetection could be automized
and that's so the assessor canbe focused on the assessment
part instead.
You know the causality part.
Is there a relationship herebetween the drug and ADR or the
vaccine and the ADR?
And I think that could be agood help to be able to release
the assessors for thisadministrative task and instead

(31:11):
focus on the assessment itself.

So it's kind of two coins.
So it's this very procedural,repetitive that it's just like
it's a pain sometimes, and thenyou have the very complex which
is normally at the end of this.
So you have two dimensionsreally where I guess if, in
regards to your question, maybethe first one like this

(31:36):
procedural part, this process,this very bureaucratic and
repetitive, is something thathaving a human in the loop and
having these tools is somethingthat maybe we are more
comfortable having somethinglike this and then just train,
educate, give time for the othervery complex, which is at the
end of that process.

So more fun work for the assessors.

Exactly, please.

Now, if we're looking at the study as a whole,
what do you think is the mostimportant takeaway?

So we identified several key insights from the
study that present, you know,actionable opportunities with
the potential to drivemeaningful impact.
So the primary objective of thestudy was to deepen our
understanding of an area thathas not been explored.
So, by conducting research inthis nature, we gained the
ability to gather real-worlddata that highlights genuine
challenges, but alsoopportunities for improvement.

(32:35):
We knew from before, which wasalso well confirmed in the study
, that case narratives areimportant when trying to assess
a possible connection between amedicine and an adverse drug
event.
To be able to continue doingpharmacovigilance and ultimately
save patients and consumers, wemust together acknowledge that
the narratives are important,but there is a huge need, which

(32:57):
is like opportunities in theinteractions with it, but also,
of course, how to write anarrative is also very important
, and there is a big need foreducation, training, but also
other you know tool systems thatcan support this.

Yeah, I think the key thing here is that, as
researchers working at a globallens through the Uppsala
Monitoring Center, engaging ininitiatives like this and
talking to where the cyclebegins I guess where the report
begins.
So and I think if we don't doit, normally there's a lot of

(33:33):
degrees of separation and in ourwork, a lot of it is based on
assumptions, and I thinkvalidating assumptions and doing
research about the knowledgethat they have and the
challenges that they have, Ithink is key because we can be
very separated from the mainthing.

So you are both very experienced in the PD
field, so to say experiencedhere at UMC.
Were there any results in thestudy that surprised you?

Not really.
To be honest, it was more orless what we expected.
I think there is a significantneed, as I said you know, around
optimization to support humansin managing and interacting with
increasing volume ofpharmacovigilance data, but also
the handling of this type ofdata.
But what I take away with me iswhat Joanna also said, like
this is you know, this is truedata, this is true needs,

(34:31):
challenges that are out there.
It's hard to see that maybefrom sitting here in Uppsala and
working with the global and toknow that these are actually the
struggles.
The PV assessors, ourcolleagues out there that do
have and that know that someuses ChatGPT but also other use
this and this, was very, ofcourse, informative, but in

(34:52):
general, it was also, as yousaid, we are, we have been
working with this for a while.
It is also feels confirmingthat we are doing this in more
or less the same way.

Yeah, and I think, a key – I don't know if it was
surprising again, because Ithink we feel a lot of these
tensions as well when we look ata report – but this tension of
the narrative being fundamentaland also being the biggest
challenge because just how itcan be incomplete or redundant.

(35:26):
So this tension, I think, isinteresting to explore and to
think more in the future,because we are saying that the
most important thing it's theone that provides you a lot of
challenges and a lot of back andforth and a lot of just
questioning.

Speaking about the future.
This, as you've said, well,it's an exploratory knowledgey,
but you have been able toextract quite a lot of well, if
not new knowledge, y ou've beenable to confirm sort of
knowledge that you already had,but perhaps it also gave you
some inspiration as to what tolook at next when it comes to

(36:09):
narratives.
?

I think there's some, we sometimes call them
low-hanging fruits, but in thissense I think it's more the
spread of some concerns, so howspread these concerns are.
Even in the literature you canfind, for example, there was a
mention of a challenge that theassessors use the narratives to

(36:32):
rip code, for example, and thatis verified in multiple
literature articles that haveattempted to build automated
models to extract from thenarratives and fill in in the
structured reports.
So in a way, I think, if youalign what is in the literature
and what the assessorsexperience, I think this is and

(36:56):
with new large language models,it could be something for the
future.

And I also think that this study of course, as
you said, this is an exploratorystudy, but also this inspires
us to do more of this kind ofstudy for the future before we
start working with the researchproject, try to understand the
problem itself first, talking to, as I said, stakeholders,
colleagues out there, and try toreally understand their problem
in their settings and, based onthat, see what we at the

(37:22):
Uppsala Monitoring Center can doto help them.

Okay, thank you very much for coming to the
show.

Thank you.
Thank you so much.

I learned a lot today.

Yeah good.

I hope to have you here soon again.

Thank you so much .

Thank you, thank you for your time

And if you'd like to learn more about Alem's
and Joanna's study or how towrite useful narratives, check
out the links in the episodeshow notes.
Apart from these conversationswith experts, we offer a series
of audio stories from UppsalaMonitoring Center's
pharmacovigilance news sitecalled Uppsala Reports Long
Reads, so why not check that outtoo?
That's all for now, but we'llbe back soon with more on

(38:06):
medicines safety.
In the meantime, we'd love tohear from you.
Reach out on Facebook, linkedinand X, send comments or
suggestions for the show orquestions for our guests next
time we open up for that andvisit our website to learn more
about what we do to promotesafer use of medicines and
vaccines for everyone,everywhere.

(38:26):
If you like the podcast, pleasesubscribe to make sure you
won't miss an episode; andspread the word so other
listeners can find us too.
For Drug Safety Matters, I'mFredrik Brouneus.
Thanks for listening.

All Episodes

#33 Narrative fields and signal assessors, an exploratory study – Joana Félix and Alem Zekarias

Episode Transcript

Popular Podcasts

Las Culturistas with Matt Rogers and Bowen Yang

40s and Free Agents: NFL Draft Season

Crime Junkie

.css-15opob5{left:0;position:absolute;top:0.8rem;} All Episodes

.css-14f5ked{margin:0;word-break:break-word;display:-webkit-box;-webkit-box-orient:vertical;box-orient:vertical;-webkit-line-clamp:2;overflow:hidden;}#33 Narrative fields and signal assessors, an exploratory study – Joana Félix and Alem Zekarias