Summary

In this conversation, Martín Ramírez and Georgiann Keyport discuss the impact of AI on regulatory compliance professionals. Georgiann shares her background in regulatory affairs and the challenges she has faced in establishing quality systems for startups. They explore the role of AI in regulatory compliance and the need for a compliance culture within organizations. They also discuss the challenges of incorporating AI into medical devices and the regulatory framework for AI-enabled products. Georgiann emphasizes the importance of understanding AI algorithms and the need for trust and transparency in the technology. They also touch on the changes in academia and the education of future regulatory affairs professionals in relation to AI. The conversation concludes with a discussion on the parallels between adopting AI and digitizing quality management systems.

Takeaways

• Establishing a compliance culture is crucial for organizations to effectively incorporate AI into regulatory compliance.

• Understanding AI algorithms and their impact on medical devices is essential for regulatory professionals.

• The regulatory framework for AI-enabled products is still evolving, and there is a need for trust and transparency in the technology.

• Academic programs are starting to incorporate AI into their curriculum for regulatory affairs professionals.

• The adoption of AI in regulatory compliance can be compared to the digitization of quality management systems.

Transcript

Martín Ramírez (00:00.974)

Hi, it's good to see you. How are you doing today?

Georgiann Keyport (00:03.868)

I'm doing great. How are you today?

Martín Ramírez (00:05.72)

I'm well, actually on the road. So I'm not usually in my comfortable setup that I have in our office, but we have the schedule and I'm super excited for today's conversation. And before we go down the rabbit hole and start talking about AI and the impact of AI on regulatory compliance professionals, tell our audience a little bit more about you and your career and your area of expertise, if you will, within the regulatory affairs, regulatory compliance

Georgiann Keyport (00:36.156)

Well, sure. It really, my background really started that supports regulatory affairs starts way back as a researcher. So I was, I'm a cell biologist by training and I worked in the area of growing and manipulating cells to grow up and then squirt something of interest or in some cases the cells themselves were the product. And so I developed the processes to grow them into

turn them, trigger them to start producing their product, their protein, whatever it may be. But then eventually I got into establishing quality systems for small startups. So I worked at a number of startup firms and I love that environment because you are able to really make decisions and in this case, develop these processes that work for our companies.

it was a way for me to really understand the regulations. So I learned really the drug and biologic regulations to start. And then eventually, having done that for a number of years, I moved into, actually it was a firm that made a medical device, but it walked and talked like a drug. It was a product that was a bacterial.

cell that was fermented, so a fermentation process, not anything like medical device manufacturing, but this particular product worked in a physical way, so it was considered a device. But many people who came to inspect them thought they walked and talked like a drug, so they wanted to incorporate the drug, the pharmaceutical regulations into their device manufacturing quality system. So I did that and

Martín Ramírez (02:20.356)

Anyway.

Georgiann Keyport (02:31.934)

the company began to develop their own products. They were a CMO, a contract manufacturer. So we got audited a lot. And that's where I really learned the medical device regulations. And so we, go ahead. Did you have, okay. So we eventually, the company decided to develop their own products. And that's where I got into regulatory affairs.

Martín Ramírez (02:47.482)

No, I was just nodding in agreement, following along.

Georgiann Keyport (02:57.502)

And I began to write about what it was that we made. And I think my background in quality really helped inform me on how to write about and how to think about a particular product. And as I've always said, we tell the story. We write it down and tell the story to the regulators. Most recently, what's been about eight, nine years now, I teach a class in combination products.

Martín Ramírez (03:13.316)

Mm -hmm.

Georgiann Keyport (03:26.346)

because my background sort of lends itself to that. I know the drug pharmaceutical regulations and I know the device regulations of course, combination products is a combination of those. And so I teach a course at a local university here. And that of course keeps me current as well. it's, let's see, I think it was in 2004 I started, I left the industry really to start a consulting business.

really for personal reasons just to have my time and basically have control over my own time. And so I've been doing that for a number of years now and it allows me to work on a variety of different products and I have to learn about them. You have to, in order to write about a product, you have to really understand it, how it works and why it's important to a patient and understanding the unmet need that

that it hopefully brings to the industry. And so that's where I am today. And this is a great topic that we're going to talk about because this particular area of AI technology has really added quite a new flavor, if you will, to the products that are on the market.

Martín Ramírez (04:43.404)

Indeed. Yeah. And thank you so much for sharing your career trajectory. In our conversations that we've had since we met a few months ago, I always learn a lot from your experience and the way you look at it as well. I think as we start talking more about AI and looking at this new toolkit that can help with some components of regulatory compliance, regulatory affairs in general, I think a question that came to mind when I was listening to

career trajectory is for startups or even bigger companies What are some of the common challenges that you're seeing when you are being brought in to build their quality systems? I don't know if you have anything top of mind around usually you stumble upon a pattern or or common challenges or misconceptions around what a quality system should be and how do you handle those

opportunities if that makes

Georgiann Keyport (05:44.946)

Absolutely, are, especially for startups in general, are the initiator of the innovation knows their science. And they don't really understand the process that you need to go through with FDA or any regulator. It's a quality system is something that's used across all products. And they need to understand that

a general process and they really don't understand the regulation and why FDA has established quality systems or GMPs in the drug world. And so it's a really, it's like, what, we have to do that? Really? That seems, we have all this effort and brain power in the product itself and to worry about how we go about making a product time and time again seems not as important.

Martín Ramírez (06:30.36)

Yeah.

Georgiann Keyport (06:43.708)

And it's critical that the product is made the same way every time. So that allows changes to be minimized and the patient and the end user to have a consistent experience with the product. And so it's really becomes, I have found in least my practice, there's a lot of education that goes on. And part of it is assessing whether or not that particular manufacturer or innovator

Martín Ramírez (07:05.402)

Duh.

Georgiann Keyport (07:14.592)

is open to, if they're coachable, are they open to learning it? And that is a struggle sometimes. It's like, I don't need to worry about that, that'll come later. No, it needs to advance along with the development of the product. And that usually is a big stumbling block for them.

Martín Ramírez (07:17.857)

Mm -hmm.

Martín Ramírez (07:24.784)

Thank you.

Martín Ramírez (07:32.737)

Yeah, and you bring up a very important point that we learned very early in our own journey into this industry. And I will even say it's something that we see across multiple manufacturing subsectors, not unique to one where, yes, there are some efficiencies and optimizations that can be gained by incorporating AI into a process or a workflow.

But it's truly the organizations that have a compliance culture or a safety culture, they have prioritized, they have embedded into the entire product life cycle compliance as a core competency of what they do. It's not a checklist at the end of the process. It's more relevant and it's more palpable in terms of value delivery when we are in conversations with companies that even

at the earliest onset of researching a new drug or a new product, they already take into account compliance and regulatory as a requirement or part of the requirements set of the products that they're building. that is a very interesting learning for us where again, along the way, it is not only technology, any technology, not only AI in isolation, you need the people, you need the processes to be in alignment and the technology will follow.

Now, that being said, I'm on the AI side of the equation. We're part of the Allen AI Institute Incubator here in Seattle. And we see a lot of information system problems that we believe in, we're confident that we can solve. But from your perspective as a practitioner in regulatory compliance, as over the past, I would say three years, we continue to hear more more promises of what AI can do. What is AI from your perspective? How do you see it enabling

your practice or even not delivering any value today because the promise versus the realization might be too far apart. So you hear about AI, you see many of new technologies coming your way. How is that impacting your line of work and your expertise?

Georgiann Keyport (09:48.008)

Well, for the most part, it's really learning about it. I am not, and many regulatory practitioners are not software engineers. And it's a whole nother subject matter that in terms of knowing technology that a lot of us are not exposed to. So for instance, it was a real light bulb for me to understand that these algorithms, these AI algorithms are

program to do anything directly like traditional medical software. The magic in these algorithms are they are intended to learn to do something based on rules and data. And then they create a model that's intended to improve and optimize that medical product. So it might be a part of the product, might be a medical software, but it also might just have been used to teach the medical product to do something.

And so that to me was like, whoa. So I guess the answer to question, it's really impacted our industry for those of us who are not well versed in software in general is now to understand how does an algorithm work? What does it do? How is it incorporated into the device? Because like I said, we have to tell the story. We got to talk about it. In order to talk about it, you got to understand it.

And so that's, I think, one of our big challenges.

Martín Ramírez (11:15.12)

And I will assume that given the nature of the innovation and just for added context, I think there is one path in we can think about AI as a component of a medical device. And there is another path that AI is in the workflow of the company and the operators building those products. So Signify is on the workflow side. I think thinking about the product itself,

This must be a NIMR for the FDA and any other regulatory bodies where, as you well mentioned, like once you enable ML and AI capabilities into a medical device, the permutations are its own. And what I mean by it is the one deployment to the hundredth deployment, you will have 100 different variations in that product. Not only we don't have a predicate today, but we don't have quote unquote predictability

how each one of the deployments of that medical device is going to behave. So I don't know if you have any opinions or perspective on how the regulator side is trying to keep up with the innovation and how are the innovators trying to tell the story and finding the appropriate pathway to take these things to market because they do have the opportunity of making devices more efficient, but also they introduce a lot of entropy because again,

They literally change as they

Georgiann Keyport (12:44.2)

Yeah, yeah, there are lots of examples that I've been involved in that are exactly that issue. And these are so it's amazing to be the power that the AI brings to these products. And, and so that's the upside. But of course, the downside is that now how do we explain how do we talk about it? It's not doesn't really fit into FDA's current regulatory framework, I would say. So the way that

currently works is that FDA has what they call panels. these panels are specific to really, they were based on a human anatomy. So that is, we have panels that are specific to ophthalmic products or cardiac products or urology products. And so FDA has employed experts in those areas and they know them very well so they can evaluate them.

But many of these products, and one that I am familiar with, it's a laboratory test where blood is taken and the laboratory method is able to detect small bits of DNA. So cancer circulating tumors will give out tumors, small amounts of DNA. Normal cells do that too. In the process of cells breaking down, they release all their innards, if you will.

and you will see DNA, what they call cellular free DNA floating in your blood, in your plasma. And if you're able to identify a single gene mutation of that DNA from the cancer, you can identify what they call minimal residual disease. And so this is, know, patient who is in remission for a cancer is on edge, you know, waiting to hear

get the results of the next PET scan to see if their cancer has advanced. These technologies allow you to find that really needle in a haystack type of single gene DNA that's in the blood and then tell the clinician this so the clinician can decide, let's put you back on chemo or before. mean, really the current technologies are not sensitive enough to pick that up. And so the cancer has to grow big enough to

Georgiann Keyport (15:09.564)

revealed in these scans. And so these are, this is the type of a product that can identify many small bits of DNA. And so the current framework that FDA uses is, okay, you submit an application for the one that's in a urologic cancer, and it's the assay. And then in order to get the indication,

Martín Ramírez (15:30.606)

Hmm.

Georgiann Keyport (15:38.164)

for one that perhaps it's a biomarker for a breast cancer has now got to go through a completely different panel. You have to submit another application to them using the very same IVD or the same in vitro diagnostic test. And it's just not efficient. And it's because FDA's regulatory framework isn't set up to do that. And so they're struggling with it as well.

Martín Ramírez (15:58.618)

Mm -hmm.

Georgiann Keyport (16:08.078)

And the example that you gave where the product is learning post -market, it's using real -world data now. So an IVD kit, instance, it's using, or there's several different examples of different products that are released to the market. And in one area of the United States even, you've got a high population of Hispanic.

patients in another area of the country you've got high population of Irish say that software that algorithm is continuing to learn that and is developing differently. And so the change control that is necessary, that's the quality system side of things that cross the board as we were talking about is how does a company manage that? I mean, it's changing at real time.

Martín Ramírez (16:38.021)

Mm -hmm.

Martín Ramírez (17:01.422)

Mm -hmm.

Georgiann Keyport (17:05.148)

And I mean, just think about the risk analysis you have to do to manage that. And then how do you explain that to FDA in terms of, it's sort of reactive that it's not proactive. And so FDA actually is working on some systems to or approach, I guess I would call it to that for the industry to sort of manage that, because they know it's happening, they know it will happen. And so their proposal is

Martín Ramírez (17:08.055)

See you.

Martín Ramírez (17:17.221)

Mm -hmm.

Martín Ramírez (17:32.41)

Mm -hmm.

Georgiann Keyport (17:35.006)

think in advance what can happen. So anticipate it and then provide to us really a proactive change management program. And how you do that, what that looks like, that's up to us, the industry, figure out. So there's huge gaps in understanding and different technologies or different companies are developing those processes probably differently. And in time, you know, I think

Martín Ramírez (17:44.346)

Mm -hmm. Mm -hmm.

Martín Ramírez (18:00.869)

Mm -hmm.

Georgiann Keyport (18:04.858)

in time, it'll all we'll learn from each other. We will adapt and some of these common, well established, probably good practices will come out of it. But at this point, it's all we're all feeling through the dark, I

Martín Ramírez (18:18.074)

See you

Would it be correct to say that most of these products, their regulatory filing or pathway that they file will be at Denalvo because there is no point of comparison?

Georgiann Keyport (18:33.19)

Right, if so for a medical device that uses predicates, if it's considered a minimal risk or a medium risk or low risk device, if you have a predicate, you can use the 510K process. But you're right, where there are no predicates, and currently there are essentially none, I know for sure the example I gave you on the minimal residual disease type product, there is no predicate. And so those would all be de novo.

So probably a lot of these will start as genovals. And as you know, they become the predicate for anything that comes after

Martín Ramírez (19:05.872)

Right. Something you mentioned multiple times that really captures my attention is I like how you articulate that part of your job is to tell the story, to help the product manufacturer, the product designer, the makers tell the story of how this is in alignment or in conformity with the applicable regulations and standards and the like. And what is the process to do that? Like I know you have

A lot of scientific data, have clinical trials in some cases, you have the actual design or the formula of the product, you have to conduct research on what exists out in the market. But I don't know if you want to share a little bit of the magic behind the craft around how do you take all of these pieces and inputs and tell a story to regulatory so we can have a dialogue around the safety and readiness of that product to go to market.

Georgiann Keyport (20:06.044)

Yeah, that is, I think, really the biggest challenge is we are, I've often said we're the tail at the end of the snake in that all the product development is going on and research is done, reports are written, testing is done, and we have to write about it. And that is the deliverable, if you will, that goes to the FDA.

That's what they're going to review. So our challenge is to really talk about it in a way that's complete. You do not want to not include something that's pertinent. It might look like an omission and omission is never a good thing. So it needs to be complete. It needs to be accurate. So we have to understand it in order to talk about it. Right. And it has to be concise. And that's that's our the edges of the box for

Martín Ramírez (21:03.692)

Okay.

Georgiann Keyport (21:04.178)

our craft, our submission, our work product. And so if you break each of those down, the first step is really for the regulatory professionals to understand that AI. And of course, what its purpose, we've talked about a couple of different types of products that are either static, where the AI doesn't change, and that's a little bit, that's on one end of the extreme.

Of course, the other one is when the AI is using, it's machine learning and it's using real world data and changes constantly. And so within that continuum, the level of degree that somebody like myself has to learn is different. To what level of degree do I need to understand it to write about it? I can tell you that there are people out

in my profession who are totally engaged. They are listening to every webinar, seminar, going to all of these societal meetings to learn about it, reading all the guidances that are out there. So there's FDA guidance, EU, Japan has just come out some. They are engaged, they're trying to learn it so that they can do exactly what I'm talking about is writing their submissions.

Martín Ramírez (22:22.415)

Mm -hmm.

Georgiann Keyport (22:32.286)

And then there's the other end of the extreme where they're putting their hand head under the in the sand and like are avoiding it and saying I maybe just will limit my practice to non AI type products because it seems so overwhelming and So it's really where do I fit in this continuum? How much of it do I need to understand? You know, some people will say These products are a black

Martín Ramírez (22:43.947)

Thank

Georgiann Keyport (22:59.612)

And I don't really have to understand completely how it works to be able to talk about it. That to some degree that's true. It's really to the extent how much do you need to understand? Because in order when you put pen to paper, you realize I don't really get this. I don't. And then you got to go back. And so I've had this experience many times not even involving AI.

Martín Ramírez (23:07.364)

Mm -hmm.

Martín Ramírez (23:20.282)

Mm -hmm.

Georgiann Keyport (23:28.638)

just understanding the product when I start to write about it. Like, boy, you know, I don't really completely understand the principle of use here. How is it doing that? Then I gotta go back and talk to the innovator and interview them and, okay, now I get it. It's the same thing with AI. We're have to do the same thing. And it's in a field that, you know, maybe some of us don't really have a lot of background in.

Martín Ramírez (23:37.978)

Mm -hmm. Great.

Martín Ramírez (23:46.094)

Mm -hmm. I

Martín Ramírez (23:51.992)

Yeah. And this is a great example of why I believe it's important to spread out regulatory through the entire product life cycle, not at the end. as a technologist and someone who's seen the AI industry kind of evolve over the past five years, when I started, we didn't have a generative AI industry. It's been very interesting to see like all of these new claims and use cases that came,

One of the things that I'm observing very closely is when we're speaking about high -risk products and highly regulated products, a benefit of or a consequence of being highly regulated, the claims have to be highly instantiated and everything has to be very measurable. One thing is to come up with a service that is doing, I don't know, web marketing copy. The other thing is to

a system that is scanning for cancer on a sampling taken from a patient. Right. So I think there is a lot of force function that is being added in the nature of the product in itself into providing real evidence that the AI, whatever the technology might be, generating what the claim is being spread out. And that's something that when I speak to colleagues in the industry and outside of the industry,

Georgiann Keyport (24:57.671)

Mm

Martín Ramírez (25:20.558)

I think the sci -fi hype that is around our technology might be a little bit detrimental for the progression, but I love highly regulated environments because that's where, for lack of a better term, gets real. Now you provide evidence on every claim that you have, and we will see where the technology evolves and what happens in post -market, if that makes sense. Another question that came to mind is,

Georgiann Keyport (25:47.742)

Yeah, yeah.

Martín Ramírez (25:50.628)

Given your unique advantage where you're in the industry, also in academia, and you're seeing how AI is impacting not only your role, but also the role of the manufacturer, are you seeing any changes in what's being taught in the academia and how the new generation of regulatory affairs professionals are thinking of AI differently, if at all? How is this impacting, if in any way, what is being taught in schools?

Georgiann Keyport (26:20.03)

Yeah, that's a great question. In the program I'm in, in the course that I teach, we speak of, it's really combination products, drugs and biologics. So we talk about a combination product that perhaps is the device constituent part is software. Software is a medical device. so, or it could be an instrument that's been informed by AI.

Martín Ramírez (26:42.51)

Mm.

Georgiann Keyport (26:50.076)

And so the level of detail that we get into about the AI is elsewhere. So we just know we need to incorporate it into our discussion and then into the submission. It is such a huge topic. Our particular course is in the Department of Engineering. And I know the Department of Engineering has a course in AI.

And many of the engineers take our class, the combination products class, because they would like to get into the medical products industry. And so now it's really sort of a baton passing, if you will. So in my experience, I think it will, I mean, it's pervasive in these products now already, it will become even more. We're gonna have to start.

Martín Ramírez (27:36.517)

Mm -hmm.

Georgiann Keyport (27:49.382)

including that in our curriculum. But in I think currently right now, it's sort of just out there and we know about it. But at least in our program that I know of, it's really dealt with in the development part of the product, in the engineering and whatnot.

Martín Ramírez (27:51.738)

Mm -hmm.

Martín Ramírez (28:07.888)

Makes sense and and Going back to I like to bounce all over so bear with me here. I promise I'll try to make sense, but I'm going back to designing quality systems, right and and You've probably seen a few cycles of new technologies coming to the market where initially a QMS could have been paper -based now you have an EQMS system our toolkit

became more electronic along the way. Now we have AI in the equation. Do you see any parallels to previous changes in technology or is AI fundamentally different to things that you've seen in the past? Thinking about tools of the craft, if that makes sense.

Georgiann Keyport (28:58.046)

Hmm. Tell me a little bit more about that. So are you wondering about products and the implication of the quality management system on them in terms of how

Martín Ramírez (29:07.876)

I'm thinking about the way of work, where I'm assuming there could have been some friction initially to get into an EQMS. And we're seeing some friction in the adoption of AI in the workflow, not in the products itself. And something that I'm constantly trying to find parallels around is, is this similar to when we digitize quality management systems?

Or is the change management culture behind adopting AI in the way of work, not only in the product, is completely

Georgiann Keyport (29:45.256)

Yeah, so working with a lot of small startups, a lot of them, for instance, have quality systems that are paper based. And to get them to an EMS system or a digital online quality system is a scary thing, especially if it's already been established. Now, basically, transition it to electronic. It's a lot of

Martín Ramírez (29:52.943)

Mmm.

Martín Ramírez (30:02.381)

Hmm, interesting.

Georgiann Keyport (30:13.212)

I've been on projects where we had to take paper based and then put them into an electronic system because you have to configure it in a way that's maybe differently than how they've always worked. Really due to limitations of the electronic system, you have to do things differently now. And so the brain power to change is difficult for people.

And they resist it, but I can tell you once they get there they love it. They are amazed at the time savings. The ease of having to usher even the change of a document through a group of people, whether it's two people or 20 people have to review this thing and make their edits. And now you've got to, you used to have to, you know,

manually incorporate all of those edits. my God, it's done like that. And the discussions behind, should we take all those edits or not? It's all done electronically. And it's so much simpler. The amount of people you need to have in a documentation group to manage all of the change in documentation is reduced significantly. So just in terms of the practice of the company.

Martín Ramírez (31:11.802)

Do it.

Georgiann Keyport (31:37.458)

I think getting it instituted is great. Now, I think your question was more related to using that within an AI environment and utilizing.

Martín Ramírez (31:51.616)

Right. you seeing the same patterns in embracing it? That's the idea because what you're saying about the... Sorry, go

Georgiann Keyport (31:57.54)

Yes, once they get, yeah, I was just gonna say once they get beyond trying to avoid it, their hesitation with it, trying to learn something new, I think is it, it takes brain power. I got all this work to do over here. Now you want me to learn something different? I have to change my processes. I have to change the way the synapses is my own brain work now because I'm having to change is an effort and

Martín Ramírez (32:06.149)

Mm -hmm.

Georgiann Keyport (32:23.944)

It can be exhausting to some people, but once they get there, think the same process is, they do embrace it. They lose that hesitation. They want it. And then the next change, there's always changes to these EQMS processes as well. The manufacturers are making changes and they're always usually deemed an improvement and then you do see

Martín Ramírez (32:25.253)

Mm -hmm.

Georgiann Keyport (32:52.727)

less hesitation with that next change because it's going to only help what they

Martín Ramírez (32:55.791)

Eh

Right. From our perspective, something that we tell inside in our company, we carry the burden of the proof. If we're going around town knocking on your door saying, by the way, give me a portion of your brain power to learn these new technique or this new tool. You don't have to. Like you have a way of work. You have a process. You have your craft. Right. So I think the, what I'm seeing in our industry, not only in our company

the lens from which AI is being assessed these days after the big hype in 2021, 2022 is we the providers, rightly so, carry the burden of proof. Solve real problems, show what it matters, not only the marvel of the technology, but the applicability and the value of the technology. Why is this going to help a medical device manufacturer ensure that their product is the safest product? Show me the evidence, right?

Georgiann Keyport (33:57.68)

Right.

Martín Ramírez (33:57.897)

and I, I'm only for, for data extraction of value rather than other sci -fi claims that we can be making.

Georgiann Keyport (34:07.632)

Right. And I will tell you the industry, because it's new, is nervous about trusting the innovators who are developing this AI for our tools, for us. For instance, I can give you an example. When I get a table of data from an engineer or a clinician, I need to write about it. I need to describe what's in that table and I have to summarize

Martín Ramírez (34:19.642)

Mm -hmm.

Georgiann Keyport (34:37.658)

and conclude what does it mean. And that is a bit of brain power I have to use. I have to understand what's in the table. I have to analyze it so that I can write about it and conclude about it. When I do that, depending on the complexity of the data, that will take me hours and hours to do. And I have seen software do that in minutes. But I'm nervous about

I have to go back and check to make sure, do I agree? I have to sort of quality control or quality, do my quality assurance check on it to make sure it didn't misunderstand something. Or if it did, if I'm in my thinking is in conflict with it, you have to go through and just resolve that. it's a black box, if you will, in terms of how did it do that? And so it makes me, it makes us a little nervous.

Martín Ramírez (35:13.092)

Mm -hmm.

Martín Ramírez (35:26.426)

Mm -hmm.

Georgiann Keyport (35:37.04)

about that. And then of course, there's we've really talked about the ethical issues related to using AI in with medical data. And so there's until people get to trust it and understand it how that little black black box works, there's going to be, you know, hesitation in completely just, you know, giving full range on on it without

Martín Ramírez (35:45.754)

TV.

Martín Ramírez (35:54.158)

Mm -hmm.

Georgiann Keyport (36:06.186)

checking it because ultimately the company who's distributing the product once it gets cleared or approved by FDA is if there is any errors in that, they're the ones who are gonna, the risk is to them and not necessarily the tool maker. you know, so there's, I'm sure some, that'll all have to be worked out as well as what are the best practices for that?

Martín Ramírez (36:24.185)

Isn't it?

Martín Ramírez (36:35.566)

And that's one of, if not the most important question that we ask ourselves. And I also give a lot of credit to the Allen Institute as an organization that is looking beyond the technical aspect of what we're building. What is the definition of ethical in the deployment of our technologies? I think we have the pattern in use that you describe is what we prefer, where we see an expert who has

too many things on her plate and we want to help her be more efficient. But at the end of the day, she's the expert, not a piece of technology, right? Like she has to cross reference that and make the final say. In contrast, one could argue, and these are debates that we don't have all the answers, but we think about this almost on the daily. What if an organization that doesn't have a culture of quality and safety uses

these capabilities and doesn't invest in people, in the experts and says, well, the AI told me the product is safe. Is that an ethical use? We think it is not. It's irresponsible. But once you have these automations, once you have these capabilities, one of the reasons we have regulations is because in the past, someone tried to cut corners, right? So we have to safeguard and start thinking about what is the governance

What are the experiences that we're building that help enforce the involvement of an expert, at least in the iteration of the technology as it is today? Because it is imperfect. And that's something that I spent a lot of time thinking. And again, we don't have all the answers, but once we realize, and this is obvious in many ways, but once you're building a product, you get excited about every feature and everything that it can do.

Georgiann Keyport (38:15.24)

Yeah.

Martín Ramírez (38:34.586)

But if I'm helping a regulatory affairs expert who is working on bringing a new medical device of high risk in its classification, I need to be pragmatic in the value and the role that AI will play in that equation. And we have to recognize that it should be measurable, very easy to explain, and the parameters need to be very obvious. And anyway,

Georgiann Keyport (38:53.012)

Mm -hmm.

Georgiann Keyport (39:03.496)

Yeah, yeah, I think it is. I bet it is. And I think, you know, from our perspective, what would we need from you to begin to trust, you know, the ultimate tool that has algorithms behind it that we can't really check or ensure is correct is that tool working for us. It does it provide

Martín Ramírez (39:04.504)

is a big topic in our world.

Georgiann Keyport (39:33.514)

some details to how it came to what it came to the the output of it. So I know like, well, I guess it does know but GCP will give you an answer, but it won't give you the references. I think there's a new version out that does that. Well, I can't use that then. I need to know where you got your information from. And so if alongside of the answer that is given to me,

Martín Ramírez (39:37.008)

Mm -hmm.

Georgiann Keyport (40:02.596)

is some sort of annotated list of, well, is how the algorithm worked to get to that at some level of detail that I can understand. So that's a challenge for you guys.

Martín Ramírez (40:13.114)

Yeah,

Whenever I use any language model that, you know, it's in that conversational experience, I feel like I'm a middle school teacher where I like show your work every all the time because I need to know. Like, was like, I don't know, that looks made up. Tell me more. Yeah. Right.

Georgiann Keyport (40:32.901)

Yes!

Georgiann Keyport (40:38.536)

Yeah, right. Show your math. Sort of a math. Yeah, I agree. think we and on our side, we need to learn that technology, what you do to a certain degree, as we've talked about it, we need to learn the lingo, we need to learn how to articulate it as well. So, you know, we have to come, think, toward you, toward you as well, halfway, if you will, so that that we all can communicate to one another.

Martín Ramírez (41:06.992)

I love that. the end, 100%. At the end of the day, we believe that it is in that integration. Again, you bring the expertise on the sciences, on the products, on the regulatory side. We're bringing new capabilities. The end be all is that we truly help companies bring the best and highest quality products to market. And we're trying very, very hard to find that and remain pragmatic.

Georgiann Keyport (41:08.092)

almost become bilingual, you

Martín Ramírez (41:35.76)

to what can be accomplished today versus, you know, we have a bigger story, there's a big vision, we see that we have opinions about the future of the technology, but we believe in being pragmatic right now, solve rare problems, show the work, show the math, and build trust from there. We are at about time. I don't know if you want to share anything about your practice where people can find more about you and your business, if you want to share anything with the audience.

Again, delighted to speak with you again. I always get a kick out of these conversations. Love to spend time talking with you, but anything else that you want to share and work, people can find you if you

Georgiann Keyport (42:15.476)

Sure, you can go to our website, www .canapyregulatory .com and you can get in touch with us. Our practices are generally in the field of combination products, just because that's what our background is and what we find interesting and meaty to work on. So yeah, feel free to reach out to me. I'm also on LinkedIn, Georgian Keyport under Canopy Regulatory Solutions. So yeah, I look forward

chatting with any of you about any of your projects. And I really thank you for the opportunity, Martin, to talk about this. It's always fun, talk sharp, you know, somebody and learn new things. It's really an environment for people who love to

Martín Ramírez (42:52.432)

Indeed.

Martín Ramírez (42:59.216)

Yeah, 100%. Well, thank you so much and enjoy the rest of the day. And I'm going to stop recording.

Georgiann Keyport (43:03.902)

Okay, you too.