Vital.io Translating doctor jargon into regular human English

0:00

Welcome to the prompt engineering podcast, where we teach you the art of writing effective prompts for AI systems like chat, GPT, mid journey, Dolly, and more. Here's your host, Greg Schwartz.

Greg: 0:15

Welcome to a joint episode of the

Wes: 0:17

Prompt Engineering podcast and the How to Talk AI podcast.

Greg: 0:21

We've got some awesome guests, so go ahead and introduce

Wes: 0:24

yourselves, guys.

Aaron Patzer, CEO of Vital: 0:25

Yeah, I am Aaron, er, the co-founder and c e o of Vital.

Felix Brann, VP Data Sci: 0:30

And I'm Felix Brand at

Aaron Patzer, CEO of Vital: 0:31

Vice President of Data Science.

Wes: 0:34

And they have a terrific product that they have a launch. Today we're gonna hear all about it. I think it's something that would resonate with everyone and anyone that's been to the doctor and had questions about what, what was being told

Aaron Patzer, CEO of Vital: 0:46

to them. Yes. I

Greg: 0:47

already tested it after watching your talk. Cool. Really? Yeah. So I have sleep apnea. Yeah. I put in a long diagnosis with a bunch of stuff that I'm like, okay, I think I know what that is. Yeah. I don't know what the hell that is. Yeah. And it was like, sleep apnea, obstructive. Yeah.

Aaron Patzer, CEO of Vital: 1:02

And two other things. Yeah. Oh.

Wes: 1:04

Fantastic. Okay. That's great. I, I think like you said what person hasn't seen a whole long list of doctor's notes or even been in a situation where you're maybe an inpatient in the hospital and then the doctor on rounds is coming by and telling you something in a million miles a minute because he's got 20 other people to see. But it's probably important because it affects your own health and being and like, you're Probably already out of it anyway, because you're in the hospital. What a terrific way to,

Aaron Patzer, CEO of Vital: 1:32

Provide something. Doctor's notes are really almost like a foreign language. Yeah. As I said in my talk, doctors don't say nosebleed, they say epistasis. They don't say, hey, your mom has had a stroke. They say, oh, she's had a cerebral infarction. They use all of these abbreviations. It's almost impossible to... Understand. And so we use the large language model, As the core of what we call our Doctor to Patient Translator, and it's at vital. io slash translate. It's free to the public, available worldwide, literally as of today. You're just catching me at a good time. And we're happy to tell you, a bit about the prompts, the classifiers, the free parts, and all the things that we offer. To make that possible, technically.

Wes: 2:19

Yeah, that would be great. I would love to I would love to delve into some of the the technical aspects. Maybe this is a better question for Felix. Could you tell us a little bit about how the model was going to be trained and what data was used to be able to produce these great

Aaron Patzer, CEO of Vital: 2:29

completions? Sure.

Felix Brann, VP Data Sci: 2:31

We've tried a number of different prompts because there are actually a lot of different types of doctor's notes. And with the public facing stuff, we know that we're going to get the whole gamut from imaging all the way to discharging stuff. We People, when they get their paper discharge instructions upwards of 90% of them chuck them straight in the bin as soon as they leave the hospital. And the literature people understand their care, and understand the follow up instructions the doctors are giving them, their post care situation is way lower. So we've looked at different prompts for different situations and then built a pre model classifier, a pre LLM classifier, also using a language model, a small one deciding which of our various prompts should you have to write some nodes and then we have a whole bunch of post parsing, it comes out, we take sections out of translation, we plug those sections of the website, maybe when you saw it, you could see that you get like a very brief summary. And then also a much more sort of technical breakdown. Yes. So we're getting the LLM to pull out a lot of information about what's in your justice note, but we want to show you in like a digestible

Aaron Patzer, CEO of Vital: 3:39

summary first. Yeah. I think an important piece of context is a lot of these doctors notes, they're 10 or 15 pages long, and they have 80% boilerplate. Yeah. They have a, Hey, don't smoke. Or I don't. Hey, here's COVID education. Okay. You're two years out of date. And they put a lot of filler in there. And this is actually just a fraction of our primary business. Our primary business is patient experience offer. It guides you through an ER visit, or if you have to stay overnight in the hospital, it explains your lab results, how long you're going to wait. And then your notes. Yeah. And because we have experience with a million patients a year using it, we know the structure of notes. from all over the country. And so we can pre parse, and instead of a 10 or 15 page, we can get it down to actually we only need to pass 3 or 4 pages into the LLM. That's an important business and engineering consideration, because cost and speed. Also context window. If you're doing, especially if you're using few shot training with an LLM, which is a good idea so that you know what output you want to get. You'll blow through your prompt, your future shot, your data, and then your output, it has to fit into a 4k window or a 16k window. And so you need to do a few things to give yourself as much profit as possible. That makes

Wes: 5:02

complete sense, but having the almost sub prompts acting like little sub agents themselves trained to say just get rid of all the boilerplate stuff that's not unique to that patient's differential diagnosis.

Aaron Patzer, CEO of Vital: 5:17

Exactly. So deciding which part you're going to do... Are more or less with your own code or your own classifiers, and then how much to send, especially if you're using a, like a commercial l m. And we've used both. Felix's got, Lama up and running and too Yeah. Med Palm lm, which is medical specific, obviously. The open ai, we can't actually use open AI directly. You have to use it through Azure because you. You need this to be hit. We're in a regulated industry. Open AI will not sign all of those things. You actually have to like Work your way through Corporate Microsoft. Yep, they'll determine whether you're a worthwhile person or not, and whether they're willing to take the risk, and so if you put all of it, you can, with a sophisticated prompt, put it all through WebMGT. You can say, classify this. Is this a discharge report? Is this a physical therapy report? Or is this a hostile input? By the way, you should always protect against hostile input. Is this a non English input? Is this something else entirely? So you want, and then... In your prompt, you can say based on the classification, then do this. But if you do all that, your prompt starts to get very complicated and very big. You can use that to prototype, but when you go into production, this is also very slow, it gets very expensive, you run a classifier that's much simpler and much quicker on top of it, and then you don't have the expense, your prompt's shorter. And then you can say, if it's this, go to this prompt. If it's that, go to that prompt. You can also templatize prompts. So if you say, I want the output in Spanish, you can put a variable in your prompt. So the prompts, don't think of them as static strings. Think of them as a programming language that is frankly pseudocode, yeah? One of the things that, this is a bit like medical specific, but the part that's very important to patients is the plan and assessment, what the

Wes: 7:07

doctor says you're supposed to do. Here's the

Aaron Patzer, CEO of Vital: 7:09

problem. In some hospitals it's called plan and assessment. In other hospitals it's called assessment. In other hospitals it's called plan. In other hospitals it's got like an abbreviation. And with classic programming if I say match panda and I give it pandas with a plural, it's no

Wes: 7:23

match. Or you got a space in your column

Aaron Patzer, CEO of Vital: 7:25

header. Exactly. But with an L M, I can just be like, it's gonna be called this, or probably this. It's got stuff that kind of looks like this and like it's good enough that if I explain it to you guys, you'd be like, oh, okay. I know what you're looking for. That's the power of LLMs is you can give them. Vague pseudocode. Yeah, and to me, that's mind blowing. This guy actually knows a map of how that's passed. So

Greg: 7:48

real quick before we get into that, just for the audience, part of what I do on my podcast is like, What are all these technical terms? Content window, number one. It's literally how much stuff you're putting into the prompt, but also how much it's filling out, and if you do too much, it forgets the stuff outside the prompt window. Sorry, the context window. And so you have to be careful how long everything is. That's what they're talking about when you're saying, if I can pull pieces of the prompt out and only run them separately, it's way better.

Felix Brann, VP Data Sci: 9:16

It's a key reason to innovate in your own models, because for a long time you've been working with 4K context window, and if you're doing this few shot in context learning, as Aaron says, you just run through it.

Aaron Patzer, CEO of Vital: 9:27

Yeah. And also, I'm the CEO as well as, maybe you can tell I have a bit of an engineering background, not as good as this guy. I don't have the British accent, which is, that's true. And

Wes: 9:38

also, that adds

Aaron Patzer, CEO of Vital: 9:39

20 IQ points, right? Yes. But as the CEO, I have to think through the economics, right? If you were using GPT 4 and you give it the 16K 32K window, the maximum one, it's going to cost you, if you fully fill that thing, it's going to cost you about 48 cents per, translation or transformation, right? Yeah. We have a million patients on our platform. They have about five nodes each. You do the math on that and you're spending 5, 000 a day. Yeah. If that's what you do. You don't need to. You use smaller context windows, or you use 3. 5 Turbo, or you run Llama. Yeah. Or you use one LLM to pre parse for a different LLM. You can do, those are the tricks that like, practically speaking, this is an immature industry because you have to hand do All of that.

Felix Brann, VP Data Sci: 10:28

And what's really interesting is, some of these problems are really exciting and new. As Aaron says, you're trying to pull out something that's very undefined in free text document. Okay. So that's you need some modern stuff to do that. But some of these problems are pretty traditional. Classifying a document and you've got, plenty of examples. You don't need to go and use your OpenAI LLM to do this classification problem. We've been doing this for a long time. And you can do them a lot cheaper.

Aaron Patzer, CEO of Vital: 10:52

Yeah, it's slow and expensive to use OpenAI, or Google, or NetApp for basic classifications. But it's great for prototyping. So the key

Felix Brann, VP Data Sci: 11:02

insight, is work out the piece that you really need the expensive tech for, and ensure that you boil down the problem only to that, using other pieces of technology

Aaron Patzer, CEO of Vital: 11:10

upstream. Yeah. So how do you handle,

Wes: 11:12

Like the, if you have all these prompts essentially acting as agents, and you have to have this sequence occur. In a specific order, how do you asynchronously is there a specific layer that's doing the handoff? Are they doing the, are they doing a turnover at rounds? In between

Greg: 11:29

synchronization, let

Aaron Patzer, CEO of Vital: 11:30

me get a little technical. So we use an event. sourced architecture. So this is outside of AI, which basically means that we handle streaming data quite well. So we have data that's streaming from over 100 hospitals now, more or less real time. It comes out of Cerner, Epic, whatever the electronic medical record system is. So a doctor writes a new note, finishes it, it hits our system and goes on to the parsed, classified, cut up into little bits, and then divvied out to the yeah, you need to synchronize it so you have queues of work. Those queues can back up. We just launched this.

12:07

Unfortunately at this point, we had some audio challenges. So the video will continue. But going forward, we're only able to use audio from a much lower quality source. So it's going to get kind of noisy from here. I'm sorry about that. The rest of the interview is definitely very interesting. But it was a pretty noisy room.

Aaron Patzer, CEO of Vital: 12:28

I've been so busy with talking to people. For all I know, the system is, got an hour wait queue back up. But it won't

Greg: 12:33

fall over. It will just queue up. It took two tries, and it was about 45 seconds, but it worked! That

Aaron Patzer, CEO of Vital: 12:39

means, eventually, that's actually, I'm like, happy to hear that, not from your experience, but it means that we're putting serious load on this. It means that people are, this is a good day in the history of Python. But you have to have a robust architecture to handle that and not get things out of order and handle server restarts and all of that, so that's a, it's a pretty engineering response, but yeah, it can be

Felix Brann, VP Data Sci: 13:02

handled. And to speak to Aaron's answer earlier, this is something new that we're doing, but we have, what, a good four products at the moment? Yes. We have a patient experience product, which is going to guide your experience through the emergency room. Yeah. And we're doing a bunch of AI there. We're predicting, how long are you going to wait for a bed? How long are you going to wait until a doctor comes and sees you? Yeah. What are the lab results that you're, that are coming back, what do they really mean? for you. We've got a product for care teams. We're providing clinical decision support alerting. Are you likely to get sepsis at some point in your stay? How likely are you to be admitted? Like allowing doctors to manage their workflows using this kind of alerting system. We've got a system which allows you to find follow up care afterwards. And so basically, we've been doing this for a long time. We've been doing it, what are we, like six years now? Six years, yeah. Yeah, we and we've been dealing with this huge pipe of patient data for a long time. We're not new to this. The event sourcing stuff, that's not for the LLM stuff. That's running our systems. That's running our systems at a hundred hospitals, a million patient visits. That's, that stuff has been the

Aaron Patzer, CEO of Vital: 14:01

easy part for sure. That's right. So if, if this sounds foreign or if you don't have a system like that with the robust retry mechanism it'll take you a couple of years of engineering to get to that solid

Wes: 14:11

system. That's some getting your hands dirty, just in the mud. Yeah. Noting and debugging just to get there.

Felix Brann, VP Data Sci: 14:16

Medical data, the messiest data I've

Greg: 14:19

worked with

Wes: 14:20

so far. That's a great, that's a great segue maybe into can you tell us a little bit about the process that you had to go through to have an LLM, Handling HIPAA, HIPAA secure patient data. Yeah. I know this is a big fear that a lot of enterprise customers have. We don't want our trade secrets to get out there. We have legal, proprietary, interactions with our clients.

Aaron Patzer, CEO of Vital: 14:43

Yeah. We're in a regulated industry, right? This is, fortunately or unfortunately, not new to me. I was the founder of a company called Mint. com. We took, The usernames and passwords for 25 million people and a hundred million bank accounts, including

Greg: 15:00

me. Yeah, it was a long time ago. Including me! That's right.

Felix Brann, VP Data Sci: 15:03

No. Yeah,

Aaron Patzer, CEO of Vital: 15:04

and have never had a security breach. At least to my knowledge. I sold the company about a decade ago. So we're used to dealing with sensitive information. You want outside penetration testing outside audits. HIPAA and HITRUST is even more thorough, is routine outside security audits. Honestly, it can sometimes be a pain to log into our own systems requires multi factor fingerprints and a drop of blood, but it is very secure. You can You cannot do this with OpenAI. You have to go with, Google will sign what's known as a BAA, a Business Associates Agreement. And it's part of the medical chain of liability that says, hey, we have the right insurance. If we mess up, we have to legally report it to you, and you have to report it back to the health system. Here's our security practices, and we have to look at those, and we have a whole compliance office. To do all

Wes: 16:06

of this.

Aaron Patzer, CEO of Vital: 16:07

And so you actually can't go in some sense with the l and m startups. Yeah. Microsoft Azure is a fantastic choice to start out with. Google's been aggressive once they saw what we were doing.'cause this has been this has been out internally in our products for two or three months. And yeah, but they're also Google and Microsoft. They know what they're doing when it comes to. security. Honestly, when it comes to medical information, it's all the people who are still running local servers with. Yeah, that's it. You want to know why they have so many like Malware attacks. They're on an old version of Windows. They don't patch their stuff. And, they may or may not be the, the best IT people in the business. I absolutely trust the security of AWS and Microsoft and Google. Because they have too much to lose as companies. We have a super secure system. And we trial it all the time.

Felix Brann, VP Data Sci: 17:02

And obviously, our BAA includes none of our data being used for training.

Aaron Patzer, CEO of Vital: 17:07

Of course. Yeah. Nice.

Wes: 17:10

Speaking of the patient experience, right? Yeah. If, is it a bespoke interaction each time I log onto the app? Yeah. Or does it keep my health record, so to speak, so I can refer back to the last time I used it? And then, is that stored locally on my device, or is it used, in any sort of... process to make

Aaron Patzer, CEO of Vital: 17:29

the tool better. So our primary business is a tool that guides you through your visit at an ER or inpatient. And that is visit based. So we know what your health history is and we might show you a little bit of your past visit, but it's meant to use at the time that you're at the hospital or the emergency room or having surgery or something like that. And it's just walking you through that experience and understand your lab results. These are the videos you should watch so you can understand it. These are the medications and what you need to know about the side effects. We give you access to that data for the couple weeks following your visit, but we always hand you off to the patient at least for now. And I will be tight lipped about whether you will ever have a full health history. IE, I've been pitched probably a dozen times on, we're the mint for healthcare. And I was like, I could do the mint. It's

Wes: 18:18

vaguely familiar as a

Aaron Patzer, CEO of Vital: 18:20

business concept. I've done this before. So nothing to announce today, it's in the back of my mind. I'm sure people would, of course it would resonate with

Wes: 18:28

someone to be able to query. years and years of interactions, and not to mention the opportunities that if you apply some machine learning over top of some of that diagnostic opportunities to catch

Aaron Patzer, CEO of Vital: 18:40

things early. Now it's like you're inside what my long term business vision is. Theoretically, I could calculate your health future if I had a big enough data set. Yep. And keep in mind that I... At Vital, we now see 2% of all U. S. Emergency medicines. Wow! For a startup that's been around for not that long, that's a pretty good sample size. We can see how diseases progress and, that there's more of this type of fall in the winter than there is in the summer, right?

Wes: 19:12

I know there's entire industries, like the health insurance

Aaron Patzer, CEO of Vital: 19:15

industry

Wes: 19:16

that has it modeled on curves exactly when you're gonna die based on, the fact that you went skydiving once when

vital interview - wide: 19:22

you

Aaron Patzer, CEO of Vital: 19:22

were 31. Sure. Yeah. I could probably predict whether, Bird and Lime are doing business well based on the number of elbow injuries and wrist fractures that we can plot over time. That's unfortunately not a joke. Wow.

Greg: 19:37

Okay, then I have to ask, since Google got rid of the, I forget what they called it, but the flu predictor feature that they had for so long? Is that something you guys might potentially product?

Aaron Patzer, CEO of Vital: 19:47

No, we won't use that sort of stuff. It's really interesting and we probably could do it internally. But And we did come up with a COVID checker. We did come up with a COVID checker that was used a million and a half times. Wow. Yeah. We were the first one out before Google, before Microsoft. We were, CDC considered using us. I was literally on the phone with the White House Task Force in the middle of the night developing this thing. A million and a half uses within the first month. We did the COVID checking for the state of

Felix Brann, VP Data Sci: 20:12

Oregon, right? Yeah. The whole state. We pivoted the whole company as soon as the pandemic started. Yeah. Said, okay, we've got all this health data coming in. We've got the data science chart. Let's try and do something quickly with that. Yeah, nice.

Aaron Patzer, CEO of Vital: 20:24

But the sort of north star for the company is what's right for the patient. will it improve patient outcomes? I'm really tired of most of healthcare. I'm looking at you. Medicare Advantage. Who is, frankly, just financial arbitrage. They're basically like, Okay, so the government says New York's a more expensive place. We'll pay 1, 400 a month for somebody over 65. Phoenix is cheaper, so we'll pay you 1, 100. And Medicare Advantage companies are just like, You know what, we'll advertise in rich zip codes to get healthy, wealthy people and we'll leave the rest to the public system. They're not improving patient outcomes. They're not increasing utilization. They put up barriers and blocks. Like you have to get a referral from your primary care doctor. We will do none of that. There are lots of ways to make money in healthcare. Our investors sometimes push us towards that. I have fortunately had a successful start up. I don't know, I'm not doing this, for the money primarily. Just want to do what actually affects patient health.

Felix Brann, VP Data Sci: 21:25

Yeah, we all have other things that we could be doing, there are other ways to make money, but we, I've never been in a more mission led company,

Aaron Patzer, CEO of Vital: 21:31

so thank you first.

Wes: 21:33

Yeah, and it resonates with everybody, even if they're healthy, we've got parents, we've got grandparents, who like who wouldn't feel empowered and able to help them out just with their care make them feel a little more at ease during a A time of struggle.

Aaron Patzer, CEO of Vital: 21:46

Completely. And actually I think one of the best use cases for what we launched today, Vital. io slash Translate, is if you have an elderly parent or somebody that you're caring for, especially if they're elderly and they're a little confused and they went to the doctor's office and they're like, hey dad, what did ahhh. Put their notes in there and see what actually comes out. Diseases and issues they actually have. Yeah. I was talking I don't know the full story, but, I had a friend whose sister died basically because they didn't catch something that was on page three or four. Because humans can't scan text that quickly. And you might have hundreds of pages of medical history if you're a chronically ill person. And sometimes that history really matters. And doctors give it like two or three minutes to maybe scan through. AI does a way better job of picking out. The stuff that they might need. The fair comparison, this is, listen, This, we've marked it as 99. 4% safe for animal adopters, independent people, employed by the company. It's not. Without risk. If 1 in 200 times, it'll miss something small. But doctors miss something big. 1 in every 10 times. And so the stats are actually much better for AI than they are for humans, and that's the problem. And

Felix Brann, VP Data Sci: 23:08

when we, we have some great clinicians in our team, when I talk to our clinical staff, our advisory board about the stuff that they really want to see, all of them talk about patients paying attention to and understanding their discharge instructions. The value there is enormous. The value in terms of long term care and in terms of immediate outcomes is huge.

Aaron Patzer, CEO of Vital: 23:28

Nice. Were there

Wes: 23:30

different specialties within medicine that were a little more challenging?

Aaron Patzer, CEO of Vital: 23:35

We started out with medical imaging. Medical imaging is nice because it's confined. CT scans, x rays, MRIs and then, what we released today I don't know what people are going to put into it. And so it has to be pretty robust to doctor's notes, nurse's notes lab results, all sorts of things. It's time to wrap it up. Yeah. That's a good time to wrap it up. We really

Wes: 24:07

appreciate your time today to, come talk to us guys. Such a product that I think everyone can. You can benefit from learn from other family members of these. And just remind listeners at HTTTA The Rob's Engineering Podcast.

Aaron Patzer, CEO of Vital: 24:21

Yeah, thank you. We really appreciate the time. Yeah, Thank you for having us. Yeah, we never get to talk about the nerdy tech stuff. Dude, we can go even harder. Oh,

Wes: 24:30

I'm gonna change the memory card for that, yeah, I think I held off. I'm like, all right, tell us about your air handler later. Yeah, I was like no, we're not going that

Greg: 24:37

deep. We're not going

Aaron Patzer, CEO of Vital: 24:37

that deep. Fantastic. Thank you. Thank you

Greg: 24:42

guys.

24:44

Thanks for coming to the prompt engineering podcasts podcast dedicated helping you be a better prompt engineer Episodes are released every Wednesday I also host weekly masterminds where you can collaborate with me and 50 other people live on zoom to improve your prompts Join us at promptengineeringmastermind. com for the schedule of the upcoming masterminds. Finally, please remember to like and subscribe. If you're listening to the audio podcast, rate us five stars. That helps us teach more people. And if you're listening to the podcast, you might want to join us on YouTube so you can actually see the prompts. You can do that by going to youtube. com slash at prompt engineering podcast. See you next week.