Session #30 Breaking Down Silos: Resolving Academic, Medical, and Research Interests Once and for All Session #30 Breaking Down Silos: Resolving Academic, Medical, and Research Interests Once and for All [109:10] [Eric] Okay. We’re going to start with just a couple of housekeeping items. I’m sure most people are familiar with the app by now. But make sure to interact with the app. While the presentation is going, you can submit questions any time and you can vote for the questions that you really want to hear the answers to. And also use the applause button. We’ll have a brief review of the applause at the end of the program.
Transcript
Session #30
Breaking Down Silos: Resolving Academic, Medical, and Research Interests Once and for All
Session #30 Breaking Down Silos: Resolving Academic, Medical, and Research Interests Once and for All [109:10] [Eric] Okay. We’re going to start with just a couple of housekeeping items. I’m sure most people are familiar with the app by now. But make sure to interact with the app. While the presentation is going, you can submit questions any time and you can vote for the questions that you really want to hear the answers to. And also use the applause button. We’ll have a brief review of the applause at the end of the program.
Before we get started, I would like to review the results of the pre-‐session poll question, “what best describes my organization?” And there are several choices there. And we’ll turn that over to Marshall. [Marshall] The pre-‐session question was “what describes my organization?” And those results are just coming up right now. We have 40 percent a) what describes my system would be an integrated delivery system; 10 percent community hospital; 40 percent academic medical center; and the results are changing just slightly with a 10 percent accountable care organization. Thank you. [Eric] Thanks, Marshall. And we do have some other analysts back there. We have Joe and Dan Hopkins who will be helping us out. It’s my pleasure to introduce Dr. Sam Volchenboum who is the assistant professor of Pediatrics, the director for the Center for Research Informatics, and associate chief research information officer at the University of Chicago Medical Center. The title of his talk is Breaking Down Silos: Resolving Academic, Medical, and Research Interests Once and for All. [Samuel Volchenboum, MD, PhD] Great. Thanks, Eric. So I’m really glad to be able to come here and speak today. I am Sam Volchenboum from the University of Chicago. I’m a practicing pediatric oncologist and I’m also an informatician and one of the main roles I serve at the hospital is running our Center for Research Informatics. And today, I’m going to tell you about my view of why the center has been successful, where it’s come up short, and what are some of the pain points that we have experienced along the way.
The Problem [111:30] So everybody is familiar with the problem, right? It’s very difficult to do all the things that an academic medical center wants to do and it wants to serve patients and give great clinical care, but at the same time, the hospital wants to support research and it wants to support its educational missions. And as you can imagine, when times get tight and there’s no money, which is the one that’s going to win every time? The clinical mission, right? Every single time the hospital’s needs – taking care of patients -‐ wins over somebody’s clinical trial or somebody’s seminar that they want to teach. And I think a lot of hospitals now are suffering from not being able to balance these issues with the budgets that they have. One of the things that happens -‐ one of the ways that institutions try to get around this is they hire a very famous researcher to set up a lab and the researcher comes in and says, “Okay, I’m going to set up this genomics lab and then as part of this lab, I’m going to give services to every other group here. So I’ll be sort of a core but I’ll do my research mission as well” and they give a lot of money to that person and they set up their group. And then, what inevitably happens over time, is that that person’s own research program starts to get bigger and bigger and you get more of a silo and then people aren’t able to access the services they need.
And that’s the problem that we saw as well with some of the groups that we had set up when it came to providing the kinds of informatics services that I’ll talk about.
In this session, you will learn… [113:03] So today I’m going to talk about three things – how can an academic medical center serve those three missions, what are some of the pain points as you try to develop services that can be used across the enterprise (I think having very strong top-‐down leadership here, all the way from the dean on down is really important), and some of our strategies that we’ve used to convince the leadership that this is really necessary.
The Setting [113:31] So this is Chicago, it’s where the University of Chicago is. This is our new hospital here. This tiny thing here is our pediatric hospital and you can see we’re about 6 or 7 miles south of the city. Barrack Obama’s house is a couple blocks north of the hospital. But the interesting thing about the medical center is that the medical school is here, the hospital is here, and campus is here, and there aren’t a ton of places where you have that very concentrated set of opportunities all in one place. You have the med students and you have the computer science department and you have the business school and you have social sciences. Everything is right there, which makes for a lot of opportunities for collaboration. I think it’s somewhat of a common model, the physicians are employed by the university but they work at the hospital and there’s one dean that’s over everything. There is one dean for the hospital and the med school and for the biological sciences and what this allows us to do is to have one point of leadership when it comes to negotiating a lot of these complex issues of who is going to pay for what. And so, I think as I was talking to people this week, I find that this is not necessarily common. Often, there’s a president of the hospital or a CEO and then there’s a dean of the medical school and there’s no reporting structure there. And so, there often can be a lot of stress around budgeting and how the priorities are going to be set.
UChicago’s Situation in 2011 [115:17] So back in 2011, so not so long ago, there were very few centralized resources for researchers and there was a growing need and there was a lot of rambling from the faculty that they wanted to be able to get clinical data to do research. So there was no data warehouse, there was no way to get this kind of data. There was not a lot of bioinformatics resources except caught up in these silos of these groups that were supposed to be working with everybody but they weren’t. The storage and backup was mainly through the university or the hospital and it wasn’t a great service and it wasn’t enough to do the kind of research people wanted to do. There was High Performance Computing, again it was through the university. And biological sciences is one part of this giant university. And so, when the priorities come to partitioning out storage and High Performance Computing, the researchers would often lose out. There wasn’t a lot of opportunity for people to develop applications to do clinical trials and the Department of Medicine was the Defacto group that people would go to, to help them build their applications to help conduct trials. And there were no educational opportunities in informatics.
So the dean saw this need and the office of our Chief Research Informatics officer was established in 2011. It started with just six people. It was a $10 million 3-‐year investment. And with that investment, in very short order, there was on-‐premises storage set up that was HIPAA secure, there was High Performance Computing cluster put in place, a very small bioinformatics team built by hiring a director and bringing in some bioinformaticians. The very first research data warehouse and still the only data warehouse at the institution was built by the folks on this team. There was a group set up to do application development and I’m not going to talk about governance today but this was a critical part of our success – was the ability to set up some governance and get faculty members and other committees involved to govern the data, and then some informatics educational opportunities.
Center for Research Informatics 2015 [117:36] So that was then and this is our center now. So we were over 40 people, we’ve grown really fast. I was thinking about that during the talk yesterday where he was talking about the reasons you go out of business and one of them was growing too fast. But I think we’ve been responding to the need quite well and we have more work than we can handle right now. So it’s a great, well-‐engaged group of people and I’ll talk about some of the strengths.
Center for Research Informatics – 2015 [118:03] We split our work into five different channels of operation and I’m not going to read through all these, but just to quickly tell you what we do, because I think it’s important to understand that the reason that I think this works so well is that everybody is in the same shop. So we have an administrative group that does communications and we have project managers in that group. We also do all the financials and budgeting. We have an IT infrastructure group and this is becoming more and more unusual, right? To have your own IT group within the center where we run our own storage, our own backup, our own VM firm, our own High Performance Computing. We have a bioinformatics team. I have eight Ph.D. bioinformaticians and all they do all day long is analyze NextGen sequencing data using Open Source pipelines they have developed. We now have a very robust research data warehouse and data requests come in much faster than we can fill them. There’s an incredible need for clinical data for research. And then we have an applications group and the director of our applications group, Bryan, is here today, and this
group builds applications for clinical trials, and I’ll talk a little bit about some of the work that he does in a little bit.
Storage and Backup [119:22] So how many of you guys are totally happy with your storage and back-‐up systems that you have? Well one guy. So, we knew that there was this need for storage and backup and everybody knew the pain of having hard drives across the labs and having to walk into the office and you’d see a hard drive sitting there and you’d say “What’s on there? Oh, my study data, my patient study data are on there.” So there were all sorts of problems with storage. And so setting up a mountable set of shares for people to put their data in where you know it’s HIPAA secure, you know it’s monitored, you know it’s audited, has been really a real boon for us. And we have over 140 groups now which means a majority of our research lab is now using our storage space. We have over 600 Terabytes of storage but our data center actually is a 1.3 Petabyte data center just for our storage and we have plans to grow that as needed. We have some groups
that will ask for 70 Terabytes of storage. We have one project now that they’ve projected 300 Terabytes of storage needed for this genomics project. So it’s a lot of storage.
Bioinformatics Publication-‐ready Results [120:40] Our bioinformatics team, one of the reasons they’re successful, is that everybody on that team is a Ph.D. level scientists who on their own could be running a lab somewhere and they all see themselves as collaborators in the projects. I encourage them to meet with the faculty, to sit with the faculty, and be part of their projects from the planning phase, to applying for the grant, to running the project, and to actually doing the analysis. And this is something that I think is pretty unique among groups. When we give results back to researchers, we don’t give out Excel files or PowerPoints. We give back results that are publication ready. Figures are all formatted, with references, methods, and everything done. And this is obviously, as you can imagine, pretty well received from the folks that we work with. We have more business than we can handle with this group. Out of the eight informaticians, they have an average of five projects each right now.
REDCap Usage continues to soar [121:41] Who are REDCap users here? I thought there would be more. So REDCap is a free open source system that allows users to build forms. In its simplest incarnation, it allows users to build forms and collect data using those forms. It’s very straightforward to make that system HIPAA-‐compliant and the controls that are in the software are already HIPAA-‐compliant and it has that capability of being 21 CFR Part 11 compliant. So it’s a really nice system for researchers to build their own sets of intake data. And our REDCap use has just exploded. We have over a thousand projects in there now and often times we’ll have a researcher come to us to say, “Oh, I need this very complex web-‐based system for entering patient information and all this.” And we say, “If you try REDCap…” and the next you know, one of their students is building a REDCap database. It solves their problem. And it’s pretty easy to get data out of REDCap and then repurpose that data to do something else with it. I didn’t like REDCap when I first got there because I thought it was like Survey Monkey for doctors. But given how much it’s allowed us to concentrate on the more complex projects, I’m a fan now.
Clinical Research Data Warehouse [122:57] Our CRDW is probably the one thing people are most interested in at this meeting. The group just got in there and got their hands dirty and built it. We didn’t have a lot of top-‐down consulting on how to build the warehouse, and Bryan and another data warehouse guy in our center basically just started using the data sources that they already had, like centricity, access to Clarity tables, and started building the warehouse off of that. And it’s been a huge, huge success. Our average wait time for a data request now is eight weeks or more, six to eight weeks, because there is so much need for the ability to get clean, well-‐annotated data out of the warehouse. And it’s really surprised me how much – you know, we started charging for request about a year ago a hundred bucks an hour, and I’m like, that’s going to kill our business. Nope. Still, it’s packed just as much as we can we handle. And regarding the hospitals working on their Enterprise Data Warehouse, we have the opportunity right now to try to work with the hospital to build the warehouse that’s going to feed off the same sources as ours. I’m taking leadership roles in the governance on both the research and hospital sides. So I think we have an opportunity to bridge those two things together where I think that a lot of institutions, the hospital just sort of drives ahead and does its own thing.
Poll Question #2 On a scale of 1 to 5, how supportive of research is your institution? [124:36] So this is our poll question #2. On a scale from 1 to 5, how supportive of research is your institution? And supportive here can mean whatever you want it to be. [Facilitator] Fantastic. Again, the question, on a scale from 1 to 5, how supportive of research is your institution? 1 – not at all supportive; 2 – somewhat supportive; 3 – moderately supportive; 4 – supportive; 5 – very supportive; 6 – unsure or not applicable. Just a couple more seconds and we’ll display the results. [Samuel Volchenboum, MD, PhD] I bet it’s just going to be a bell curve. Very supportive. Wow. I’m surprised about that. Come on, moderately supportive. Let’s go.
So I think that’s interesting. So to me supportive comes in different shapes and forms. It could be everything from the ability of someone like me to spend 90 percent of my time, even though I’m a clinician running the center, offering up all the faculty time to do governance or in the form of money to support the mission. And one thing that I’m finding more and more is that support comes in the form of giving the hospital the directive to help the researchers and to give them the mandate to help the researchers do their clinical research and that’s something that is becoming more and more important.
Data Shopping [126:14] So we had a lot of problems with data shopping at our institution. We had lots of groups. It was all about like I know a guy and I have a guy who’s got data, he’d meet some guy to give you data and it was really causing a lot of trouble for a couple of obvious reasons – one is you had multiple groups trying to fulfill the same data requests, you had people getting data from multiple sources. And so, they would have two data sets that they thought were going to be the same and then the data were different because they are being fed by different sources. And then it’s just the economics of a hospital having to do this and to waste their time.
You know, before we put in some of these changes in place that I’ll tell you about, we had people that would request data from us and our analyst would spend 10, 15, 20 hours doing the data request and they have never even picked it up. They say, “Oh yeah. No, That wasn’t…I’m not interested in that project.” So, until you have some sort of ownership yourself in the process, it’s really hard to engage people.
Data Shopping [127:17] And so, this group has really done a great job and it’s been led a lot by Sarah who is in this room somewhere as well. I just saw Sarah walked in. And what this group did is it formed what we’ll be calling an analytics core. And what the analytics core did is take this problem where you have a million requests going to all different analysts and it creates a single point of intake. And I was really doubtful. When we built the form, put it up, and started collecting data requests, I was like, everybody will still pick up the phone and call their guy and get their data. But it’s been a remarkable success.
Analytics Core [127:54] So this group, the 13 groups that I’ve put on the page before, they all meet every Friday, they meet for an hour and they go over all the requests from the week and they triage them out to the different groups. And so it’s fully transparent. Every group knows what every other group is working on and it allows for a level of oversight and accountability that we just never had before. And I think this was real cool to be able to pull this off because if I had looked at that list beforehand, I would have said, you know, half of those folks aren’t even going to talk to you. But they’ve all been ready and willing. And I think part of the reason is that there’s time so we can get at a certain data set that they don’t have access to or another group has a certain set of financial data that we don’t have access to and there’s a lot of sharing that can go into that.
Major 2015 Initiatives [128:43] Alright. So what’s on tap for our next year as we go forward, we’re going to try to release a self-‐service de-‐identified data portal. Does anybody here have something that resembles that? Because some people hear that and they think we’re not. Do you guys have one where you can go do a query and get the data out immediately? Yeah. We think we should have it. We think you should be able to go and enter a query and pull your data request. Of course, the lawyers that we start with know and then we have to sort of pull them back but I’ve been surprised that we’re still getting a lot of pushback from IRB and places like that. This is not as straightforward as I thought it was going to be. I thought, while the data are de-‐identified, or even if they are coded, you should still be able to give them to our researchers without them having to go through a long IRB process. But we’re getting there. What the applications team has done is that they have developed a whole new interface for doing cohort discovery. So we have I2B2 but Bryan’s team built a much shinier interface and connected to that is going to be a full text search of all of our inpatient notes, all of our discharge summaries and admission, H&P, and those are all going to be used for cohort discovery and we hope to be able to export the data for researchers; although probably not the clinic notes.
One of the big things we’re working on this year is we have several big projects doing national sample enrollment and tracking. And this is another business I never thought we would be in. So several groups have come to us and said, “We want to do a multi-‐center trial, we want to collect data from all over the country, we want to enroll patients, we want to send samples all over the country, and we want you guys to build the system that’s going to keep all that in order.” And so at first I was like, you wouldn’t even want to be part of that. I mean it just sounds like a nightmare. But what we’ve actually done is work with several great groups, one out of data farmer, one with the March of Dimes, and then one with our local surgery group, and we’re putting together three very large trials where people will enroll at various sites and tissue or DNA or blood will all get sent to a centralized place and then the platform that we’re building will let you track the sample through the system, reconcile the samples with the patients. And then for one of the projects, we even built a pretty complex molecular pathology portal, where people can actually follow the variants into the reporting through the system and they could tell you with any single time, here is a sample that’s in this phase of analysis, here’s a sample in this phase of analysis, and it’s going to be used for this next 5-‐year trial. So this has been a really surprising opportunity for us and I’m glad that we’re doing it. Again, the lawyers, right? So a lot of times the hurdles are less technological and more medical-‐legal but I think we’re getting there and I think we’re starting to chip away at some of the problems that we’ve had. The last thing on here is especially relevant. So about a year ago, we’ve implemented IBM’s Cognos analytic stack and we got Teaser Box and we got data stage and Cognos, spent a lot of money on it, and we did it because of some pressure we had from the hospital to play along with the games that they were playing. But we also saw this opportunity to have these analytics tools that not many groups had. So we have a large set of licenses to give out to researchers and we’re just spinning this up now to provide a layer of analytics for researchers. One of the biggest problems that I noticed now, and I would be interested to hear other stories about this, is that we provide lots of data for clinical research and we give the researcher all these CSV files with tons of data and the researcher doesn’t know what to do with it because they don’t have an analyst, they don’t have a statistician. And so, we end up having to try to figure out how we’re going to do the analysis for them. And so, this is my mission now with our groups to try to build up this army of modelers and statisticians that we have, so that we can fill in that gap and help the researchers use the data – because data are getting more complex and also as you see on your points there, when it comes to data, researchers can hurt themselves if they’re not careful.
Our Metrics of Success [133:16] So what are our metrics of success? Well we want to obviously lower barriers to research and the way that we track ourselves is by how many faculty are using us, how many grants are submitted or awarded, papers published, our faculty being recruited based on the services that they think they are going to get, and the kind of educational opportunities we’re providing. So these are all, especially the middle three, really hard. Like finding out when we’re on grants has been a pain. If anybody has a better solution, I’d love to hear it because right now anybody can request the facility’s page from us. If they request a letter, we can track that, but if they just write in their grant that storage will be done by the CRM, we don’t know that but yeah, that’s an impact that we’re having and we’re not going to know about it until they get the grant or they need help later. With grants awarded, we do a little bit better tracking that. With papers published, we had to do this ridiculous exercise last year where we PubMeddit all the faculty that we give data to and we sent them a list of all their publications and we said, “Please just mark the ones that are related to data warehousing” and 25 percent of them took the time to respond and let us know. So there’s got to be a better way to track these things because I know that our impact is
far and wide. And then we’re doing a lot of educational initiatives now and I’m happy to talk about that later, if anybody is interested.
Enterprise-‐wide impact [134:38] So, we track the number of users we hit last year: over a thousand. And I was pretty surprised that it was that many people that we had touched in some way, whether it was through storage, HPC, or RVMs. And I expect this number to grow. And I think the reason is I tell everybody to see themselves as collaborators in these projects. I don’t want to sell ourselves as service lines or as a core. People love to use those terms but I tell everybody, when you go meet with the faculty member, you are a collaborator on their project, and more and more were getting put on the papers, they’re getting written in the grants, and I think that is having a real impact.
Lessons Learned [135:22] So what are the lessons learned? I think developing your brand is really important. And I’m sure many of you have these same stories. We have like 40 groups on campus that have the word ‘computing’ somewhere in their title and have ‘informatics’ somewhere and people are always like, “Oh I thought you were this group, I thought you were that group or can you fix my EPIC password?” And it’s fine if somebody calls us and asks us to change their EPIC password because we can tell them to go here. But what I worry about is when all these people call these other groups and they say, “I need NextGen Sequencing help” and the other groups were like “What does that mean?” And they hang up on them. So, I can tell you that developing a brand for your group with logos and with communications is important. I give all these roadshows about the work that we do. It is really important and helps establish your center as a place where people think about it and they know that it’s you and I think that’s really helped. You have to have data governance in place. Obviously, we’re doing a lot of catchup right now because we’re putting in some new committees to address some security issues and it’s been very difficult to backfill the governance areas where we needed help. And a lot of these, especially issues with commercialization, using data from the hospital, it’s come up time and time again now and we don’t have the greatest facility to understand what to do with those faculty requests.
Never underestimate researchers’ ability to hurt themselves with data and this comes in many formats. It could come on the format of somebody putting all the data on their laptop and not having it encrypted and losing their laptop. I mean that’s ridiculous, right? But we still have people that refuse to encrypt their devices and then they get stolen. Then we have to do a big audit and a big forensics and pay for it. The other way they hurt themselves with data is that there’s more and more tools around that anybody can load them up and push a button and get an analysis done. And if you’re not trained in some of the science, you can actually do an analysis and you can really screw yourself with some of the results because you wouldn’t understand how to get those results or what algorithms were used. So having the right kind of statisticians and experts to meet with people has been really important. And being aware that the researchers will injure themselves if you’re not careful. One of the best things I did in the last year and a half was hire somebody with an MBA to be the deputy director and that’s been incredible – because before that we were run well but we were sort of run like any other academic shop and then all of the sudden we were running like a startup. Doctors don’t know how to do anything with money, let alone run a group like this. And so, having somebody with an MBA background who was very aggressive about the budgets and very aggressive about how we were going to spend was really important. We pay for it though. The leaders in our group, they get a decent salary for academic medicine and I know that we’re paying for it because when I go to other groups within the university and I tell them what we’re paying our leadership, they can’t believe it. They can’t believe it’s oftentimes 50 percent more than they are paying people that are in their groups. But you have to; otherwise, the person is going to go to Goldman Sachs, they’re not going to come work for you if they can go elsewhere and make a lot more money. I think those days of “oh I just want to work in medicine and help people,” are over. I think you have people that come out of grad school, come out of working at a company, and they want to make more money and you have to pay them. I have weekly meetings. I try to have weekly meetings with all my direct reports and we have a weekly executive meeting. I can’t stress this enough. Very often, I hear about CIOs or leaders that are just not present and they just let all the little channels of operation run together by putting everybody in the room and having these leadership meetings has been really helpful to us to understand how we can do things across our lines. And I spend a lot of time trying to figure out how to get to the dean, how to make us visible, how to go to a researcher and say, “Hey, did you like that paper that just came out that we helped you with?” “Hey, would you mind sending the dean an email and let the dean know that we really helped you guys.” I spend a lot of time with the CFO because budget season lands on his desk and I want him to think favorably of us. I spend a lot of time with the various department chairs. It really pays off. And so, having somebody in your group who’s got a cache within your organization to go around and to meet with all the leaders and do that has been really important for us.
I think that’s it. So we can have questions and answers. [Facilitator] We’ll go over to Dan for analytic insights and questions.
Analytic Insights / Questions & Answers [140:04] [Dan Hopkins] And I will have questions as well. What I wanted to share was the most applause came from when you discussed an analytic core and the weekly source of data in a singular state. And the other one interestingly enough was around data governance. What I wanted to share from an insightful standpoint is that organizations that are very supportive of the individuals in the room are the most worried. So, obviously they are thinking about their commitment to that organizational thrust. [Samuel Volchenboum, MD Ph.D.] I saw a lot of nodding when I brought up governance but nobody wants to talk about governance after lunch.
[Dan Hopkins] Okay. So now we’ll get to the questions. [Samuel Volchenboum, MD Ph.D.] And I saw a lot of people nodding when I said, “Do you have this or do you have that?” And so, I hope people will offer up their observations when it’s handy.
QUESTIONS ANSWERS Is this progress achievable without a dedicated center for informatics or similar group?
Of course things are achievable if you put enough money behind them. But what I think we found is that if that $10 million had gone to support the latest and greatest genomics researcher because they were going to help everybody else, they would have set up their group, it would have worked for a couple years, but inevitably they would have turned inward and not been successful. I think having a center where all the groups can communicate and work together – if my bioinformatics guy needs to run a pipeline and he does not have enough space on the HPC, he writes an email to our systems guy and they give them more space on the HPC. I mean you can’t do that when it’s with the university or if you’re at the hospital’s mercy. So of course it can be done other ways but I think this model has been very good for us.
How do you avoid multiple EDWs?
That’s a great question. A couple weeks ago, at one of our data governance council meetings, we learned about the Cogito Warehouse that was being implemented. I’m at the data governance council meeting and we’re learning about the upcoming Cogito rollout and it was the first time that some of us didn’t even know that. And so, communications has to be the number one thing. And so, what we’ve done is on our data governance councils, we’ve tried to get representation from every single walk of life that we could find, whether that was the health information management or whether it was patient services, nursing, etc.. Try to get somebody from every group so that when you make a decision, one of the people can raise their hand and say, “Oh wait a minute, we’re already doing that or we know who’s doing that or that’s already been budgeted for next year.” There can’t be enough communication. And what we found in the hospital is it’s run very corporate, all very vertical, and so there’s not a lot of communication going on between the channels, and having a governance council overall, that can really help.
The other thing about multiple data warehouses is that it is okay to have multiple ways to set up your data warehouse, as long as you are pulling from the same source and you provide validated data sets that are going to be the same. And so, having a stamp of approval for your data is what’s important on having clean sources of data.
What is the role of a university medical center in engaging other providers and driving standardized practices in the region?
Yeah, so I think we can serve the best function by – and I’ll just think about areas in my own interest – pediatric oncology. We do best by making sure that the community understands exactly what we’re doing and why for the patient. So all patients, for instance with cancer, almost all kids go on clinical trials. And when the kids are done with their trial, they have a very regimented path to follow that’s set up by the children’s oncology group or whatever consortium built the trial. And if you don’t communicate that well and if you don’t have a lot of back and forth between the local docs, the kids just have terrible follow-‐up. And so, it’s in your best interest to try to communicate this out. We don’t have a lot of good ways to do that yet and I think we have to figure out how we’re going to do that better because FACTS is still state of the art, right? So we have to figure out better ways to communicate.
How do you get scientists to work with software engineers?
So you either need to have a software engineer that’s very personable and understands the science and can talk the science language. And so, we have a few of those, like Bryan here. Or you needed a translator, somebody who knows just enough of both. I mean I think myself was a decent example. So I know enough programming to write a script here and there but I’m not ever going to be a developer and I know some science and I know some medicine and oftentimes I find that I am like the one in the room that is saying to one group and to the other group, “you know, here, the computer scientist can do this for you. This is the biologic problem. Here’s how you’re going to have to face this.” And having those translator folks is really important because it is almost like somebody French trying to speak to somebody Japanese. You have to have somebody in the middle to translate often or have somebody, like I said, really personable which isn’t always big in the computer science industry.
Any insight into appropriate staffing or resource allocation in this area? And then what percentage of your resources are dedicated to integration in your EDW?
Yeah, so I’ll take the second one first because it’s easy, because it’s like nothing. So we’re really trying to get into the EDW, but again, like that very first slide, if we go to the hospital and say, “Listen, we have this really great tool that’s going to allow people to write these
interesting reports on genomic data for patients, let’s push them into EPIC and have them there as notes,” they’re like, “Wow, that’s not part of our annual operating plan and we don’t have budget for that.” So we’re really not doing well on that front. And this is where the top-‐down leadership comes into place because if the dean understands that that’s what’s going to drive the hospital forward, then that information can trickle down and you can have more support for that. So we don’t spend – So we have very little funds allocated other than the time that I and my colleagues in my center spend with the hospital folks helping them. But as far as projects, we don’t have a lot allocated for that. And the first one was about staffing and resources? And the question was what? How much do you need? Yeah, so most of our budget is FTEs. We have the budget to keep our storage in our HPC going. But by far, most of our budget is the FTEs that we have and we never have enough people to do the work we need to do. For the data warehouse as an example, we have folks that are really good at writing SQL queries and really good at getting the data out but we also have to staff up with people that understand how to go to a clinician or a researcher and know what a central line is and know what it means to try to construct a query that finds all the patients through the central lines. And so, having people fill in those little roles is really important in taking through the whole process. What it leads to though is that you don’t become very redundant. And so then you always have this case where, “Man, if that person left, he wouldn’t be able to do X.” And so I think you have to be careful when it comes to staffing up. That if you get too specialized, you could lead yourself in trouble. It’s probably not what they were asking but okay.
What are your biggest challenges?
Our biggest challenge right now is actually working with the EPIC side of things and trying to share the burden of these research projects with the clinical team. I’ll give you a good example. So, we have a couple teams that have come to us to build systems to take data into the point of care for the patient. The patient comes in and goes on a trial. The research nurse comes in, opens up the form on their iPad or their computer, enters the patient into the form and then they become registered on the trial. We would
love to have those things flagged in EPIC or put into EPIC or have some sort of way to communicate between the systems. And the technology, I think, is rather straightforward from what I’m told, but we are not given the prioritization to do that. So that’s one of my biggest challenges now. So I’m spending a lot of time courting the CIO and the CMO and spending time with them and trying to help them understand that these things are important. So I have a lot of standing meetings with all these guys trying to help bring them into the phone.
With regard to data requests, have you defined the source of truth?
That’s a great question. So one of the things our data governance council is doing now is figuring out ways that we can define the source of truth better. Right now, the source of truth is whatever our guys have found to be the most reliable source of data. What we are going to do and what we’re doing is we’re having – I think this is actually sort of unique – every data source is going to be give a score of zero until proven otherwise, and then we will have the people in who own the data source, we’ll have the stakeholders in, we’ll do validation of the source. And once the governance council finds that it’s a validated source of data, we’ll get our stamp of approval and then have that all be transparent so people can know that, hey, the admit time in the ED, that’s actually been validated as the real admit time. Whereas I, as a physician, know that the patient gets a room and then they sit in the ED for another two hours waiting to get for transport to come. So until you validate what the numbers mean, you’re not going to know what the source of truth is. But that is impossible to do without data governance and buy-‐in from up top to take the resources to try to validate the sources.
There was a second question on that one, so it is a follow-‐up. Did the various keepers of the data even know what data others had?
Yeah. So this goes back to the communication part of me. It’s been dismal, the amount of communication. I mean I’ll give you an example. So as we were mastering our patient data, we found that there were a lot of test patients in the data – because a lot of test patients are created for, you know, the patient comes in in the emergency room, they get a unique patient ID. There are all sorts of patients created in the record that aren’t appropriate to be using for a research. And when we started interviewing groups to understand who’s making the test patients, nobody knew who else was doing this, and it ended up that a lot of groups had the capability to do it but nobody was in control of the process. But what we have done then is
we’ve gotten everybody in the same room and we could say, “Alright, let’s address the issue of test patients. Let’s address the issue of how do we define length of stay.” And by doing all that, bringing all the people together, we increased the communication. But yeah, no, it’s still a lot of who you know when it comes to getting data out and a lot of times people don’t know who those people are.
What have you found to be the best practices for keeping a clean set of data, no duplicates?
Well you have to have a lot of good quality procedures, you have to have a lot of good QC. And one of the things our data warehouse group has done really well, I think, is that they really pride themselves on giving out good quality data, and part of that is obviously making sure that your data sources aren’t duplicated or they aren’t too sparse, that you don’t have the same patients in there more than once. And each of those has different ways of validation. But I think the part that’s been important to me is that I can tell that our data guys are super proud of the data that they put out. And we’ve had researchers bang it on the door that they need the data today and I’ve had my data warehouse guys say, “Can’t have it until tomorrow. We have a whole set of validation checks to do before we can give that data out.” And I think that having somebody who is dedicated to the data, who loves the data as an asset, I think, is really important. So I think that’s what’s really going to help.
[Dan Hopkins] Thank you so much, Sam. This session is great.
Choose one thing… [152:49] Everyone, make sure you take a minute to fill out the one thing you would do differently at the bottom of your lessons learned sheet. If you don’t have a lesson learned sheet in front of you, our ushers have extras in the back and you can pick one up in the back. The next session starts at 02:20. [Samuel Volchenboum, MD, Ph.D.] I brought a whole bunch of informational tri-‐folds about our group. If anybody wants to – I don’t want to take them on the plane home because they’re heavy. So I’ll just leave some of these up on the front table if anybody wants them, or you can ask me for them.
