System Design for Data Scientists and Engineers in Healthcare and Life Sciences

[0:01]Hello everyone in this video I'll be discussing what is system design and various factors associated with it for the healthcare and Life Sciences industry system design is one of the most important aspects for any software solution and is probably asked in every interview be It software engineer or machine learning engineer in this document I'll be focusing on system design for the healthcare and Life Sciences industry so what is system design system design is the process of defining the architecture components modules interfaces and data for a system to satisfy specified business requirements it involves creating a detailed blueprint that serves as a foundation for the construction of the

[0:54]Actual system the goal of system design is to address the functional and non-functional requirements of the system ensuring it is efficient scalable maintainable and meets the needs of its users in the past decade the integration of data science and engineering has changed the way Healthcare Systems operate with the vast amount of data generated daily in these industries it has become almost imperative to design robust systems that can efficiently manage analyze and derive sites from this data so here you can see a typical uh diagram for machine learning uh Healthcare Systems so these are the four stages uh and you can see that uh each stage uh is dependent on each other

[1:51]And these are the main areas for any system design in machine learning so designing so as I said designing a machine Learning System is a imper iterative process so there are generally four main components of this process with these are project setup data pipeline modeling and training and serving the output from one step might be used as the input for the next as you can see in the diagram so here are some of the scenarios uh which are involved so let's say you are examining the a aable data for a use case and you realize that it's important to get the data needed to solve the problem you

[2:38]Previously defined so you have to frame the problem differently then another thing is after training you realize that you need more data to make your model more efficient so you need to re uh relabel your data or get some more data then a further after after serving the model to the users you realize that the way the product is being used is very different from what we actually thought and so you uh have to update your model from whatever training decisions you have made so you can see that designing a system for machine learning is uh significantly different from uh designing a non-machine learning system so now uh let's go to the

[3:36]Different categories of data which are there in the healthcare and Life Sciences system so we have electronic health records ehrs so these contains patient specific information such as medical history medications allergies immunization records then we have the medical imaging data this category includes various types of medical images such as x-rays MRIs CT scans and ultrasounds so they play a vital role in diagnosis and treatment planning then we have the genomic data which involves genetic makeup like the DNA sequences variations and mutations followed by we have uh clinical trial data data from clinical trials includes information about design protocols outcomes and safety of experimental treatments then we have the

[4:36]Laboratory data data which encompasses results from various diagnostic tests blood tests pathology reports and other laboratory analysis further we have Public Health Data this involves population level data used for monitoring any public health including disease prevalence vaccinations Etc then we have the administrative and billing data these includes information related to healthcare admins insurance claims and billing then there is data which patients report themselves directly those are symptoms lifestyle habits treatment adherance and the patient reports then we have the variable data which is gathered by different variable devices like like the remote Technologies Healthcare professionals can access the data then we have the healthcare provider data which is about the

[5:40]Credentials training and work history and then there's the health insurance data which is involves related to coverage claims and reimbursement process which is also plays a crucial role so now what are the challenges in healthcare data management so Healthcare data presents unique challenges as compared to other Industries because of its volume and sensitivity so the patient data is extremely sensitive because it's personal information for the patient and you need to handle it very carefully uh as we see we have seen above the sources of data ranges from Healthcare records to variable devices and this data is structured and unstructured in nature so we need complex data

[6:33]Analysis uh in order to derive traits and make it usable for our machine learning model and as I said extreme care must be taken regarding the privacy and security of this data so here are the four main uh challenges which we have first one is privacy and security so strict regulations like Hippa mandate robust data protection so data must not be leaked uh anywhere because it's patient sensitive and it's personal information for the patient then there is data diversity because there are lot of data sources which we see above uh the data is in inconsistent format and it requires data cleaning then there's data quality and integration so not all data

[7:25]Is good so you there's incomplete data so which can cause our systems to uh reduce the efficiency so we need to increase the quality of data and the scalability and performance so the systems which handle massive data uh need to efficiently deliver timely insights using complex algorithms so here are some of the system design considerations so now keeping all the these challenges in mind we have to come up with a system that uh provides solution for either the healthcare or and the life sciences industry so here are the basic uh considerations that you must have for Designing any sub system so the data governance and security the system must

[8:24]Have a module that implements role based access control and encrypt mechanisms so encryption and decryption of data is very important for such systems uh it is called Data masking in terms of the healthcare and Life Sciences industry so you can there are various algorithms uh which you can use to encrypt and decrypt this data then uh regulatory requirements must be considered uh in order to you know use this data store and access it then there is data inje and integration module so we already discussed how complex the data is so as a result any Healthcare System should build flexible data pipelines so uh we discussed above that machine Learning

[9:23]System design is a iterative process so we might need to you know change the data after the system is built or we might need to relabel the data retrain the data so the pipe data pipeline should be flexible in nature and uh we must consider data standardizations and cleaning processes so the another important part is the analytics infrastructure so every machine Learning System should have a strong analytics component so we need tools uh such as Cloud platforms and data Frameworks like Sparks and tensor flow so this system should be easily scalable to handle large data sets now here you can see a generic machine Learning System design uh

[10:24]Diagram for any Healthcare or Life Sciences industry or any machine learning application for that matter so here user is any application whether it's a mobile app or a website which the user is accessing the user is making request to the application so here application is the core product or the business logic which uh you are serving which you have made so in between sits the load balancer which uh splits the incoming uh requests and redirects them to the appropriate server so this business logic will sit in pool of servers so it won't be a one it won't be a single server because if that server fails then

[11:15]Your application will go down and users cannot access it so this load balancer will help to direct the request to a particular server so that not single server is loaded at a time so this application uh communicates with the API and communication layer so API and communication layer uh is uh like a interface which uh talks which sends the requests from the application and talks to the ml models and analytics mod system so this is a subsystem here ml models and analytics so before coming to that we have data source or data leak so it can have uh all can have the data which is required

[12:09]For your system so this data is inconsistent unstructured incomplete and so you need data processing data cleaning so Technologies like uh streaming data uh we have a Pachi Kafka for that which can be used to stream it then there's ml mod so after processing and cleaning this data uh this data is sent to the ml models and analytics system so you can uh build your machine learning models train here and serve those serve these models so after that the data is encrypted using encryption algorithms and stored in the data storage so you can select SQL or nosql databases based on uh your application needs so it can

[13:09]Be Cassandra redis which is no SQL and you can use my SQL or post G SQL for your application and then data is decrypted uh as per the requirements and send back to the user based on uh as per the needs so yeah so this is a general generic system design diagram I would say for any uh machine learning system which involves data analysis and cleaning training and training a machine learning model so now let's come to uh case study let's say we are doing realtime patient monitoring system system and which collects data from variable devices and medical devices which are there at the bed side so this

[14:07]System utilizes AI algorithms to detect early signs of patient detoriation and alert healthcare providers so you can see here the broad system design for this use case so we have the data processing module here which involves different sub modules that is visualization then knowledge extraction context management and raw data sensing then in the cloud we have data processing tasks Management ealth Services then the variables device variable devices are connected to a Gateway and they provide the data which is used here and then as I said security is extremely important so we have security mechanisms in place uh for the data so to conclude I would say that

[15:07]Designing effective systems for data scientist and engineers in the healthcare requires careful consideration of scalability interoperability and data quality uh by addressing these key aspects and following best practices we can design a robust Healthcare application that has full potential to improve patient outcomes enhance operational efficiency and drive innovation in healthcare delivery so yeah that was a brief overview of how system design is done in healthcare and Life Sciences industry uh thank you so much