Clinical Risk Prediction & EHR NLP — Cognizant

A patient readmission model over 2M+ EHR records reaching 0.87 AUC-ROC, a clinical NLP pipeline extracting structure from physician notes, and the deployment and explainability work that got clinicians to actually use it — under HIPAA constraints throughout.

Machine Learning Engineer · Cognizant, India · Jan 2019 – Oct 2020

← All projects What I built Architecture Tech specs Interview questions Glossary

What I built

Patient readmission prediction — an XGBoost model over structured EHR data from 2M+ patient records, achieving 0.87 AUC-ROC and letting care teams identify high-risk patients before discharge.
Scalable PySpark pipelines on Databricks — processing laboratory results, vital signs, and ICD-10 diagnosis data, reducing feature generation time by 60% for daily model refreshes.
Clinical NLP pipeline — spaCy and BioBERT extracting symptoms, medications, and comorbidities from unstructured physician notes, improving structured clinical data coverage by 35%.
Drift and data quality monitoring — Evidently AI detecting distribution changes early and supporting automated, HIPAA-compliant retraining.
Model deployment — REST APIs via FastAPI, Docker, and AWS SageMaker, integrated with hospital EHR platforms through HL7/FHIR interfaces for real-time risk scoring.
Clinician-facing explainability — SHAP-based dashboards highlighting key patient risk factors, improving transparency and increasing physician adoption across pilot deployments.
Model lifecycle automation — MLflow and Jenkins automating experiment tracking, versioning, and deployment, reducing release cycles from two weeks to three days.
Clinical A/B evaluation — partnered with clinical and analytics teams to evaluate model-guided interventions, contributing to an 18% reduction in 30-day readmissions during the pilot.

Architecture

Two ingestion paths — structured and unstructured — converge into one feature set. The explainability layer is what made the output usable by clinicians rather than merely accurate.

Technical specifications

Dataset	Structured EHR data from 2M+ patient records
Model	XGBoost, 0.87 AUC-ROC on 30-day readmission prediction
Data processing	PySpark on Databricks — labs, vitals, ICD-10 diagnoses; 60% faster feature generation
Clinical NLP	spaCy pipeline with BioBERT for symptoms, medications, comorbidities; +35% structured coverage
Monitoring	Evidently AI for drift and data quality, supporting HIPAA-compliant automated retraining
Serving	FastAPI, Docker, AWS SageMaker endpoints
Integration	Hospital EHR platforms via HL7/FHIR interfaces, real-time risk scoring
Explainability	SHAP-based clinician-facing dashboards of patient risk factors
Lifecycle	MLflow experiment tracking and versioning, Jenkins automation — release cycle from two weeks to three days
Clinical outcome	18% reduction in 30-day readmissions during the pilot A/B evaluation

Interview questions 136 questions

Healthcare ML interviews test three things beyond modelling: whether you understand that a prediction has to change a clinical decision to be worth anything, whether you respect the privacy constraints, and whether you know why clinicians reject accurate models. Terms are defined in the glossary.

1 · Clinical problem framing (12)

Why does 30-day readmission matter to a hospital?It is a quality measure and, in many systems, financially penalised — so the model attaches to an existing incentive rather than creating a new one.
What exactly is the prediction target, and when is it made?
Why predict before discharge rather than at admission?
What intervention was the prediction supposed to trigger?
Why is a model that predicts accurately but changes no decision worthless here?
How did you define readmission — all-cause, or condition-specific?
How did you handle planned readmissions that should not count?
What is the prediction horizon and why 30 days rather than 7 or 90?
What were the exclusion criteria for the cohort?
Who were the users, and what did their workflow look like before?
How did you decide how many patients could realistically be flagged per day?
What would have made this project a failure even with good model metrics?

2 · Modelling with XGBoost (12)

Why XGBoost rather than logistic regression or a neural network?
How does gradient boosting actually work, in your own words?
What is the difference between bagging and boosting?
Which hyperparameters mattered most and how did you tune them?
What does max_depth control and what happens when it is too high?
How did early stopping work in your setup, and on which set?On a validation set separate from test — using test for early stopping quietly leaks and inflates the reported number.
How does XGBoost handle missing values natively, and why does that matter for EHR data?
What is regularisation in XGBoost and which terms did you use?
How did you handle class imbalance in readmission prediction?
Did you use scale_pos_weight, and what does it do?
How did you check the model was not just learning length of stay?
What would you try next to improve beyond 0.87 AUC?

3 · Metrics and validation (12)

What does an AUC-ROC of 0.87 actually mean?Given one readmitted and one non-readmitted patient at random, the model ranks the readmitted one higher 87% of the time. Say it this way, not "87% accurate".
Why AUC-ROC rather than accuracy?
Should you have reported PR AUC instead, given the imbalance?
What is calibration and why does a clinician need a calibrated probability?
How did you check calibration, and what did you do if it was off?
How did you choose the operating threshold?
What is number needed to treat and how does it connect to your threshold?
How did you split the data — random, temporal, or by hospital site?
Why is a random split dangerous with longitudinal patient data?
How did you prevent the same patient appearing in both train and test?
How did you validate across hospital sites with different populations?
What is external validation and did the model ever get it?

4 · Data engineering with PySpark and Databricks (12)

What made feature generation slow before, and what changed to make it 60% faster?
What is a Spark shuffle and why is it expensive?
How did you handle data skew — a few patients with enormous record counts?
What is the difference between a narrow and a wide transformation?
When did you cache or persist, and when is that a mistake?
How did you partition the data, and by what key?
What file format did you write, and why does columnar storage matter here?
How did you make the daily refresh idempotent and safely re-runnable?
How did you handle late-arriving lab results?
What is point-in-time correctness and how did you enforce it for vitals?The single biggest leakage risk in clinical ML — using a lab value recorded after the discharge decision.
How did you validate the pipeline output was correct, not just complete?
What did you monitor on the pipeline itself?

5 · Clinical NLP — spaCy and BioBERT (12)

What is BioBERT and how does it differ from BERT?
Why does biomedical pretraining matter for clinical text?
What is named entity recognition and what entities did you extract?
How did you handle negation — "no evidence of pneumonia" must not become a pneumonia label?Negation detection is the first thing that breaks in clinical NLP, and the reason naive keyword extraction fails badly.
How did you handle family history — "mother had diabetes" is not the patient's diagnosis?
How did you deal with abbreviations that mean different things in different specialties?
How did you map extracted entities to a standard vocabulary?
Why combine spaCy with BioBERT rather than using one alone?
How did you get labelled training data for clinical entities?
How did you measure the 35% improvement in structured coverage?
How did you evaluate NER quality — exact match or partial overlap?
What did extraction errors do downstream, and how did you contain them?

6 · Deployment and integration (12)

Why FastAPI rather than Flask or Django?
What did Docker solve for you in a hospital IT environment?
What does SageMaker manage that you would otherwise build?
What is HL7 and what is FHIR, and how do they differ?
How did the model receive patient data through the EHR interface?
What was your latency budget for real-time risk scoring, and why?
How did you handle a patient record arriving with missing required fields?
How did you version the API so hospital integrations did not break?
What happened when the model service was down — did clinical workflow stop?
How did you handle authentication and authorisation for the API?
How did you log requests without logging protected health information?
How did you deploy an update without disrupting a live hospital system?

7 · Explainability and clinician adoption (12)

Why did clinicians need explanations rather than just a risk score?
What did the SHAP dashboard actually show a physician?
How do you present a SHAP value to someone who has never seen one?
What is the difference between global and local explanation?
How did you avoid explanations that were technically correct but clinically meaningless?
What made physician adoption increase — the explanations, or something else?
How did you handle a clinician disagreeing with the model?The right answer treats disagreement as signal to investigate, not as user error — and the clinician usually has context the model does not.
What is automation bias and how did you guard against it?
How do you present uncertainty to a clinician without being ignored?
Did explanations ever reveal a data problem rather than a clinical insight?
How did you decide which risk factors to surface and how many?
What would you change about the dashboard now?

8 · Monitoring, drift, and MLOps (12)

What is data drift and how does it differ from concept drift?
What did Evidently actually monitor, and at what cadence?
What statistical test detects a distribution shift in a feature?
How did you avoid alert fatigue from drift warnings that did not matter?
What triggered retraining, and was it automatic?
How do you retrain in a HIPAA-compliant way?
What did MLflow track, and how did that help months later?
How did Jenkins fit into the release process?
How did you cut release cycles from two weeks to three days — what was the bottleneck?
How did you validate a retrained model before it replaced the incumbent?
Could you roll back, and had you ever needed to?
What would page you at 3am, and what would you do?

9 · Clinical evaluation and ethics (12)

How was the A/B test designed, and what was randomised?
Is randomising patients into a "no model" arm ethical? How was that handled?
What does the 18% reduction in readmissions measure, and over what period?
How did you separate the model's effect from the intervention's effect?The model only flags; the care team acts. A good answer acknowledges you measured the combined system, not the model alone.
What guardrail metrics did you watch during the pilot?
What is HIPAA and which parts constrained your design?
What is de-identification and what is the re-identification risk?
How did you check the model performed equitably across patient groups?
What happens if the model systematically under-flags an ethnic group?
How can historical care disparities become encoded in a readmission model?
Who is accountable if a flagged patient is missed and readmitted?
What documentation would a hospital review board require?

10 · What if — data and population (10)

What if the hospital changed its EHR vendor?
What if ICD-10 codes were replaced by ICD-11?
What if a new clinical protocol changed what discharge means?
What if a pandemic changed the patient population entirely?
What if physician notes stopped being dictated and became structured forms?
What if you had to deploy to a hospital with a quarter of the data volume?
What if lab results arrived hours after the discharge decision?
What if 30% of records had missing vitals?
What if you discovered readmission labels were undercounted because patients went to other hospitals?
What if you had to build this for paediatrics instead of adults?

11 · What if — model and production (10)

What if AUC held at 0.87 but clinicians stopped trusting the model?
What if the model flagged twice as many patients as the care team could handle?
What if the NLP pipeline started mis-extracting one medication class?
What if drift monitoring alerted every day for a month?
What if a retrained model was worse but passed automated validation?
What if the SageMaker endpoint hit its latency limit during morning rounds?
What if a bug caused the API to return the previous patient's score?
What if a clinician acted on an explanation that turned out to be an artefact?
What if the model performed well in the pilot ward and badly hospital-wide?
What if legal asked you to delete a patient's data from the training set?This is genuinely hard — deletion from a dataset does not remove the influence from an already-trained model, so the honest answer involves retraining.

12 · What if — design alternatives (8)

What if you could only use a model a clinician could compute by hand?
What if you had no access to physician notes at all?
What if you had to run entirely on-premises with no cloud?
What if the hospital wanted continuous risk updates rather than one score at discharge?
What if you replaced XGBoost with a large language model over the whole record?
What if you had to serve ten hospitals with different populations from one model?
What if the goal changed from prediction to recommending a specific intervention?
What if you had unlimited data but no labels?

Glossary

Term	What it means
EHR	Electronic Health Record — the digital record of a patient's care: demographics, diagnoses, labs, vitals, medications, notes.
30-day readmission	A patient returning to hospital within 30 days of discharge. A standard quality measure, often financially penalised.
Planned readmission	A scheduled return, such as staged surgery. Usually excluded, since it is not a care failure.
Prediction horizon	How far ahead the model predicts. Shorter is easier and often less actionable.
Cohort / exclusion criteria	Which patients are in scope, and which are deliberately removed — deaths, transfers, planned returns.
ICD-10	International Classification of Diseases, 10th revision — the standard diagnosis coding system.
Vitals	Routine physiological measurements: heart rate, blood pressure, temperature, respiratory rate, oxygen saturation.
Comorbidity	An additional condition present alongside the primary one. A major driver of readmission risk.
Length of stay	Days between admission and discharge. Predictive, but risky as a feature because it can encode the outcome.
XGBoost	Gradient-boosted decision trees with regularisation — the strong default for tabular data.
Gradient boosting	Building trees sequentially, each fitted to the errors the ensemble has made so far.
Bagging vs boosting	Bagging trains models independently on resamples and averages (random forest); boosting trains them in sequence, each correcting the last.
max_depth	Maximum tree depth. Higher means more interaction capacity and more overfitting risk.
Early stopping	Halting training when validation performance stops improving. Must use a set separate from the test set.
scale_pos_weight	An XGBoost parameter weighting the positive class to counter imbalance.
Regularisation	Penalties (L1, L2, minimum child weight) constraining the model so it generalises rather than memorises.
AUC-ROC	The probability the model ranks a random positive above a random negative. 0.5 is chance; 0.87 is a strong ranker.
PR AUC	Area under precision-recall. More informative than ROC when positives are rare.
Calibration	Whether predicted probabilities match observed frequencies. Essential when a clinician reads the number as a risk.
Operating threshold	The score above which a patient is flagged. Set by capacity and cost, not by the model.
Number needed to treat	How many patients must receive an intervention for one to benefit. Connects a threshold to clinical value.
Temporal split	Training on earlier data and testing on later, so evaluation reflects real deployment order.
Grouped split	Keeping all records for one patient in the same fold, so the model cannot memorise individuals.
External validation	Testing on data from a different hospital or system — the real test of generalisation.
Data leakage	Information in training features that would not be available at prediction time. Inflates offline scores and collapses in production.
Point-in-time correctness	Building each training row from only what was known at that moment.
PySpark	Python API for Apache Spark, distributing computation over a cluster.
Databricks	A managed platform for Spark with notebooks, job scheduling, and Delta Lake storage.
Shuffle	Redistributing data across the cluster for joins or aggregations — the expensive part of most Spark jobs.
Narrow vs wide transformation	Narrow operations act within a partition (map, filter); wide ones require a shuffle (groupBy, join).
Data skew	A few keys holding far more data than others, so one task runs long after the rest finish.
Caching / persist	Keeping a computed DataFrame in memory for reuse. Wasteful if the data is used once or does not fit.
Columnar format	Storage such as Parquet where a query reads only the columns it needs.
Idempotent pipeline	Re-running produces the same result rather than duplicating rows — essential for safe backfills.
spaCy	A fast production NLP library: tokenisation, POS tagging, dependency parsing, and named entity recognition.
BERT	A bidirectional transformer encoder pretrained by predicting masked tokens from surrounding context.
BioBERT	BERT further pretrained on biomedical literature, so clinical vocabulary and usage are already represented.
Domain pretraining	Continuing pretraining on in-domain text before fine-tuning. Large gains where general text differs from the target domain.
NER	Named Entity Recognition — labelling spans of text as entities such as medication, symptom, or diagnosis.
Negation detection	Recognising that "no evidence of pneumonia" asserts absence. Skipping it produces confidently wrong extractions.
Family history attribution	Distinguishing a relative's condition from the patient's. Another classic clinical NLP failure.
Clinical vocabulary mapping	Normalising extracted text to a standard terminology such as SNOMED CT or RxNorm so downstream systems agree.
Partial-match evaluation	Scoring NER by span overlap rather than exact boundaries — often the fairer measure for clinical text.
FastAPI	A Python web framework deriving validation and OpenAPI docs from type hints, serving requests asynchronously.
Docker	Packages an application with its dependencies into an image that runs identically across environments.
SageMaker	AWS's managed ML platform — training jobs, model registry, and autoscaling inference endpoints.
HL7	Health Level Seven — the long-standing messaging standard for exchanging clinical data between hospital systems.
FHIR	Fast Healthcare Interoperability Resources — the modern HL7 standard, REST and JSON based, organised around resources like Patient and Observation.
Real-time risk scoring	Producing a score on demand within a clinical workflow, rather than as an overnight batch.
PHI	Protected Health Information — identifiable health data. Governs what may be logged, stored, or transmitted.
HIPAA	US legislation governing the privacy and security of protected health information, including access control and audit.
De-identification	Removing identifiers so data is no longer legally protected. Never perfectly reversible-proof — re-identification risk remains.
SHAP	Attributes a prediction to its features using a fair, additive game-theoretic allocation.
Global vs local explanation	Global describes the model's overall behaviour; local explains one specific prediction. Clinicians need local.
Automation bias	The tendency to defer to an automated recommendation even when your own judgment disagrees.
Data drift	The input distribution changing over time — new coding practice, different case mix.
Concept drift	The relationship between features and outcome changing, so a once-accurate model decays.
Evidently AI	An open-source library for monitoring data quality, drift, and model performance with prebuilt reports.
Kolmogorov-Smirnov / PSI	Statistical tests commonly used to flag distribution shift in a feature between reference and current data.
Alert fatigue	So many alerts that real ones are ignored. A monitoring design failure, not a user failure.
MLflow	Tracks experiment parameters, metrics, and artefacts, with a registry for versioned models.
Jenkins	An automation server running build, test, and deployment pipelines.
Guardrail metric	A secondary metric watched during an experiment to catch harm the primary metric would hide.
Equity evaluation	Checking performance separately across demographic groups rather than trusting an aggregate number.
Historical bias	Past inequity encoded in the data — if a group historically received less follow-up, the labels reflect that, not their true risk.

Back to all projects.

Kiran Davuluri