Real-Time Fraud Detection & GenAI Investigation — JPMorgan Chase

A fraud detection platform scoring 5M+ card transactions daily, a streaming feature store serving features in under 50 ms, and a RAG-based investigation assistant that cut analyst time by 40% — built under the explainability and governance constraints that financial services imposes.

AI/ML Engineer · JPMorgan Chase & Co., California · Sept 2023 – Oct 2025

← All projects What I built Architecture Tech specs Interview questions Glossary

What I built

Real-time fraud detection platform — processing 5M+ daily card transactions with LightGBM and Isolation Forest models, reducing confirmed fraud losses by 22% while improving transaction risk scoring.
Feast-based feature store — integrated with Apache Kafka to serve streaming features at sub-50 ms latency, enabling real-time detection and decisioning.
GenAI fraud investigation assistant — LangChain, Pinecone, and Retrieval-Augmented Generation, reducing analyst investigation time by 40% through contextual case retrieval.
Transformer-based anomaly detection — trained in PyTorch to identify abnormal transaction patterns, increasing detection accuracy by 15% while holding false positives below 2%.
Automated ML lifecycle — training, validation, versioning, and deployment through Kubeflow Pipelines, Terraform, and AWS EKS, improving deployment consistency and accelerating release cycles.
Model explainability — SHAP and LIME providing transparent fraud risk predictions to support regulatory compliance, model governance, and audit.
Inference optimisation — ONNX Runtime and post-training quantisation improving scoring throughput 3× while reducing latency for real-time processing.
Responsible AI controls — with Risk, Compliance, and Data Science: bias monitoring, output validation, and prompt injection safeguards enabling secure GenAI deployment.

Architecture

The scoring path is latency-bound; the investigation path is quality-bound. Explainability is what connects them — a flagged transaction arrives at the analyst with reasons attached.

Technical specifications

Volume	5M+ card transactions scored daily
Primary models	LightGBM (supervised), Isolation Forest (unsupervised anomaly), transformer-based sequence models in PyTorch
Feature serving	Feast feature store fed by Apache Kafka, sub-50 ms online lookup
GenAI assistant	LangChain orchestration, Pinecone vector store, Retrieval-Augmented Generation over historical cases
Orchestration	Kubeflow Pipelines for training and deployment, Terraform for infrastructure, AWS EKS for runtime
Explainability	SHAP and LIME for per-prediction attribution supporting audit and model governance
Inference	ONNX Runtime with post-training quantisation — 3× throughput improvement
Governance	Bias monitoring, output validation, prompt injection safeguards, with Risk and Compliance
Reported outcomes	22% reduction in confirmed fraud losses; 15% detection accuracy improvement at < 2% false positives; 40% reduction in investigation time

Interview questions 134 questions

Fraud detection interviews concentrate on class imbalance, the cost asymmetry between error types, and latency under regulation. The "what if" groups are where interviewers find out whether you understand the system or only remember it. Terms are defined in the glossary.

1 · Problem framing and business context (10)

How do you frame fraud detection as a machine learning problem?
What is the cost of a false positive versus a false negative here, in real terms?A blocked legitimate transaction is a customer service event and possible churn; a missed fraud is a direct loss plus liability. They are not symmetric and the ratio should drive the threshold.
Where did your labels come from, and how long did they take to arrive?
What is label delay and how does it affect training and evaluation?
How did you handle fraud that was never reported and therefore never labelled?
What does the 22% reduction in confirmed fraud losses actually measure?
How did you attribute that reduction to the model rather than to other changes?
Who consumed the risk score, and what action did it trigger?
What was the baseline before the platform existed?
How did you decide the threshold at which a transaction is held?

2 · Model choice and design (12)

Why LightGBM rather than XGBoost or a neural network?Tabular data, speed of training and inference, native categorical handling, and interpretability that survives a compliance review.
What does Isolation Forest do that LightGBM cannot?
Why run a supervised and an unsupervised model together?
How do you combine their outputs into a single decision?
What did the transformer add over the gradient-boosted model?
What is the sequence in your transformer — transactions per card over time?
How did you prevent the transformer from simply relearning what LightGBM already caught?
How did you decide the 15% accuracy improvement was real and not overfitting?
What features mattered most, and were you surprised by any of them?
How did you handle categorical features with very high cardinality, such as merchant ID?
How did you deal with new merchants or cards with no history?
Why not just use rules? What did the model give you that rules did not?

3 · Class imbalance and metrics (12)

What was your fraud base rate, and why does that number dominate everything else?
Why is accuracy a useless metric here?Predicting "not fraud" always would score above 99%. Say this plainly — it is the fastest way to show you understand the problem.
Why precision-recall AUC rather than ROC AUC on imbalanced data?
What is precision at a fixed recall, and why is it the metric operations actually cares about?
How did you keep false positives below 2%, and what did that cost you in recall?
Did you resample — SMOTE, undersampling, class weights? What did you choose and why?
What goes wrong when you apply SMOTE before splitting the data?
How did you set the decision threshold, and did it differ by segment?
What is calibration and why does a fraud score need to be calibrated?
How would you explain the precision-recall tradeoff to a non-technical risk manager?
How did you evaluate over time rather than on a random split?
What is a cost-sensitive loss and would it have been a better objective here?

4 · Feature store and streaming (12)

What is a feature store and what problem does it solve?
What is training-serving skew, and how does a feature store prevent it?The same feature definition computed once, served to both training and inference. Without it, the batch SQL and the online code drift apart silently.
What is the difference between the online and offline store in Feast?
What is point-in-time correctness and why does it matter for fraud features?
How did you compute rolling aggregations — transactions in the last hour, for example — in a streaming context?
How did Kafka fit into the pipeline, and what were the topics partitioned by?
Why does partitioning by card ID matter for ordering?
How did you achieve sub-50 ms feature lookup, and what was the storage backend?
What happens to a scoring request when a feature is missing or stale?
How did you handle late-arriving events in the stream?
What is watermarking and did you need it?
How did you monitor feature freshness in production?

5 · The RAG investigation assistant (12)

What exactly did the assistant retrieve, and from where?
Why RAG rather than fine-tuning a model on historical cases?
How did you chunk case documents, and what did you experiment with?
Which embedding model did you use and how did you evaluate it?
Why Pinecone rather than pgvector or FAISS?
How did you measure retrieval quality separately from answer quality?Retrieval precision and recall at k, evaluated independently — most "the model is wrong" complaints are retrieval failures wearing a costume.
How did you prevent the assistant from hallucinating case details?
Did you enforce citations back to the source case? How?
How did you measure the 40% reduction in investigation time?
How did you handle personally identifiable information in retrieved cases?
What did the assistant do when no relevant case existed?
How did analysts give feedback, and did that feedback improve the system?

6 · MLOps — Kubeflow, Terraform, EKS (12)

What did your Kubeflow pipeline actually do, stage by stage?
How were models versioned, and could you tie a production score back to a training run?
What triggered retraining — a schedule, drift, or performance decay?
How did you validate a new model before it replaced the incumbent?
What is a champion-challenger setup and did you run one?
How did you deploy — blue-green, canary, or shadow mode?Shadow mode is the strong answer for fraud: score in parallel without acting, compare, then promote.
How did you roll back a bad model, and how quickly?
Why Terraform rather than configuring infrastructure by hand?
What did EKS give you over running on EC2 directly?
How did you handle autoscaling for a workload with a daily traffic pattern?
How did you separate the training and serving infrastructure?
What did your on-call runbook say to do when scoring latency spiked?

7 · Explainability and governance (12)

What is SHAP and what does a SHAP value actually represent?The contribution of one feature to the difference between this prediction and the average prediction, averaged fairly over all feature orderings.
How does LIME differ from SHAP, and why use both?
Why is TreeSHAP feasible on a gradient-boosted model when exact SHAP is not?
How did you deliver explanations at the latency required for real-time scoring?
Who consumed the explanations, and did they change any decisions?
What does a regulator actually want to see in model documentation?
What is model governance in a bank, and what artefacts did you produce?
How did you monitor for bias, and against which attributes?
What is disparate impact and how would you detect it in a fraud model?
What do you do when a protected attribute is not in the model but is inferable from other features?
How did you handle the tension between accuracy and explainability?
Could you explain a transformer's decision to a compliance officer?

8 · Inference optimisation (12)

Where was the 3× throughput improvement actually coming from?
What is post-training quantisation and how does it differ from quantisation-aware training?
How did you verify the quantised model still scored transactions the same way?
What accuracy loss did you accept, and who approved it?
Why ONNX Runtime rather than serving the PyTorch model directly?
What is graph optimisation in ONNX Runtime — what does it change?
What is the difference between throughput and latency, and which one mattered here?Both, at different points: throughput for the daily 5M batch, p99 latency for the real-time authorisation path.
How did you batch requests without adding unacceptable latency?
What was your p99 scoring latency and what was the budget?
How did you profile to find the bottleneck before optimising?
How much of end-to-end latency was feature lookup versus model inference?
What would you have optimised next?

9 · Responsible AI and security (10)

What is prompt injection and how does it apply to a fraud investigation assistant?
How could an attacker manipulate the assistant through transaction data it reads?
What output validation did you apply before showing an answer to an analyst?
How did you prevent sensitive data leaking into the model provider's logs?
What is data residency and did it constrain your design?
How did you handle the right to an explanation for a declined transaction?
What is adversarial drift — fraudsters adapting to your model — and how did you detect it?
How often did you retrain, and was that fast enough against adaptive adversaries?
What controls did Risk and Compliance require before GenAI could touch production data?
How would you red-team this system?

10 · What if — data and distribution (10)

What if the fraud rate doubled in a week?
What if fraudsters found and exploited a blind spot in your features?
What if a major merchant changed its transaction format overnight?
What if the Kafka stream lagged by ten minutes?
What if a feature became unavailable in production but was present in training?
What if your labels turned out to be systematically biased toward one customer segment?
What if a bug meant three months of training data had a corrupted field?
What if the model performed well overall but badly on one country?
What if a holiday shopping spike looked like coordinated fraud?
What if you had to launch in a new market with no historical fraud data?

11 · What if — production incidents (10)

What if scoring latency exceeded the authorisation timeout during peak traffic?
What if the model started declining 10× more transactions than yesterday?
What if the feature store returned stale values without erroring?The dangerous failures are the silent ones — this is why freshness needs its own alert rather than relying on error rates.
What if the new model passed offline evaluation but degraded in production?
What if you had to disable the model entirely — what is the fallback?
What if Pinecone was unavailable during an investigation surge?
What if an analyst acted on a hallucinated case summary?
What if a rollback restored a model trained on data you have since deleted for privacy reasons?
What if two models in the ensemble disagreed sharply on a high-value transaction?
What if you discovered the 22% loss reduction was partly caused by an unrelated policy change?

12 · What if — design alternatives (10)

What if you had to score in 10 ms instead of 50 ms?
What if the bank forbade any model that could not be explained by a linear equation?
What if you could not store any customer data at all?
What if you had to run entirely on-premises with no cloud services?
What if you had 100× the transaction volume?
What if the assistant had to work for analysts in five languages?
What if you replaced the whole ensemble with a single large model — what would you lose?
What if the business wanted the model to explain itself in natural language to customers?
What if labels became available in real time instead of weeks later?
What if you had to hand this system to a team who had never seen it — what would you document first?

Glossary

Term	What it means
Base rate	The proportion of transactions that are actually fraudulent — typically well under 1%, which is why imbalance dominates the problem.
Class imbalance	One class vastly outnumbering the other, so a model can score well by ignoring the minority class entirely.
False positive	A legitimate transaction flagged as fraud. Costs customer trust and support effort.
False negative	Fraud that was not caught. Costs money directly and may carry liability.
Precision	Of the transactions flagged as fraud, the share that really were.
Recall	Of the fraud that occurred, the share that was caught.
PR AUC	Area under the precision-recall curve. More informative than ROC AUC under heavy imbalance because it ignores the abundant true negatives.
ROC AUC	Area under the receiver operating characteristic curve. Looks flatteringly high on imbalanced data.
Precision at fixed recall	The operational metric — "at the recall we need, how many false alarms must the team absorb?"
Calibration	Whether a predicted probability means what it says — of everything scored 0.8, roughly 80% should be fraud.
Decision threshold	The score above which action is taken. A business decision, not a modelling one.
Cost-sensitive loss	A loss function weighting errors by their real cost rather than treating them equally.
SMOTE	Synthetic Minority Over-sampling — generating synthetic minority examples by interpolating between neighbours. Must be applied inside the training fold only.
Label delay	The lag between a transaction and a confirmed fraud label, often weeks. It limits how quickly you can learn and how you must evaluate.
LightGBM	Gradient-boosted trees using histogram-based splits and leaf-wise growth — fast on large tabular data with native categorical handling.
Isolation Forest	An unsupervised anomaly detector that isolates points with random splits; outliers separate in fewer splits. Needs no labels, so it can catch novel fraud.
Ensemble	Combining several models' outputs, ideally with uncorrelated errors so their mistakes cancel rather than compound.
High-cardinality categorical	A field with very many distinct values, such as merchant ID. One-hot encoding explodes; target or hash encoding is usual.
Cold start	A new card or merchant with no history, so history-based features are empty.
Feature store	A system that defines features once and serves them consistently to both training and inference.
Feast	An open-source feature store with an offline store for training data and an online store for low-latency serving.
Online vs offline store	Online is a fast key-value lookup for serving; offline holds full history for training. Both derive from the same definitions.
Training-serving skew	Features computed differently at training and serving time — a leading cause of models that work offline and fail in production.
Point-in-time correctness	Building training rows using only data available at that moment, so the model never sees the future.
Apache Kafka	A distributed append-only log. Producers write to partitioned topics; consumers read at their own offsets, with ordering guaranteed within a partition.
Partition key	The field deciding which partition an event lands in. Keying by card ID keeps one card's events ordered.
Late-arriving data	Events that reach the stream after events that happened later, breaking naive time-window aggregations.
Watermarking	A stream-processing mechanism declaring how late data may be before a window is closed.
Feature freshness	How recently a served feature was updated. Stale features fail silently, so freshness needs its own monitoring.
RAG	Retrieval-Augmented Generation — retrieve relevant documents at query time and put them in the prompt so answers rest on evidence.
Chunking	Splitting documents into retrievable pieces. Chunk size and overlap materially affect retrieval quality.
Embedding	A dense vector representation placing semantically similar text near one another.
Pinecone	A managed vector database serving approximate nearest-neighbour search with metadata filtering.
Retrieval precision at k	Of the top k retrieved documents, how many were genuinely relevant — measured separately from the final answer.
Hallucination	Confidently stated output not supported by the evidence. In an investigation tool, actionable and dangerous.
Citation enforcement	Requiring every claim to point at a retrieved source, so an analyst can verify rather than trust.
PII	Personally identifiable information. Governs what may be retrieved, logged, or sent to a model provider.
Kubeflow Pipelines	Orchestrates ML workflows as containerised steps on Kubernetes, with tracked inputs and outputs per run.
Terraform	Declares infrastructure as versioned code and reconciles cloud resources against that declaration.
EKS	AWS-managed Kubernetes — AWS runs the control plane, you run the workloads.
Champion-challenger	Running a candidate model alongside the incumbent and promoting only if it wins on agreed metrics.
Shadow mode	Scoring live traffic with a new model without acting on its output, to compare safely before promotion.
Canary deployment	Routing a small share of traffic to the new version first, expanding only if metrics hold.
Blue-green deployment	Two full environments with traffic switched between them, giving an instant rollback path.
SHAP	Shapley Additive exPlanations — attributes a prediction to its features using a game-theoretic allocation that is fair and additive.
TreeSHAP	An exact, polynomial-time SHAP algorithm specific to tree ensembles, which makes explanations affordable on gradient-boosted models.
LIME	Local Interpretable Model-agnostic Explanations — fits a simple surrogate model around one prediction to approximate local behaviour.
Model governance	The controls around a model in a regulated setting: documentation, validation, approval, monitoring, and audit trail.
Disparate impact	A model producing systematically different outcomes across protected groups, even without using the protected attribute.
Proxy variable	A feature correlated with a protected attribute, letting bias enter a model that never sees the attribute itself.
Bias monitoring	Tracking outcome differences across segments in production, not only at training time.
ONNX Runtime	Executes models in the ONNX format across hardware, applying graph optimisations such as operator fusion and constant folding.
Post-training quantisation	Reducing weight and activation precision after training, using a calibration sample. No retraining required.
Quantisation-aware training	Simulating reduced precision during training so the model learns to tolerate it — better accuracy, more expensive.
Throughput vs latency	Transactions per second versus time for one transaction. Batching raises throughput and usually raises latency.
p99 latency	The value 99% of requests come in under. The number that reflects the worst experience users actually get.
Adversarial drift	Distribution shift caused by adversaries adapting to your defences — unlike ordinary drift, it is deliberate and responsive.
Concept drift	The relationship between features and label changing over time, so a once-accurate model decays.
Prompt injection	Instructions hidden in data the model reads, hijacking its behaviour. Relevant wherever a model consumes untrusted content.
Output validation	Checking model output against rules or schemas before it reaches a user or triggers an action.
Red teaming	Deliberately attacking your own system to find failures before an adversary does.

Back to all projects.

Kiran Davuluri