Document Intelligence & Edge Inference — Infiswift.ai

Consolidating fragmented ML repositories into one inference API, building a content-aware document validation agent on Vertex AI Gemini with LangChain and LangGraph, and shipping wake-word models to edge hardware through ONNX and TensorRT.

AI Engineer · Infiswift.ai, California · Nov 2025 – Present

← All projects What I built Architecture Tech specs Interview questions Glossary

What I built

Modular NLP extraction API — consolidated multiple machine learning repositories into a unified inference pipeline, eliminating duplicate LLM workflows and improving maintainability across engineering teams.
Content-aware document validation agent — built with Vertex AI Gemini, LangChain, and LangGraph to classify uploaded RFI documents by their actual content, reducing downstream failures caused by incorrect file categorisation.
Centralised FileType glossary injection — a mechanism for injecting business rules into LLM prompts, enabling instant rule updates without model retraining and keeping classification consistent across environments.
Self-learning classification framework — automatically generated and validated regex patterns from newly identified document titles, expanding deterministic coverage while minimising manual rule creation.
AWS serverless orchestration pipeline — Lambda, SQS FIFO, and Amazon S3 managing automated rule promotion, deduplication, version control, and deployment across Development, Preview, and Production.
Wake-word detection optimisation — multi-layer perceptron architectures over Mel-spectrogram features, exported through ONNX and TensorRT for low-latency edge inference driving robotic arm control.
Testing and evaluation frameworks — PyTest and Moto to validate AI workflows, cloud integrations, and model behaviour, improving deployment reliability and reducing regression risk.
LLM evaluation and A/B testing — iterative evaluation across production projects, analysing model agreement and reasoning quality to identify classification gaps and refine prompt strategies.

Architecture

Two paths run side by side: an LLM classifies what regex cannot, and confirmed patterns are promoted into deterministic rules so the expensive path is used less over time.

Technical specifications

LLM & platform	Vertex AI Gemini on Google Cloud
Orchestration	LangChain for components, LangGraph for stateful agent control flow
Rule injection	Centralised FileType glossary injected into prompts at request time
Deterministic layer	Auto-generated, validated regex patterns from observed document titles
Cloud pipeline	AWS Lambda, SQS FIFO queues, Amazon S3
Environments	Development → Preview → Production, with version control on rules
Edge model	MLP over Mel-spectrogram features for wake-word detection
Edge runtime	ONNX export, TensorRT optimisation, low-latency inference for robotic arm control
Testing	PyTest for logic, Moto for mocked AWS services
Evaluation	LLM evaluation with A/B testing, model agreement and reasoning-quality analysis

Interview questions 132 questions

Grouped by what an interviewer is probing. The "what if" sections are the ones that separate a rehearsed answer from real understanding — they change one variable and see whether your reasoning survives. Terms used here are defined in the glossary.

1 · Framing and scope (10)

What problem was the document validation agent actually solving, in business terms?
What was breaking downstream when files were categorised incorrectly?Name the concrete failure, not "errors" — wrong extraction schema applied, wasted LLM spend, bad data landing in the warehouse.
Why was content-aware classification necessary rather than trusting filenames or MIME types?
How did you decide what counted as a correct classification?
Who were the users of this system, and what did they experience before it existed?
What did "consolidating multiple ML repositories" mean concretely — what was duplicated?
How did you decide which repositories to merge and which to leave alone?
What was the cost of the duplication you removed — engineering time, compute, or correctness?
How did you avoid the consolidated API becoming a bottleneck that every team had to queue behind?
If you started this project again, what would you scope differently?

2 · LLM application design — LangChain and LangGraph (12)

Why LangGraph rather than a plain LangChain chain?The honest answer is state and branching — a document that fails validation needs to route differently, which a linear chain cannot express cleanly.
What state did your graph carry between nodes?
Where were the conditional edges in your graph, and what decided them?
How did you prevent the agent from looping indefinitely?
What did a node failure do — retry, route to a fallback, or fail the request?
Why use a framework at all rather than calling the model API directly?
What did LangChain give you that you would otherwise have written yourself?
How did you handle the model returning malformed or unparseable output?
Did you use structured outputs or schema enforcement? What happens when the schema is violated?
How did you manage prompt versioning across environments?
How much of the latency was model time versus orchestration overhead?
What would make you drop the framework and hand-roll the loop?

3 · The FileType glossary injection mechanism (10)

Explain the glossary injection mechanism to someone who has never seen it.
Why inject business rules into the prompt instead of fine-tuning the model on them?Rules change weekly; retraining does not. This is the core argument and it is about change frequency, not model capability.
What exactly gets injected — definitions, examples, or both?
How did you keep the glossary from growing until it consumed the context window?
How did you guarantee the same glossary version was used across Development, Preview, and Production?
What happens to prompt caching when the glossary changes?
How did you validate that a glossary change improved rather than degraded classification?
Who owned the glossary — engineering or the business?
How did you prevent a bad glossary edit from reaching production?
At what glossary size would this approach stop working, and what would replace it?

4 · The self-learning regex framework (12)

Walk me through how a new regex pattern gets generated, validated, and promoted.
What generated the candidate pattern — the LLM, a heuristic, or both?
How did you validate a generated regex before trusting it?The interesting part is the negative set: a pattern that matches everything scores perfectly on positives.
How did you prevent an over-broad pattern such as .* from being promoted?
How did you handle two patterns that both match the same document?
What was the precision requirement for promotion, and how was it chosen?
Why prefer deterministic regex over letting the LLM classify every document?
How much traffic ended up served by regex versus the model?
How did you retire a pattern that started performing badly?
What stops this system from slowly drifting into a thicket of unmaintainable rules?
How is this different from just fine-tuning a classifier on the same data?
How did you catch a regex with catastrophic backtracking before it reached production?

5 · AWS serverless orchestration (12)

Why SQS FIFO rather than a standard queue?Ordering and deduplication. Rule promotion is a state machine — applying version 3 before version 2 corrupts it.
What is a message group ID and how did you choose yours?
What throughput ceiling does FIFO impose, and did you ever hit it?
How did you make the Lambda consumers idempotent?
What happens when a Lambda times out halfway through a promotion?
Where did you use a dead-letter queue, and what did you do with the messages in it?
How did S3 fit — storage of what, exactly, and with what key structure?
How did you version rules, and could you roll back?
How did promotion across Development, Preview, and Production actually work?
What prevented a rule from skipping an environment?
How did you handle Lambda cold starts on this path?
Why serverless rather than a long-running service?

6 · Wake-word detection and audio ML (12)

What is a Mel-spectrogram, and why use it instead of the raw waveform?Mel spacing matches human pitch perception, and it turns a 1-D signal into a 2-D representation a small network can consume cheaply.
What were your window size, hop length, and number of Mel bands, and how did those choices affect latency?
Why an MLP rather than a CNN or an RNN for wake-word detection?
What is the input dimensionality of your MLP and how did you arrive at it?
How do you handle variable-length audio with a fixed-size MLP input?
What is your false-accept versus false-reject tradeoff, and who decided it?
How did you collect and label training data for the wake word?
How did you handle background noise, accents, and distance from the microphone?
What is the streaming inference setup — sliding window, and at what stride?
How did you measure end-to-end latency from utterance to robot action?
What happens when two wake words overlap or the phrase is cut off?
Why does a false accept matter more when the output controls a robotic arm?

7 · Model optimisation and edge inference (12)

Why export to ONNX at all — what does it buy you?
What is ONNX, and what does the exported graph actually contain?
What does TensorRT do to the model that ONNX Runtime does not?Layer fusion, kernel autotuning for the specific GPU, and precision calibration — it compiles rather than interprets.
What precision did you run at, and how did you verify accuracy was preserved?
What is a calibration dataset and why does INT8 quantisation need one?
How did you validate that the ONNX model matched the PyTorch model numerically?
What broke during export, and how did you diagnose it?
What are dynamic axes in an ONNX export and when do you need them?
How did you measure latency — mean, p99, or worst case, and why?
How much of the latency was model inference versus feature extraction?
What is the memory footprint on the edge device, and was that a constraint?
Why is a TensorRT engine not portable between GPU models?

8 · Testing and reliability (10)

What is Moto and why use it rather than hitting real AWS in tests?
What did you test with PyTest that was genuinely worth testing?
How do you test a component whose output is non-deterministic?Test the contract, not the text — schema validity, required fields, latency bounds, and behaviour on malformed input.
How did you test the LangGraph agent's routing logic without calling the model?
What is a fixture and how did you use them here?
How did you test the SQS FIFO ordering guarantees?
What did your CI pipeline run on every commit?
How did you prevent flaky tests from eroding trust in the suite?
What regression escaped your tests, and what did you add afterwards?
What is the difference between mocking the model and mocking the cloud service, in terms of what each protects you from?

9 · LLM evaluation and A/B testing (12)

How did you build the evaluation set, and how big was it?
What metrics did you use for classification quality, and why those?
What does "model agreement" mean and how did you measure it?Agreement between models or between runs is a proxy for confidence — high disagreement flags the examples worth human review.
How did you evaluate reasoning quality rather than just the final label?
Did you use an LLM as a judge? What are the failure modes of that?
How did you decide a prompt change was an improvement rather than noise?
How did you run an A/B test on an LLM feature — what was randomised?
What sample size did you need to detect the effect you cared about?
What guardrail metrics did you watch to be sure you were not trading quality for speed?
How did you identify classification gaps from the evaluation results?
How did you stop evaluation set contamination once you started tuning against it?
What did you do when offline evaluation and production behaviour disagreed?

10 · What if — scale and load (10)

What if document volume increased a hundredfold overnight?
What if the LLM provider had a partial outage for two hours during business hours?
What if per-document cost tripled because of a pricing change?
What if the FIFO queue's throughput limit became the bottleneck?
What if a single tenant submitted a million documents and starved everyone else?
What if the glossary grew to twenty thousand tokens?
What if you had to serve this synchronously with a two-second budget instead of asynchronously?
What if documents arrived as scanned images rather than text?
What if the same document was submitted a hundred times in a minute?
What if you needed to run entirely inside a customer's VPC with no internet access?

11 · What if — correctness and failure (10)

What if the model silently started misclassifying one document type after a provider model update?
What if a promoted regex began matching documents it should not?
What if the glossary and the regex layer disagreed on a document?
What if a customer disputed a classification and you had to explain it?This is really about traceability — can you reconstruct which glossary version, which prompt, and which rule produced that answer?
What if the LLM leaked content from one document into the classification of another?
What if someone embedded instructions inside a document to manipulate the classifier?
What if the wake-word model started firing on background speech in a noisy factory?
What if the TensorRT engine produced different results from the PyTorch model in production?
What if a rule promotion was applied to Production but not Preview?
What if you discovered the evaluation set itself was mislabelled?

12 · What if — design alternatives (10)

What if you had to remove the LLM entirely — how far could deterministic rules take you?
What if you fine-tuned a small classifier instead of prompting a large model?
What if you had to switch from Gemini to a different provider next month?
What if the business wanted a confidence score on every classification?
What if you had to support fifty document types instead of a handful?
What if latency mattered more than accuracy?
What if you had no labelled data at all when you started?
What if the wake-word model had to run on a microcontroller instead of a Jetson-class device?
What if you had to make the whole pipeline auditable for a regulated customer?
What if you had unlimited budget — what would you actually change?A good answer names something other than "a bigger model", because the bottleneck usually is not model capability.

Glossary

Term	What it means
RFI	Request For Information — a formal document exchanged in construction and procurement asking for clarification. Here, the document type being classified.
Vertex AI	Google Cloud's managed machine learning platform, including hosted access to Gemini models.
Gemini	Google's multimodal model family, accepting text, images, audio, and video in one context.
LangChain	A framework of composable components — prompts, models, retrievers, output parsers — for building LLM applications.
LangGraph	Models an agent as an explicit state graph with nodes and conditional edges, so loops and branches are first-class rather than implicit.
Node / edge	In LangGraph, a node is a step (call a model, run a tool) and an edge is the transition between steps. A conditional edge chooses the next node at runtime.
State	The data carried between nodes of the graph — the document, partial results, retry counts, confidence.
Prompt injection	Text inside untrusted input that the model interprets as instructions. A real risk when the model reads customer documents.
Structured output	Constraining the model to emit JSON matching a schema, so downstream parsing cannot fail on free text.
Prompt caching	Reusing the processed form of a stable prompt prefix across requests. Cheap and fast — but a prefix match, so changing the glossary early in the prompt invalidates it.
Context window	The maximum number of tokens a model can attend to at once. The glossary competes with the document for this budget.
Regex	Regular expression — a pattern language for matching text. Deterministic, fast, and free compared with a model call.
Catastrophic backtracking	A regex whose evaluation time explodes exponentially on certain inputs, effectively hanging the process. A denial-of-service risk from a badly written pattern.
Precision / recall	Precision is the share of predicted positives that are correct; recall is the share of actual positives found. A promotion rule should demand high precision.
Negative set	Examples a pattern should not match. Without it, an over-broad pattern looks perfect.
Deterministic	Same input always produces the same output. Regex is deterministic; an LLM call generally is not.
AWS Lambda	Runs a function in response to an event with no server to manage, scaling automatically and billing per execution.
Cold start	The extra latency when a Lambda has to initialise a new execution environment before running your code.
SQS	Amazon's managed message queue, decoupling producers from consumers.
FIFO queue	A queue preserving strict order within a message group and deduplicating messages, giving exactly-once processing at lower throughput than a standard queue.
Message group ID	The key that defines an ordering scope in a FIFO queue. Messages in the same group are strictly ordered; different groups run in parallel.
Idempotency	Performing an operation twice has the same effect as once. Required because at-least-once delivery means retries will happen.
Dead-letter queue	Where messages go after repeated processing failures, so a poison message stops blocking the queue and becomes visible instead.
Amazon S3	Object storage addressed by key within buckets — effectively unlimited, highly durable, the usual home for artefacts and rule files.
Environment promotion	Moving a change through Development, then Preview, then Production, each acting as a gate.
Wake word	A short phrase that activates a device, detected by a small always-on model.
Mel-spectrogram	A time-frequency representation of audio with frequency bands spaced to match human pitch perception — the standard input for small speech models.
Window / hop length	How much audio each analysis frame covers, and how far the window advances each step. Together they set time resolution and latency.
MLP	Multilayer perceptron — fully connected layers with non-linear activations. Small, fast, and adequate for fixed-size feature inputs.
Sliding window inference	Running the model repeatedly over overlapping audio segments so a wake word is caught wherever it falls.
False accept / false reject	Triggering when the wake word was not said, versus missing it when it was. The costs are asymmetric and depend on what the device then does.
ONNX	An open format for representing a model as a portable computation graph, so it can run in runtimes other than the one that trained it.
Dynamic axes	Dimensions of an ONNX input marked as variable — typically batch size or sequence length — so one exported model serves many shapes.
TensorRT	NVIDIA's inference compiler. Fuses layers, calibrates precision, and selects kernels for a specific GPU, producing a fast but non-portable engine.
Layer fusion	Combining several operations into one kernel to avoid writing intermediate results to memory — usually the largest single inference win.
Quantisation	Running weights and activations at lower precision (INT8, FP8) for less memory and faster maths, at some accuracy cost.
Calibration dataset	A representative sample used to choose quantisation scales so the reduced precision range covers real activations.
Edge inference	Running the model on the device rather than in the cloud — lower latency, no network dependency, tighter memory limits.
PyTest	Python's standard testing framework, built around plain functions and fixtures.
Fixture	Reusable setup shared across tests — a client, a temporary bucket, a sample document.
Moto	A library that mocks AWS services in-process, so tests exercise S3 and SQS logic without network calls, credentials, or cost.
Regression test	A test that re-runs known cases on every change, catching a quality drop before users do.
LLM-as-judge	Using a model to score another model's output. Cheap and scalable, with known biases toward length, position, and its own style.
Model agreement	How often two models, or two runs, produce the same answer. Disagreement is a useful signal for which cases need human review.
Guardrail metric	A secondary metric watched during an experiment to catch damage the primary metric would hide — cost, latency, or refusal rate.
Evaluation contamination	Tuning against your evaluation set until the score stops measuring generalisation and starts measuring memorisation.
A/B test	Randomly assigning traffic between variants so the measured difference can be attributed to the change rather than to who received it.

Back to all projects.

Kiran Davuluri