Hello, I’m Basant Bhattarai.
Senior Data & AI Engineer
I design data platforms, AI systems, and the architecture between them.
Nine years on systems that have to keep working: terabyte-scale ingestion, the databases underneath, and agents that write into governed stores, not a chat window.
What I'm hired for
Four problems worth calling me about.
-
Data platforms that keep working at the next scale
Ingestion, processing and serving that worked at one scale and stopped at the next: streams that stay exact through failure, lakehouse tables you can roll back, and contracts that wait for the consumers they would break.
- Case study Rebuilding a production data platform for scale and recovery
- Write-up Exactly-once held through nine failures. The window counts did not.
- Code spark-sql-kinesis Kinesis source and sink for Spark 4, on Maven Central
-
AI systems that have to touch production data
Agents with typed output, provenance and a review path, and the evaluation under them: a gate that can tell a regression from noise, and a defined answer for when the model cannot decide.
- Case study Turning model output into product state you can audit
- Write-up The eval score fell 4.8 points. Twelve answers decided whether it meant anything.
- Code evalproof LLM evaluation as measurement, gated in CI
-
Databases that scale out and survive failure
Sharding and rebalancing, failover, backup and point-in-time recovery, and the upgrades in between, measured on real clusters before anyone depends on them.
- Case study Sharding Postgres with Citus, measured against TimescaleDB and ClickHouse
- Write-up Citus removed the node. Its cleanup records stayed behind.
- Code db-failover-lab Six engines, eight failover scenarios, measured
-
System architecture with the recovery path designed first
Where state lives, what is idempotent, what a second delivery does, and how a release stops itself when it is bad — decided before the incident and written down as decisions you can revisit.
- Case study ClickHomes: an independent real-estate product
- Write-up My canary analysis aborted a good release and promoted a bad one
- Code platform-deployments GitOps with a canary that aborts bad releases
Findings
Measured, with the method in each write-up.
- p = 0.39 A 4.8-point eval drop that 12 changed answers out of 83 could not separate from noise The eval score fell 4.8 points. Twelve answers decided whether it meant anything.
- 3 of 37 auto-published agent answers were wrong, and no check in the pipeline could see them The agent got 48 of 54 right. Three of the wrong ones published themselves.
- 0.746 → 0.363 AUC-PR of the leaky fraud model: on validation, then on the next two weeks Validation picked the fraud model that had read the future
- 3.9 of 10 rows returned by post-filtered vector search at a 10% filter, with no error raised Filtered vector search returned four rows instead of ten, and nothing failed
- 59.8% of on-time events counted by 10-second windows in the worst broker kill, while every record stayed exact Exactly-once held through nine failures. The window counts did not.
- 33 retries of one Citus shard move, blocked by cleanup records for a node that no longer existed Citus removed the node. Its cleanup records stayed behind.
- 6 of 6 bad releases aborted by canary analysis, once it judged against stable and refused empty windows My canary analysis aborted a good release and promoted a bad one
- 0 breaking schema changes the contract checker passed as safe, across 51 labelled changes Removing an optional field breaks consumers. Adding one breaks history.
- 63% of a real trend recovered by counting a 95%-accurate classifier's labels The classifier was 95% accurate. The trend it counted was a third too flat.