Nirmit K. Tripathii | AI Engineering & Architecture Lab

Flagship Project 1

Indic AI Dubbing Platform: Isochrony & Voice Cloning

Phoneme-duration constrained translation across 11 Indic languages cutting syllable expansion by 79% for natural cinematic lip-sync.

📖 5,000-Word Build Log 🐙 GitHub Repo

🎙️ Syllable Duration & Lip-Sync Comparator Audio Clip: science_explainer_clip01.mp4

● 1. English Source Reference (EN) 6.50s (Target Budget)

"Electrons orbit around the atomic nucleus at quantum states."

● 2. Raw Hindi MT (Unconstrained Llama/NLLB) 11.80s (+81% Overrun · Lip-Flap Broken)

"इलेक्ट्रॉन परमाणु नाभिक के चारों ओर क्वांटम अवस्थाओं में चक्कर लगाते हैं..." (Over-expanded)

● 3. v3 QLoRA Isochrony Constrained (Nirmit's Model) 6.48s (Δ = -0.02s · Perfect Lip-Sync)

"इलेक्ट्रॉन क्वांटम स्तरों पर नाभिक की परिक्रमा करते हैं।" (Cadence Matched)

1. Audio Demux

Demucs v4

2. Syllable Estimator

Indic Syllabifier

3. Budget QLoRA

Llama-3.1-8B-v3

4. Voice Clone

IndicF5-TTS

Empirical Benchmark Metrics

Length Error (Rel): 0.495 → 0.103 (-79%)

Translation Quality (chrF++): 22.0 → 30.1 (+36.8%)

Voice Cloning Latency: 1.2x Real-Time

Supported Indic Languages: 11 Languages

💡 Recruiter / Interview Pitch:

"Demonstrates deep fine-tuning proficiency (QLoRA with custom loss penalties on sequence length), multimodal audio stem separation, and production-grade TTS synthesis without resorting to paid third-party voice APIs."

Flagship Project 2

Email Orchestrator: Multi-Account MCP Super-Server

Unified Model Context Protocol (MCP) server aggregating Gmail, Zoho, Yahoo, and Outlook into a self-healing multi-agent reasoning hub.

📖 Architecture Log 🛡️ Resilience Log

mcp-email-daemon · stdio JSON-RPC

17 Unified MCP Tools Active

> mcp_orchestrator.initialize(providers=["gmail", "zoho", "yahoo", "outlook"])

✔ [Gmail] Connected via OAuth2 (200 OK · 3 unread priority threads)

✔ [Zoho] Connected via IMAP SSL (200 OK · 1 invoice pending)

✔ [Outlook] Child process PID: 4182 listening on stdio

> mcp_orchestrator.fanout_query("action_required: true, deadline: '< 24h'")

↳ Fanout completed across 4 accounts in 38ms · 2 actionable items identified.

49 Unit Tests Passing (100%)

Architectural Innovations

MCP Server-as-Client Recursion Acts as unified super-server while multiplexing to child MCP clients.

Self-Healing Child Watchdog Detects dead sockets and spawns fresh sub-daemons in < 45ms.

Thinking Budget Optimizer Calculates token ROI per email draft to trim inference cost 68%.

💡 Recruiter / Interview Pitch:

"Demonstrates cutting-edge mastery of Anthropic's Model Context Protocol (MCP), asynchronous Node/TypeScript process supervision, and building resilient AI agent infrastructure."

Civic Full-Stack AI

India आसपास: Hyperlocal Civic Issue Reporting

Production platform for crowd-mapping potholes, garbage, and drainage across Noida & NCR with automated image verification.

🚀 Open Live Production App → 📄 White Paper

📍 Noida Sector 62 / Sector 59 / Mamura Live Pin Hub OpenStreetMap API

Selected Incident: Open pothole near Fortis Hospital (Sector 62) ✔ Verified by Community (Score: 12)

Production Infrastructure

Hosting & DB: Render + PostgreSQL

Total Monthly Spend: ~$3 / month

Social Card Generation: Server-Side Canvas SSR

Trust Model: 2-Tier Verification

💡 Recruiter / Interview Pitch:

"Demonstrates building real-world civic impact technology with hardened uploads, server-side caching, spatial GIS coordinates, and extreme cloud-cost frugality."

Autonomous Multi-Agent

Podcast Shorts Factory: 10-Agent Autonomous Studio

Autonomous swarm of 10 specialized LLM agents turning long-form podcasts into viral short clips with automatic QA gates.

📄 White Paper 🐙 GitHub Repo

🤖 10-Agent Sequential & Parallel State Machine MIT Licensed · $0 Cloud Stack

Agent 01

📥 Ingest

Agent 02

📝 Whisper STT

Agent 03

🔥 Hook Detector

Agent 04

⏱️ Boundaries

Agent 05

✂️ Auto-Crop

Agent 06

💬 Subtitler

Agent 07

🖼️ B-Roll Gen

Agent 08

🛡️ QA Gatekeeper

Agent 09

📅 Scheduler

Agent 10

📊 Analytics Loop

Failover Route: Groq (Llama-3.3-70B) → Gemini 1.5 Flash → Local Ollama ✔ Unattended Execution

Agentic Orchestration Edge

Automated QA Gate Rejects clips with poor hook scores or audio misalignment before publishing.

Learning Feedback Loop Feeds YouTube retention telemetry back into the prompt optimizer.

💡 Recruiter / Interview Pitch:

"Demonstrates multi-agent swarm design, deterministic QA feedback gates, and building full media automation systems that run self-reliantly for $0."

Resilient Generative AI

AI Video Style Cloner: $0 Self-Healing Media Pipeline

Reverse-engineers creator pacing, script structure, and visual hooks with a 3-tier LLM failover and AST schema repair.

📄 Architecture White Paper

🎬 Pacing Ingest: youtube.com/@veritasium (148 WPM · Hooked) Self-Healing AST in 18ms

Tier 1: Multi-LLM Cascade Groq (Llama-3.3-70B) → Gemini 1.5 Flash (200 OK)

Active Route

Tier 2: JSON AST Structural Validator Repaired unescaped quote in script payload in 18ms without re-querying API

Auto-Repaired

Tier 3: MoviePy Compositor & Edge-TTS Rendered 1080p MP4 with animated subtitles & procedural visual backdrops

$0 Cost Stack

✔ Zero-Network Procedural Fallback

Production Resilience Features

Zero-Cost Infrastructure Combines generous free tiers with local headless render pipelines.

AST Self-Healing Fixes malformed LLM JSON stream anomalies locally in 18ms.

💡 Recruiter / Interview Pitch:

"Demonstrates building unbreakable AI applications that never crash on end-users even when upstream LLM providers return 429 or 500 errors."

Multilingual GraphRAG

IKS Oracle — Gyanstrot: Indian Knowledge Systems

Semantic retrieval engine and 3D Knowledge Star Map navigating 14,205 Sanskrit sutras, Upanishads, and mathematics treatises.

🚀 Open Live Space → 🐙 Space Repo

🔱 Multilingual Vector Graph: LightRAG + Sanskrit-MiniLM 14,205 Nodes Indexed

Sanskrit Semantic Query: 0.968 Cosine Similarity

"बिम्बार्धस्य फलं..." (Siddhanta-Siromani, Grahaganita)

→ Explores instantaneous motion (Tatkalika Gati) and early differential calculus for planetary positions.

Architecture: Gemini 1.5 Pro + LightRAG Multi-Hop Graph Reranker Satyamev Jayate

Research & NLP Competencies

Sanskrit Morphological Parsing Handles complex compound sandhi splitting and ancient grammatical declensions.

3-Hop Knowledge Star Map Discovers cross-disciplinary links between Vedic mathematics and Ayurvedic medicine.

💡 Recruiter / Interview Pitch:

"Demonstrates building specialized domain GraphRAG systems in non-English low-resource languages with complex graph retrieval schemas."

Sports AI & Spatial Modeling

VAEP Football Action Valuation (StatsBomb & Wyscout)

Probabilistic framework valuing all on-ball events across Top 5 European Leagues, World Cup 2018, and Indian Super League (ISL).

🚀 Open Live Space → 🐙 GitHub Repo

⚽ Match: Bengaluru FC vs Chennaiyin FC (Goal 11') +0.742 Total VAEP Value

1. Pass (Udanta)

2. Cross (Roshan)

⚽ Goal (Basafa)

Formula: V(a) = ΔP(scores) − ΔP(concedes) Offensive: +0.754 | Defensive: -0.012

Data Engineering Features

StatsBomb 360 Freeze-Frames Calculates defensive density around ball carrier at moment of pass.

LLM Tactical Reporter Synthesizes coach-level tactical summaries with Groq Llama-3.

💡 Recruiter / Interview Pitch:

"Demonstrates mastery of complex spatial analytics, SPADL event streams, Markov-based Expected Threat (xT), and modern sports analytics architectures."

Supervised ML

Shot Quality Expected Goals (xG) Simulator

XGBoost classifier predicting probability of goal conversion using Euclidean distance, Law of Cosines angles, and foot type.

🎯 Try Live Simulator → 🐙 GitHub Repo

🎯 Click Anywhere on Attacking Half to Calculate xG 0.785 xG · High Prob

Distance to Goal:

11.5 meters

Law of Cosines Angle:

36.8°

Shot Conversion xG:

0.785

Mathematical Formulations

Law of Cosines Geometry Calculates exact visual angle between posts (100,46) and (100,54).

Body Part Weighting Strong Foot (2.0x), Weak Foot (0.6x), Headers (0.4x).

💡 Recruiter / Interview Pitch:

"Demonstrates classic machine learning feature engineering, spatial trigonometry, and deploying low-latency model inference endpoints."

Dense NLP Embeddings

AI Resume Matcher: SBERT Semantic Radar & Skill Gap

Sentence-Transformers (all-MiniLM-L6-v2) vector matching + O(n) precompiled regex skill extraction.

🔬 Open Full Matcher → 🐙 GitHub Repo

🎯 SBERT Cosine Similarity & Gap Matrix 91.4% Cosine Match

91.4%

Cosine Match

Skill Gap Analysis (O(n) Regex):

✔ LangGraph (98%) ✔ PyTorch (94%) ✔ Indic NLP (91%) ✔ Docker / MLOps (95%) ✘ Missing: Rust

Optimization: Pre-loaded Model in Worker RAM + Compiled Regex 24ms Latency

Engineering Optimizations

Pre-Warmed SBERT Model stays resident in memory to eliminate 3s cold-start on incoming requests.

💡 Recruiter / Interview Pitch:

"Demonstrates building high-speed semantic document matchers using state-of-the-art transformer embeddings with memory-optimized inference."

Computer Vision & DVC

Plant Disease Classification: Deep Agro-Vision Scanner

CNN vision classifier on PlantVillage dataset with DVC data pipelines and automated treatment prescriptions.

🌿 Live Scanner Demo → 🐙 GitHub Repo

🌿 Target: Solanum lycopersicum (Tomato) 98.4% Confidence

🍃

Tomato Early Blight (Alternaria solani) 98.4%

Late Blight (Phytophthora) 1.2%

💊 Rx: Copper Oxychloride 50 WP (2.5g/L) · 7-day spray cycle

MLOps: DVC Data Pipelines (dvc repro) + Flask + Docker 14ms Edge Inference

MLOps Architecture

DVC Pipeline Reproducibility Tracks dataset versioning, model checkpoints, and hyperparameter metrics.

💡 Recruiter / Interview Pitch:

"Demonstrates computer vision classification combined with disciplined MLOps engineering practices using Data Version Control (DVC)."

Low-Latency API Gateway

Global Weather Dashboard & Throttled Proxy

In-memory cached API proxy with Flask-Limiter safeguarding third-party quota caps with 12ms responses.

🌤️ Check Weather Live → 🐙 GitHub Repo

🛰️ Live Telemetry: OpenWeatherProxy Gateway Cache HIT · 12ms

⛅

28°C

Partly Cloudy · Noida

Wind Velocity: 14 km/h NW

Relative Humidity: 62%

Barometric Pressure: 1013 hPa

Rate Limits: 1000/day · 60/hr · 10/min per IP 97% Upstream Bandwidth Saved

Gateway Architecture

TTL In-Memory Cache 10-minute cache window reduces external API bill to zero.

💡 Recruiter / Interview Pitch:

"Demonstrates building robust proxy gateways that protect underlying services against denial-of-service and quota exhaustion."