Aaron Leevord Fernandes
Full-stack AI software engineer, ML engineer, and data scientist. I take AI systems end to end, from data and models to the backend, infrastructure, and product they run in.
I work across the full stack of AI systems: training models in PyTorch, building the data and inference pipelines around them, and writing the backends, APIs, and interfaces that turn them into products people can actually use. That spans generative models, computer vision, reinforcement learning, LLM and retrieval systems, and applied data science, with a real focus on performance, scalable training, and production deployment (ONNX, CoreML, Docker, AWS, FastAPI, vLLM).
Currently a research assistant on synthetic driving data for autonomous systems at Indiana University. I came to AI from software engineering (production backend work in enterprise banking before a master's in AI), so I am comfortable owning a system from data and experiment through to deployment and the product around it.
- 2026 – now Graduate Research Assistant, Astemo × Indiana University, working on generative models for autonomous driving: image editing & generation, video / driving world models, vLLM-Omni multimodal inference, and geometry & computer-vision benchmarking to validate synthetic data. Earlier in 2026: backend engineering at Heartland.
- 2024 – 2025 ML engineering & research, spanning computer vision at Leung Research Group and reinforcement learning at Indiana University.
- 2024 NCAA Women's Basketball Prediction, a Random Forest tournament model; ranked top 5 of 67 in a national competition.
- 2023 – 2025 M.S. Intelligent Systems Engineering, Indiana University Bloomington (GPA 3.9). The pivot into AI: computer vision, diffusion, reinforcement learning.
- 2022 – 2023 Software Engineer, Tata Consultancy Services, building Java / Spring Boot microservices for HDFC Bank (100K+ daily transactions).
- 2017 – 2021 B.S. Electronics & Telecommunications, Goa College of Engineering. Networking & IoT; built a fuzzer for the Contiki IoT operating system.
A real-time voice agent for talking to and searching across a whole video collection: you speak, it watches, answers questions about what is on screen, and surfaces the moment or scene you ask for. Browser mic to speech-to-text, a routing layer, a vision-language model, and streamed text-to-speech, with barge-in so you can interrupt mid-sentence. A zero-temperature router sends every utterance to the right specialist, so the model that can actually see answers what-is-visible questions instead of the text model guessing from memory, while retrieval questions go to hybrid Graph-RAG over FalkorDB (vector KNN plus graph hops in one Cypher query). The demo runs on nuScenes driving footage, with ego- and lead-vehicle motion read from telemetry as ground truth rather than pixels. Four models are self-hosted with vLLM across university GPUs over SSH tunnels.
An LLM-assisted retrieval and knowledge-graph system for grounded-theory qualitative research, built with researchers at NICC Brussels. Extracts knowledge graphs from interview transcripts and supports source-traceable querying across 200+ documents, pairing vector embeddings with graph-backed retrieval so every answer traces back to its origin passage.
ComicPaliGemma: a Vision-Language Model from scratch
A from-scratch implementation of Google's PaliGemma VLM (SigLIP ViT vision encoder plus a Gemma decoder), validated against the full pretrained checkpoint. Fine-tuned on 5K+ comic panels with 4-bit QLoRA to cut peak VRAM by 65%, with a long-context autoregressive generator producing coherent narration across 10+ panels. Custom Triton / FlashAttention-2-style kernels gave 5 to 10× faster inference than standard PyTorch attention.
A from-scratch latent Diffusion Transformer trained with rectified flow on 2M LAION-Aesthetics images. Attention, AdaLN-Zero conditioning, patchify, the rectified-flow objective, Euler sampler, and classifier-free guidance are all hand-written (~600 lines). A profiler-driven attention fix gave 2.2× throughput and 2× less memory; streaming data pipeline measured at 757 imgs/s over 1,976 WebDataset shards.
A small, readable diffusion repo that starts from a vanilla DDPM ε-prediction model and moves toward modern diffusion in minimal additive steps. U-Net with timestep embeddings and selective self-attention on CIFAR-10, EMA, checkpointing, and FID reporting every 10k steps, built to be instrumented and legible rather than a heavy framework.
Unsupervised Industrial Anomaly Detection
A Vision Transformer autoencoder with a Capsule Network and Gaussian mixture scoring for detecting and localizing defects in industrial imagery. Trained on the BeanTech (BTAD) dataset, improving AUROC from 0.78 to 0.94 over ResNet-50 baselines.
Goal-conditioned reinforcement learning combining Soft Actor-Critic, Hindsight Experience Replay, and Stein Variational Goal Generation for curriculum-style goal sampling in sparse-reward robotics environments like FetchReach. Hydra-configured, W&B-tracked.
A model-based RL algorithm inspired by Deep Value-and-Predictive-Model Control: learned dynamics, value approximation, and Cross-Entropy Method planning over a prediction horizon for continuous control. Effective in dense-reward settings, with sparse rewards analyzed as the open challenge.
Apr 2024 – Dec 2025
Machine Learning Engineer, Computer Vision
Leung Research Group
Self-supervised vision and computational imaging for OCT video-to-histology translation. Built U-Net surrogate models approximating FDTD optical simulations on HPC (SLURM/Linux), cutting inference from ~3 min to sub-second while holding SSIM > 0.85.
Jan 2025 – Jun 2025
Machine Learning Engineer, Reinforcement Learning
Indiana University
Goal-conditioned RL (HER, curriculum policy optimization) for sparse-reward navigation, with experiment infrastructure orchestrating 100+ multi-seed runs via Hydra, MLflow, and W&B.
Mar 2026 – present
Graduate Research Assistant, Synthetic Driving Data
Indiana University (Astemo-sponsored)
Building and benchmarking generative systems for autonomous driving: diffusion-based image editing and generation, video generation, and driving world models for synthetic data. Serving large-scale multimodal inference with vLLM-Omni, and applying geometry- and computer-vision-based benchmarking and validation to ensure the generated data is good enough to train on.