Blog

Deep science. Built from scratch. Shared openly.

Why Cross-Entropy Loss Can Never Reach Zero

Why softmax can’t reach the corners of the simplex, and what that does to the predictor

Machine Learning

Deep Learning

Optimization

Why Autograd Needs Floats, Not Ints

Calculus is continuous — and integer tensors quietly kill the gradient signal

Deep Learning

Machine Learning

Make ML Systems Ship Again

A practitioner’s guide to finding and fixing the one bottleneck that governs your system’s performance.

ML Systems

Hard-Learned Lessons in Shipping Software (AI/ML) Projects

A Guide for Engineers and Product Managers

Machine Learning

Deep Learning

Software Engineering

From Forgetting to Fluency: How to Learn Smarter, Not Harder

Evidence-based tips from cognitive science to transform the way you absorb, retain, and apply knowledge in any field.

Problem Solving

Why Your Final Layer Shouldn’t Have Softmax

The log-sum-exp trick, numerical instability, and when raw logits are the right choice

Deep Learning

Cutting the Fat: A Practical Guide to Neural Network Pruning

From weight sparsity to hardware-aware strategies—learn how to shrink deep learning models without shrinking their performance

Deep Learning

Building GPT(2/3) from Scratch: Turning Theory into a Working Transformer

A hands-on journey through implementing the 124M GPT architecture in PyTorch—complete with attention mechanisms, optimizations, and the lessons learned along the way

NLP

Byte Pair Encoding from Scratch

Building a BPE tokenizer step by step — the algorithm that decides how language models see text

NLP

The RAG Optimization Playbook

Proven tactics, pitfalls, and fine-tuning methods to build faster, smarter, and more accurate retrieval-augmented generation systems

NLP

Inside Python’s Modules and Packages: The Machinery Behind import

Explore the inner workings of Python’s import system, including path resolution, package types, importlib internals, and advanced import hacks

Software Engineering

Automatic Differentiation Demystified

From dual numbers to backpropagation — the intuition, the trade-offs, and what breaks in practice for LLM training

ML Systems

Git from the Inside Out

An intuition-first, reference-grade guide to Git’s object model, branching mechanics, history rewriting, and the professional workflows that tie it all together

Software Engineering

I Built My Own PyTorch (Tiny Version) — Here’s Everything I Learned

Inside the engineering decisions, optimizations, and trade-offs behind a homegrown deep learning framework

ML Systems

Breaking Text Apart (The Smart Way)

From single characters to advanced subword splits — see how modern tokenizers like WordPiece and SentencePiece prepare language for AI.

NLP

C Program Startup

Does C program really start at main?

Software Engineering

The Transformer Architecture: A Deep Dive

An intuition-first, reference-grade breakdown of every building block—from why attention was invented to how encoder, decoder, and encoder-decoder architectures work

NLP

Inside LSTMs: Implementing and Optimizing Sequential Models from First Principles

A deep dive into LSTM internals—covering the math, gates, performance considerations, and a full PyTorch-aligned implementation from scratch.

NLP

Anomaly Detection

Machine Learning

Gradient Descent Algorithm and Its Variants

Deep dive into gradient descent algorithm: Batch vs. Mini-batch vs. Stochastic.

Deep Learning

Machine Learning

Optimization

Conda Essentials

Enough background about Conda to be productive!

Software Engineering

K-means Clustering: Algorithm, Applications, Evaluation Methods, and Drawbacks

Deep dive into K-means algorithm to find subgroups within data.

Machine Learning

Coding Neural Network Part 5 - Dropout

What is Dropout, its use, and how to implement it?

Deep Learning

Coding Neural Network Part 4 - Regularization

What is regularization and how it helps NN generalizes better?

Deep Learning

Coding Neural Network Part 3 - Parameters’ Initialization

The role of parameter initialization in training and different ways to initialize parameters.

Deep Learning

Coding Neural Network Part 2 - Gradient Checking

How to check numerically if the implementation of backward propagation is correct?

Deep Learning

Coding Neural Network Part 1 - Forward & Backward Propagation

What it takes to go from input to output? And how to compute the gradients?

Deep Learning

Bandit Algorithms: epsilon-Greedy Algorithm

What is epsilon-Greedy Algorithm and how to use it in A/B testing?

Machine Learning

Predicting Loan Repayment

Trying different modeling techniques to deal with imbalanced data, missing values, and ensemble models.

Machine Learning

Character-Level Language Model

How an RNN learns to generate names one character at a time — and what it teaches us about language models

NLP

Predicting Employee Turnover

Experimenting with different models on employee turnover data.

Machine Learning