Deep dives into reinforcement learning, computer vision, and robotics research, complete with mathematical formulations and architectural breakdowns.
A PyTorch implementation of Proximal Policy Optimization (PPO) with a clipped surrogate objective, shared actor-critic network, Gaussian policy, and value/entropy losses.
A PyTorch implementation of Soft Actor-Critic (SAC) with a stochastic squashed Gaussian policy, twin Q-networks with target networks, automatic temperature tuning, and a replay buffer for continuous control.
A PyTorch implementation of Neural Radiance Fields (NeRF): positional encoding for positions and view directions, MLP with skip connections, density/color prediction, stratified sampling, and volumetric rendering.
A PyTorch implementation of Rotary Position Embedding (RoPE): inverse frequency buffer, precomputed cos/sin cache, query/key rotation, and shape verification.
A minimal PyTorch implementation of a Denoising Diffusion Probabilistic Model (DDPM): sinusoidal time embeddings, U-Net with skip connections, forward noising, reverse denoising, and ancestral sampling.
Master NeRF (Neural Radiance Fields): 5D radiance fields, differentiable volume rendering, Fourier positional encoding, hierarchical sampling, and PyTorch.
A compact PyTorch implementation of a Transformer encoder with multi-head self-attention, sinusoidal positional encoding, feed-forward layers, residual connections, and padding masks.
Master LTX-2: asymmetric dual-stream video/audio DiT, 1D/3D RoPE alignment, cross-modality adaLN, multimodal guidance (CFG/STG), and PyTorch pipelines.
A PyTorch implementation of Direct Preference Optimization (DPO): a small LSTM language model, a preference dataset, and a DPO trainer that optimizes a policy against a frozen reference via the sigmoid logistic DPO loss on chosen/rejected pairs.
A PyTorch implementation of Group Relative Policy Optimization (GRPO): a Group-relative RL approach that partitions sorted trajectories into groups, weights group-relative advantages, and applies clipped surrogate updates with PPO-style ratios.
A PyTorch implementation of Deep Deterministic Policy Gradient (DDPG): an Actor that outputs tanh-bounded deterministic actions, a Q-value Critic over state-action pairs, a replay buffer, target networks with soft updates, and a smoke test that exercises action selection and a short training loop.
A PyTorch implementation of LLaVA: a ResNet-50 vision encoder, a Transformer-based language model, a custom cross-attention fusion layer, and an enhanced tokenizer for image-grounded text generation.
Master Diffusion Transformers (DiT): latent patchification, adaLN-Zero conditioning, multi-modal MMDiT joint attention, scaling laws, and PyTorch pipelines.
A PyTorch implementation of the Diffusion Transformer (DiT): a patch-embedded Vision Transformer with sinusoidal timestep conditioning, multi-head self-attention, MLP blocks, and a transposed-conv head that predicts the image-space noise.
A PyTorch implementation of Deep Q-Network (DQN): a Q-value MLP, a replay buffer, an ε-greedy agent with target network, Bellman loss update, and a CartPole-v1 training/test loop using Gymnasium.
A PyTorch implementation of Low-Rank Adaptation (LoRA): low-rank A/B matrices wrapped around a frozen linear layer, scaled by alpha/rank, with shape, freezing, and rank-property tests.
A PyTorch implementation of a Denoising Diffusion Probabilistic Model (DDPM): a linear-beta forward diffusion, a reverse sampling loop, a time-conditioned UNet denoising network, and a training-free smoke test that forward-diffuses, predicts noise, and samples images.
A PyTorch implementation of FlashAttention-style multi-head attention: a softmax-scaled dot-product attention block, a tiled/blocked variant that processes sequence chunks for memory efficiency, and basic shape tests.
A PyTorch implementation of the Mixture of Experts (MoE) layer with a top-k gating network, multiple expert MLPs, and weighted expert outputs for sparse expert routing.
A minimal PyTorch ControlNet implementation: a control encoder with conv blocks, residual skip connections, channel attention, and a zero-initialized projection that adds residual control features into a backbone so that the model behaves as identity at init.
A PyTorch implementation of a Denoising Diffusion Implicit Model (DDIM) sampler: a UNet denoiser with GELU time conditioning, an eta-controlled stochastic-to-deterministic reverse process on a sub-sequence of timesteps, and a smoke test that verifies noise addition, model output, sampling shapes, and deterministic reproducibility.
A PyTorch implementation of a Denoising Diffusion Probabilistic Model (DDPM): a UNet denoiser with sinusoidal-free linear time conditioning, a linear-beta forward diffusion, an iterative reverse sampling loop, and a smoke test that verifies noise addition, model output, and sampling shapes.
Master PagedAttention: virtual memory KV cache paging, block tables, prefix caching (CoW), continuous batching, memory fragmentation math, and PyTorch kernels.
A PyTorch implementation of the DeepSeek-style architecture: RMSNorm pre-norm blocks, a multi-head latent attention module, a Mixture-of-Experts (MoE) feed-forward router, and a stack of DeepSeek blocks with token-level residual paths.
Master DDIM (Denoising Diffusion Implicit Models): non-Markovian sampling, probability-flow ODE derivations, deterministic inference, and PyTorch pipelines.
A PyTorch implementation of the Vision Transformer (ViT) with patch embedding, learnable class token and position embeddings, multi-head self-attention, MLP blocks, and classification head.
Master the Vision Transformer (ViT): patch projection, class tokens, multi-head self-attention over patches, DeiT augmentation, and PyTorch implementations.
A from-scratch PyTorch implementation of the LLaMA-style Transformer: RMSNorm, Rotary Position Embeddings (RoPE), GQA-style multi-head attention with causal mask, SwiGLU MLP, and a stack of pre-norm decoder layers.
Master RoPE (Rotary Position Embedding): complex rotation algebra, relative distance invariance proof, long-context scaling (YaRN), and PyTorch kernels.
Master DDPM: forward and reverse Gaussian Markov chains, closed-form ELBO derivation, noise-prediction parameterization, and PyTorch training pipelines.
A PyTorch implementation of 3D Gaussian Splatting from scratch: per-Gaussian learnable positions, scales, quaternions and RGBA, 3D covariance via R S Sᵀᵀ, projection to 2D via the perspective Jacobian, splat-based alpha compositing over a pixel grid, and a smoke test that checks initialization, symmetry/orthogonality of covariances, and the rendering output shape and value range.
Master diffusion models: forward Markov perturbations, reverse ELBO derivations, score-based SDE unifications, and production PyTorch training pipelines.
A PyTorch implementation of Asynchronous Advantage Actor-Critic (A3C): a shared Actor-Critic network with a softmax policy head, multiple Worker processes that collect CartPole trajectories, compute n-step returns and advantages, and sync gradients into a global model, plus a greedy test harness that runs the trained policy in render mode.
Master 4D Gaussian Splatting (4DGS): HexPlane spatiotemporal neural fields, canonical deformation fields, and real-time dynamic novel view synthesis.
Master 3D Gaussian Splatting (3DGS): covariance matrix decomposition, tile-based differentiable rasterization, adaptive density control, and real-time CUDA pipelines.
Master LLaVA: visual instruction tuning, 2-stage cross-modal projection, CLIP/SigLIP feature alignment, AnyRes scaling, and PyTorch VLM pipelines.
Master Stable Diffusion: latent diffusion models (LDM), perceptual VAE compression, cross-attention U-Net, classifier-free guidance, and PyTorch pipelines.
Master LoRA (Low-Rank Adaptation): low-rank matrix decomposition, intrinsic rank analysis, QLoRA NF4 quantization, DoRA, and PyTorch PEFT pipelines.
Master ControlNet: zero-convolution weight initialization, trainable U-Net copy architecture, multi-condition composition, and PyTorch training pipelines.
Master 3D Transformers: vector self-attention on point clouds, sparse voxel hashing, 3D Swin windowing, relative positional bias, and PyTorch architectures.
Master Llama architecture: RMSNorm pre-normalization, SwiGLU activations, RoPE relative embeddings, Grouped-Query Attention (GQA), and PyTorch pipelines.
Master the Transformer: scaled dot-product attention, multi-head projections, sinusoidal & RoPE encodings, Pre-LN vs Post-LN stability, and PyTorch engines.