Introduction to Neural Tangent Kernel

The neural tangent kernel (NTK) describes how parameter updates change a neural network’s predictions. This talk introduces the NTK through an exact gradient-flow identity and explains the assumptions under which it converges to a deterministic kernel that remains constant during training. We outline the main proof ideas and derive the resulting squared-loss dynamics, spectral convergence rates, and test-point predictions.
We then examine the scope of the frozen-kernel approximation, distinguishing finite-width effects from the role of parametrization and feature learning. A brief literature survey compares common kernel assumptions and their consequences. As an example beyond the initialization kernel, we discuss how class-block assumptions on output and feature tangent kernels lead to simplified dynamics and conditional neural collapse. We conclude with concrete questions about the formation and stability of this structure, and its relationship to representation geometry and population approximation.