Kolmogorov-Arnold Networks versus MLPs: Expressiveness and Performance Trade-offs

Abstract

Multi-layer perceptrons (MLPs) have served as the default nonlinear function approximator in deep learning for decades, relying on fixed activation functions applied to learned linear combinations of inputs. Kolmogorov-Arnold Networks (KANs), inspired by the Kolmogorov-Arnold representation theorem, propose a fundamentally different architecture in which learnable univariate functions are placed on network edges rather than nodes, replacing linear weights with parameterized spline functions. This paper provides a rigorous comparative analysis of KANs and MLPs across theoretical expressiveness, empirical accuracy, computational efficiency, and interpretability. We review the mathematical foundations of both architectures, including universal approximation guarantees and convergence rates. Through systematic benchmarking on regression, classification, and scientific computing tasks, we demonstrate that KANs achieve superior accuracy-per-parameter on smooth, low-dimensional function approximation problems while MLPs retain advantages in high-dimensional, large-scale settings. We analyze the computational overhead of spline-based edge functions, discuss training stability considerations, and examine KAN extensions including efficient variants and physics-informed formulations. Our findings suggest that KANs and MLPs occupy complementary niches in the neural architecture design space.