| Abstract: |
Bilevel optimization, a class of hierarchical optimization problems with broad applications in machine learning and engineering, presents a significant research challenge due to its inherent NP-hard nature in the non-convex optimization problems. In this paper, we address the limitations of existing solvers. Classical, gradient-based methods are inapplicable to the non-convex and non-differentiable landscapes, common in practice. On the other hand, derivative-free methods like nested evolutionary algorithms are rendered intractable by a prohibitively high query complexity. To this end, we propose a novel framework, the Surrogate Assisted Co-evolutionary - Evolutionary Strategy (SACE-ES), which synergizes the global search capabilities of evolutionary computation with the data-driven efficiency of surrogate modeling. The core innovation of our framework is a multi-surrogate, constraint-aware architecture that decouples the complex bilevel problem by using separate Gaussian Process models to approximate the lower-level optimal solution vector and its corresponding constraint violations. We conducted a comprehensive empirical study on a suite of challenging benchmarks, including the standard SMD problems. We demonstrate empirically that on complex constrained problems, SACE-ES discovers solutions that are upto two orders of magnitude superior upper-level fitness values to those found by exhaustive search baselines. Furthermore, on a range of unconstrained, non-convex problems, our approach achieves statistically comparable solution quality while reducing the required computational cost by approximately upto 96%. Our results establish SACE-ES as a robust and highly efficient framework that makes previously intractable bilevel problems solvable, particularly in the presence of complex constraints. |