Optimization in the Small-Data, Large-Scale Regime

Abstract

This chapter introduces the small-data, large-scale optimization regime, an asymptotic setting that arguably better describes certain data-driven optimization applications than the more traditional large-sample regime. We highlight unique phenomena that emerge in the small-data, large-scale regime and show how these phenomena cause certain traditional data-driven optimization algorithms like sample average approximation (SAA) to fail. We then propose a new debiasing approach that has provably good performance in this regime, highlighting a new path forward for research and development into these types of applications.