We propose a new framework of optimization with affine sparsity constraints for model selection in statistical learning. Our motivations come from statistical variable selection with certain complicating logical conditions, including the hierarchical variable selection and group variable selection. Affine sparsity constraints are defined by a composition of linear system of inequalities and a binary indicator mapping of continuous variables. They serve as an extension of the (well-studied) cardinality constraint. We study a few fundamental structural properties of the feasible region defined by affine sparsity constraints, including its closedness and characterizations of closure, set convergence (in Hausdorff distance) of continuous approximations, and characterizations of tangent cones. These results lay a mathematical foundation for solving such problems through their continuous approximations.

This is a recent collaborated work with Jong-Shi Pang (University of Southern California) and Miju Ahn (USC).