Machine Learning Thesis Proposal

Convex optimization has developed a wide variety of useful tools critical to many applications in machine learning. However, unlike linear and quadratic programming, general convex solvers have not yet reached sufficient maturity to fully decouple the convex programming model from the numerical algorithms required for implementation. Especially as datasets grow in size, there is a significant gap in speed and scalability between general solvers and specialized algorithms.

This thesis addresses this gap with a new model for convex programming based on an intermediate representation of convex problems as a sum of functions with efficient proximal operators. This representation serves two purposes: 1) many problems can be expressed in terms of functions with simple proximal operators, and 2) the proximal operator form serves as a general interface to any specialized algorithm that can incorporate additional l2-regularization. On a single CPU core, numerical results demonstrate that the sum-of-prox form results in significantly faster algorithms than existing general solvers based on conic forms. In addition, splitting problems into separable sums is attractive from the perspective of distributing solver work amongst multiple cores and machines. We develop a system that scales to 100s of CPU cores and gigabyte-scale data, enabling general convex programming frameworks to be applied a much larger class of problems.

We apply large-scale convex programming to several problems arising from building the next-generation, information-enabled electrical grid. In these problems (as is common in many domains) large, high-dimensional datasets present opportunities for novel data-driven solutions. We present approaches based on convex models for several problems: probabilistic forecasting of electricity generation and demand, model predictive control for device energy management and source separation for whole-home energy disaggregation.

Thesis Committee:
J. Zico Kolter (Chair)
Geoff Gordon
Ryan Tibshirani
Stephen Boyd (Stanford University)
Arunava Majumdar (Stanford University)

Copy of Draft Document




Source: Machine Learning Thesis Proposal

Via: Google Alert for ML