News & Events

Preference-Guided Multi-Objective Learning: Covering and Targeting the Pareto Front

Lisha Chen, Assistant Professor, Electrical and Computer Engineering, University of Rochester

CSB 601

Abstract: Many learning systems have to balance competing objectives, such as helpfulness and harmlessness in LLM alignment, or reward and cost in decision making. No single model is best on every objective, so one question follows: which trade-offs on the Pareto front (PF) should we return to the users? This talk treats preference-guided multi-objective learning as distribution matching on the Pareto front in a unified manner. A user's preference is a target distribution over the front, and the task is to choose solutions which reproduce or represent that distribution. This leads to a bilevel structure: an outer loop steers the preference toward the target, and an inner loop solves the resulting preference-guided problem. We then study the two extreme cases of this framework. 1) When the user wants a balanced menu of trade-offs (a uniform target), we present SURF. SURF starts from the observation that uniformly sampled weights move along the PF at uneven speed, and it corrects for this by steering the weights through a learned inverse arc-length map. It provably converges to the error floor set by the number of samples. 2) When the user has a specific preference (a point-mass target), we formulate the problem as optimization over the Pareto set (OPS) and solve it with first-order methods. These methods stay tractable on nonconvex fronts, where linear scalarization fails. In experiments on multi-objective reinforcement learning and LLM alignment, SURF covers the front more uniformly, and the OPS methods reliably reach the preferred solutions. Together, these works offer unified, principled and computationally tractable tools for handling user preferences in modern multi-objective learning systems.

Bio: Lisha Chen is an Assistant Professor of Electrical and Computer Engineering at the University of Rochester. Her research lies at the intersection of machine learning, optimization, and statistical signal and image processing, with a current focus on the theory of multi-objective and multi-level optimization and learning. She develops foundations for scientific and engineering problems involving competing objectives, hierarchical decisions, and resource constraints, connecting theoretical insights with practical needs. Before joining the University of Rochester, Dr. Chen earned her Ph.D. and M.S. from Rensselaer Polytechnic Institute and her B.S. from Huazhong University of Science and Technology. She has delivered tutorials at ICASSP and AAAI and served as a reviewer or program committee member for leading venues in machine learning, signal processing, and optimization. Her honors include the IEEE Signal Processing Society Scholarship and the IBM AIRC Fellowship.