| The optimization of objective functions has long been one of the important and interesting problems in the modeling, simulation and verification of many physical processes. However, practical instances of most problems (e.g. in the areas of computer vision, VLSI circuit design, operations research, etc.), often involve large-scale nonlinear, nonconvex objective functions making the derivation of efficient techniques a challenging (and sometimes an impossible) effort.; This dissertation addresses the task of how to consistently and computationally efficiently solve problems governed by a class of objective functions associated with relaxation-based neural networks. We introduce a novel optimization framework involving a combination of techniques: deterministic annealing, clocked objectives, multiscale optimization, attention mechanisms, trust region optimization, and a pre-conditioned conjugate-gradient method.; The optimization framework is applied to representative objective functions from two research areas: computer vision and combinatorial optimization. The first problem is the 2D image segmentation problem, and the second the graph multi-partitioning problem. Large-scale objective function formulations of these problems are being solved, of {dollar}O(10sp4){dollar} to nearly {dollar}O(10sp6){dollar} size for the former problem, and up to {dollar}O(10sp4){dollar} size for the latter. Finally, the relaxation dynamics for partitioned neural networks is examined, which allows us to map the 2D image estimation and segmentation objective function onto a parallel computing environment, and to model the slower inter-module communication channels by means of certain fixed-point-preserving algebraic transformations. |
