Composite NUV Priors and Applications

Normal with unknown variance (NUV) priors are a central idea of sparse Bayesian learning and allow variational representations of non-Gaussian priors. More specifically, such variational representations can be seen as parameterized Gaussians, wherein the parameters are generally unknown. The advantage is apparent: for fixed parameters, NUV priors are Gaussian, and hence computationally compatible with Gaussian models. Moreover, working with (linear-)Gaussian models is particularly attractive since the Gaussian distribution is closed under affine transformations, marginalization, and conditioning. Interestingly, the variational representation proves to be rather universal than restrictive: many common sparsity-promoting priors (among them, in particular, the Laplace prior) can be represented in this manner. In estimation problems, parameters or variables of the underlying model are often subject to constraints (e.g., discrete-level constraints). Such constraints cannot adequately be represented by linear-Gaussian models and generally require special treatment. To handle such constraints within a linear-Gaussian setting, we extend the idea of NUV priors beyond its original use for sparsity. In particular, we study compositions of existing NUV priors, referred to as composite NUV priors, and show that many commonly used model constraints can be represented in this way.