A probabilistic model is a joint distribution \(p(\mathbf{x}, \mathbf{z})\) of data \(\mathbf{x}\) and latent variables \(\mathbf{z}\). For background, see the Probabilistic Models tutorial.
In Edward, we specify models using a simple language of random variables. A random variable \(\mathbf{x}\) is an object parameterized by tensors \(\theta^*\), where the number of random variables in one object is determined by the dimensions of its parameters.
from edward.models import Normal, Exponential
# univariate normal
Normal(loc=tf.constant(0.0), scale=tf.constant(1.0))
# vector of 5 univariate normals
Normal(loc=tf.zeros(5), scale=tf.ones(5))
# 2 x 3 matrix of Exponentials
Exponential(rate=tf.ones([2, 3]))
For multivariate distributions, the multivariate dimension is the innermost (right-most) dimension of the parameters.
from edward.models import Dirichlet, MultivariateNormalTriL
# K-dimensional Dirichlet
Dirichlet(concentration=tf.constant([0.1] * K))
# vector of 5 K-dimensional multivariate normals with lower triangular cov
MultivariateNormalTriL(loc=tf.zeros([5, K]), scale_tril=tf.ones([5, K, K]))
# 2 x 5 matrix of K-dimensional multivariate normals
MultivariateNormalTriL(loc=tf.zeros([2, 5, K]), scale_tril=tf.ones([2, 5, K, K]))
Random variables are equipped with methods such as log_prob()
, \(\log p(\mathbf{x}\mid\theta^*)\), mean()
, \(\mathbb{E}_{p(\mathbf{x}\mid\theta^*)}[\mathbf{x}]\), and sample()
, \(\mathbf{x}^*\sim p(\mathbf{x}\mid\theta^*)\). Further, each random variable is associated to a tensor \(\mathbf{x}^*\) in the computational graph, which represents a single sample \(\mathbf{x}^*\sim
p(\mathbf{x}\mid\theta^*)\).
This makes it easy to parameterize random variables with complex deterministic structure, such as with deep neural networks, a diverse set of math operations, and compatibility with third party libraries which also build on TensorFlow. The design also enables compositions of random variables to capture complex stochastic structure. They operate on \(\mathbf{x}^*\).
from edward.models import Normal
x = Normal(loc=tf.zeros(10), scale=tf.ones(10))
y = tf.constant(5.0)
x + y, x - y, x * y, x / y
tf.tanh(x * y)
x[2] # 3rd normal rv in the vector
In the compositionality page, we describe how to build models by composing random variables.
ed.models.RandomVariable
ed.models.Autoregressive
ed.models.Bernoulli
ed.models.BernoulliWithSigmoidProbs
ed.models.Beta
ed.models.BetaWithSoftplusConcentration
ed.models.Binomial
ed.models.Categorical
ed.models.Cauchy
ed.models.Chi2
ed.models.Chi2WithAbsDf
ed.models.ConditionalDistribution
ed.models.ConditionalTransformedDistribution
ed.models.Deterministic
ed.models.Dirichlet
ed.models.DirichletMultinomial
ed.models.DirichletProcess
ed.models.Empirical
ed.models.ExpRelaxedOneHotCategorical
ed.models.Exponential
ed.models.ExponentialWithSoftplusRate
ed.models.Gamma
ed.models.GammaWithSoftplusConcentrationRate
ed.models.Geometric
ed.models.HalfNormal
ed.models.Independent
ed.models.InverseGamma
ed.models.InverseGammaWithSoftplusConcentrationRate
ed.models.Laplace
ed.models.LaplaceWithSoftplusScale
ed.models.Logistic
ed.models.Mixture
ed.models.MixtureSameFamily
ed.models.Multinomial
ed.models.MultivariateNormalDiag
ed.models.MultivariateNormalDiagPlusLowRank
ed.models.MultivariateNormalDiagWithSoftplusScale
ed.models.MultivariateNormalFullCovariance
ed.models.MultivariateNormalTriL
ed.models.NegativeBinomial
ed.models.Normal
ed.models.NormalWithSoftplusScale
ed.models.OneHotCategorical
ed.models.ParamMixture
ed.models.PointMass
ed.models.Poisson
ed.models.PoissonLogNormalQuadratureCompound
ed.models.QuantizedDistribution
ed.models.RelaxedBernoulli
ed.models.RelaxedOneHotCategorical
ed.models.SinhArcsinh
ed.models.StudentT
ed.models.StudentTWithAbsDfSoftplusScale
ed.models.TransformedDistribution
ed.models.Uniform
ed.models.VectorDeterministic
ed.models.VectorDiffeomixture
ed.models.VectorExponentialDiag
ed.models.VectorLaplaceDiag
ed.models.VectorSinhArcsinhDiag
ed.models.WishartCholesky
ed.models.WishartFull