API and Documentation

Model

A probabilistic model is a joint distribution \(p(\mathbf{x}, \mathbf{z})\) of data \(\mathbf{x}\) and latent variables \(\mathbf{z}\). For background, see the Probabilistic Models tutorial.

In Edward, we specify models using a simple language of random variables. A random variable \(\mathbf{x}\) is an object parameterized by tensors \(\theta^*\), where the number of random variables in one object is determined by the dimensions of its parameters.

from edward.models import Normal, Exponential

# univariate normal
Normal(loc=tf.constant(0.0), scale=tf.constant(1.0))
# vector of 5 univariate normals
Normal(loc=tf.zeros(5), scale=tf.ones(5))
# 2 x 3 matrix of Exponentials
Exponential(rate=tf.ones([2, 3]))

For multivariate distributions, the multivariate dimension is the innermost (right-most) dimension of the parameters.

from edward.models import Dirichlet, MultivariateNormalTriL

# K-dimensional Dirichlet
Dirichlet(concentration=tf.constant([0.1] * K))
# vector of 5 K-dimensional multivariate normals with lower triangular cov
MultivariateNormalTriL(loc=tf.zeros([5, K]), scale_tril=tf.ones([5, K, K]))
# 2 x 5 matrix of K-dimensional multivariate normals
MultivariateNormalTriL(loc=tf.zeros([2, 5, K]), scale_tril=tf.ones([2, 5, K, K]))

Random variables are equipped with methods such as log_prob(), \(\log p(\mathbf{x}\mid\theta^*)\), mean(), \(\mathbb{E}_{p(\mathbf{x}\mid\theta^*)}[\mathbf{x}]\), and sample(), \(\mathbf{x}^*\sim p(\mathbf{x}\mid\theta^*)\). Further, each random variable is associated to a tensor \(\mathbf{x}^*\) in the computational graph, which represents a single sample \(\mathbf{x}^*\sim p(\mathbf{x}\mid\theta^*)\).

This makes it easy to parameterize random variables with complex deterministic structure, such as with deep neural networks, a diverse set of math operations, and compatibility with third party libraries which also build on TensorFlow. The design also enables compositions of random variables to capture complex stochastic structure. They operate on \(\mathbf{x}^*\).

image

from edward.models import Normal

x = Normal(loc=tf.zeros(10), scale=tf.ones(10))
y = tf.constant(5.0)
x + y, x - y, x * y, x / y
tf.tanh(x * y)
x[2]  # 3rd normal rv in the vector

In the compositionality page, we describe how to build models by composing random variables.


For a list of random variables supported in Edward, see the API reference page. Additional methods are available, detailed below.

class edward.models.RandomVariable(*args, **kwargs)[source]

Base class for random variables.

A random variable is an object parameterized by tensors. It is equipped with methods such as the log-density, mean, and sample.

It also wraps a tensor, where the tensor corresponds to a sample from the random variable. This enables operations on the TensorFlow graph, allowing random variables to be used in conjunction with other TensorFlow ops.

The random variable’s shape is given by

sample_shape + batch_shape + event_shape,

where sample_shape is an optional argument representing the dimensions of samples drawn from the distribution (default is a scalar); batch_shape is the number of independent random variables (determined by the shape of its parameters); and event_shape is the shape of one draw from the distribution (e.g., Normal has a scalar event_shape; Dirichlet has a vector event_shape).

Notes

RandomVariable assumes use in a multiple inheritance setting. The child class must first inherit RandomVariable, then second inherit a class in tf.contrib.distributions. With Python’s method resolution order, this implies the following during initialization (using distributions.Bernoulli as an example):

  1. Start the __init__() of the child class, which passes all *args, **kwargs to RandomVariable.
  2. This in turn passes all *args, **kwargs to distributions.Bernoulli, completing the __init__() of distributions.Bernoulli.
  3. Complete the __init__() of RandomVariable, which calls self.sample(), relying on the method from distributions.Bernoulli.
  4. Complete the __init__() of the child class.

Methods from both RandomVariable and distributions.Bernoulli populate the namespace of the child class. Methods from RandomVariable will take higher priority if there are conflicts.

Examples

p = tf.constant(0.5)
x = Bernoulli(p)

z1 = tf.constant([[1.0, -0.8], [0.3, -1.0]])
z2 = tf.constant([[0.9, 0.2], [2.0, -0.1]])
x = Bernoulli(logits=tf.matmul(z1, z2))

mu = Normal(tf.constant(0.0), tf.constant(1.0))
x = Normal(mu, tf.constant(1.0))

Attributes

Methods

Parameters:

sample_shape : tf.TensorShape, optional

Shape of samples to draw from the random variable.

value : tf.Tensor, optional

Fixed tensor to associate with random variable. Must have shape sample_shape + batch_shape + event_shape.

collections : list, optional

Optional list of graph collections keys. The random variable is added to these collections. Defaults to [“random_variables”].

*args, **kwargs

Passed into parent __init__.

Attributes

Methods

sample_shape

Sample shape of random variable.

shape

Shape of random variable.

unique_name

Name of random variable with its unique scoping name. Use name to just get the name of the random variable.

eval(session=None, feed_dict=None)[source]

In a session, computes and returns the value of this random variable.

This is not a graph construction method, it does not add ops to the graph.

This convenience method requires a session where the graph containing this variable has been launched. If no session is passed, the default session is used.

Parameters:

session : tf.BaseSession, optional

The tf.Session to use to evaluate this random variable. If none, the default session is used.

feed_dict : dict, optional

A dictionary that maps tf.Tensor objects to feed values. See tf.Session.run() for a description of the valid feed values.

Examples

x = Normal(0.0, 1.0)
with tf.Session() as sess:
  # Usage passing the session explicitly.
  print(x.eval(sess))
  # Usage with the default session.  The 'with' block
  # above makes 'sess' the default session.
  print(x.eval())

value()[source]

Get tensor that the random variable corresponds to.

get_ancestors(collection=None)[source]

Get ancestor random variables.

get_blanket(collection=None)[source]

Get the random variable’s Markov blanket.

get_children(collection=None)[source]

Get child random variables.

get_descendants(collection=None)[source]

Get descendant random variables.

get_parents(collection=None)[source]

Get parent random variables.

get_siblings(collection=None)[source]

Get sibling random variables.

get_variables(collection=None)[source]

Get TensorFlow variables that the random variable depends on.

get_shape()[source]

Get shape of random variable.

class edward.models.DirichletProcess(concentration, base, validate_args=False, allow_nan_stats=True, name='DirichletProcess', *args, **kwargs)[source]

Dirichlet process \(\mathcal{DP}(\alpha, H)\).

It has two parameters: a positive real value \(\alpha\), known as the concentration parameter (concentration), and a base distribution \(H\) (base).

Attributes

Methods

Initialize a batch of Dirichlet processes.

Parameters:

concentration : tf.Tensor

Concentration parameter. Must be positive real-valued. Its shape determines the number of independent DPs (batch shape).

base : RandomVariable

Base distribution. Its shape determines the shape of an individual DP (event shape).

Examples

# scalar concentration parameter, scalar base distribution
dp = DirichletProcess(0.1, Normal(loc=0.0, scale=1.0))
assert dp.shape == ()

# vector of concentration parameters, matrix of Exponentials
dp = DirichletProcess(tf.constant([0.1, 0.4]),
                      Exponential(lam=tf.ones([5, 3])))
assert dp.shape == (2, 5, 3)

Attributes

class edward.models.Empirical(params, validate_args=False, allow_nan_stats=True, name='Empirical', *args, **kwargs)[source]

Empirical random variable.

Attributes

Methods

Initialize an Empirical random variable.

Parameters:

params : tf.Tensor

Collection of samples. Its outer (left-most) dimension determines the number of samples.

Examples

# 100 samples of a scalar
x = Empirical(params=tf.zeros(100))
assert x.shape == ()

# 5 samples of a 2 x 3 matrix
dp = Empirical(params=tf.zeros([5, 2, 3]))
assert x.shape == (2, 3)

Attributes

class edward.models.PointMass(params, validate_args=False, allow_nan_stats=True, name='PointMass', *args, **kwargs)[source]

PointMass random variable.

It is analogous to an Empirical random variable with one sample, but its parameter argument does not have an outer dimension.

Attributes

Methods

Initialize a PointMass random variable.

Parameters:

params : tf.Tensor

The location with all probability mass.

Examples

# scalar
x = PointMass(params=28.3)
assert x.shape == ()

# 5 x 2 x 3 tensor
dp = PointMass(params=tf.zeros([5, 2, 3]))
assert x.shape == (5, 2, 3)

Attributes