aepsych.models

Submodules

aepsych.models.base module

class aepsych.models.base.ModelProtocol(*args, **kwargs)[source]

Bases: Protocol

property outcome_type: str
property extremum_solver: str
property train_inputs: Tensor
property lb: Tensor
property ub: Tensor
property bounds: Tensor
property dim: int
property device: device
posterior(X)[source]
Parameters:

X (Tensor) –

Return type:

botorch.posteriors.GPyTorchPosterior

predict(x, **kwargs)[source]
Parameters:

x (Tensor) –

Return type:

Tensor

predict_probability(x, **kwargs)[source]
Parameters:

x (Tensor) –

Return type:

Tensor

property stimuli_per_trial: int
property likelihood: Likelihood
sample(x, num_samples)[source]
Parameters:

  • x (Tensor) –

  • num_samples (int) –

Return type:

Tensor

dim_grid(gridsize=30)[source]
Parameters:

gridsize (int) –

Return type:

Tensor

fit(train_x, train_y, **kwargs)[source]
Parameters:

  • train_x (Tensor) –

  • train_y (Tensor) –

  • kwargs (Any) –

Return type:

None

update(train_x, train_y, **kwargs)[source]
Parameters:

  • train_x (Tensor) –

  • train_y (Tensor) –

  • kwargs (Any) –

Return type:

None

p_below_threshold(x, f_thresh)[source]
Parameters:

  • x (Tensor) –

  • f_thresh (Tensor) –

Return type:

Tensor

class aepsych.models.base.AEPsychMixin(*args, **kwargs)[source]

Bases: GPyTorchModel, ConfigurableMixin

Mixin class that provides AEPsych-specific utility methods.

Parameters:

  • args (Any) –

  • kwargs (Any) –

Return type:

Any

extremum_solver = 'Nelder-Mead'
outcome_types: List[str] = []
train_inputs: Optional[Tuple[Tensor]]
train_targets: Optional[Tensor]
stimuli_per_trial: int = 1
set_train_data(inputs=None, targets=None, strict=False)[source]

Set the training data for the model.

Parameters:

  • inputs (torch.Tensor, optional) – The new training inputs.

  • targets (torch.Tensor, optional) – The new training targets.

  • strict (bool) – Default is False. Ignored, just for compatibility.

input transformers. TODO: actually use this arg or change input transforms to not require it.

forward(x)[source]

Evaluate GP

Parameters:

x (torch.Tensor) – Tensor of points at which GP should be evaluated.

Returns:

Distribution object

holding mean and covariance at x.

Return type:

gpytorch.distributions.MultivariateNormal

classmethod get_config_options(config, name=None, options=None)[source]

.

Parameters:

  • config (Config) – Config to look for options in.

  • name (str, optional) – The name of the strategy to warm start (Not actually optional here.)

  • options (Dict[str, Any], optional) – options are ignored.

Raises:

  • ValueError – the name of the strategy is necessary to identify warm start search criteria.

  • KeyError – the config specified this strategy should be warm started but the associated config section wasn’t defined.

Returns:

a dictionary of the search criteria described in the experiment’s config

Return type:

Dict[str, Any]

p_below_threshold(x, f_thresh)[source]

Compute the probability that the latent function is below a threshold.

Parameters:

  • x (torch.Tensor) – Points at which to evaluate the probability.

  • f_thresh (torch.Tensor) – Threshold value.

Returns:

Probability that the latent function is below the threshold.

Return type:

torch.Tensor

class aepsych.models.base.AEPsychModelDeviceMixin(*args, **kwargs)[source]

Bases: AEPsychMixin

Parameters:

  • args (Any) –

  • kwargs (Any) –

Return type:

Any

set_train_data(inputs=None, targets=None, strict=False)[source]

Set the training data for the model.

Parameters:

  • inputs (torch.Tensor, optional) – The new training inputs X.

  • targets (torch.Tensor, optional) – The new training targets Y.

  • strict (bool) – Whether to strictly enforce the device of the inputs and targets.

Return type:

None

input transformers. TODO: actually use this arg or change input transforms to not require it.

property device: device

Get the device of the model.

Returns:

Device of the model.

Return type:

torch.device

property train_inputs: Optional[Tuple[Tensor]]

Get the training inputs.

Returns:

Training inputs.

Return type:

Optional[Tuple[torch.Tensor]]

property train_targets: Optional[Tensor]

Get the training targets.

Returns:

Training targets.

Return type:

Optional[torch.Tensor]

aepsych.models.derivative_gp module

aepsych.models.gp_classification module

class aepsych.models.gp_classification.GPClassificationModel(*args, **kwargs)[source]

Bases: AEPsychModelDeviceMixin, ApproximateGP

Probit-GP model with variational inference.

From a conventional ML perspective this is a GP Classification model, though in the psychophysics context it can also be thought of as a nonlinear generalization of the standard linear model for 1AFC or yes/no trials.

For more on variational inference, see e.g. https://docs.gpytorch.ai/en/v1.1.1/examples/04_Variational_and_Approximate_GPs/

Initialize the GP Classification model

Parameters:

  • dim (int) – The number of dimensions in the parameter space.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior.

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to Bernouli likelihood.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time.

  • optimizer_options (Dict[str, Any], optional) – Optimizer options to pass to the SciPy optimizer during fitting. Assumes we are using L-BFGS-B.

stimuli_per_trial: int = 1
outcome_type = 'binary'
classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the model.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the model, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the model.

Return type:

Dict[str, Any]

fit(train_x, train_y, warmstart_hyperparams=False, warmstart_induc=False, **kwargs)[source]

Fit underlying model.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.LongTensor) – Responses.

  • warmstart_hyperparams (bool) – Whether to reuse the previous hyperparameters (True) or fit from scratch (False). Defaults to False.

  • warmstart_induc (bool) – Whether to reuse the previous inducing points or fit from scratch (False). Defaults to False.

Return type:

None

sample(x, num_samples)[source]

Sample from underlying model.

Parameters:

  • x (torch.Tensor) – Points at which to sample.

  • num_samples (int) – Number of samples to return.

  • ignored (kwargs are) –

Returns:

Posterior samples [num_samples x dim]

Return type:

torch.Tensor

predict(x, probability_space=False)[source]

Query the model for posterior mean and variance.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

predict_probability(x)[source]

Query the model for posterior mean and variance in probability space.

Parameters:

x (torch.Tensor) – Points at which to predict from the model.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

update(train_x, train_y, **kwargs)[source]

Perform a warm-start update of the model from previous fit.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.Tensor) – Responses.

class aepsych.models.gp_classification.GPBetaRegressionModel(*args, **kwargs)[source]

Bases: GPClassificationModel

Initialize the GP Beta Regression model

Parameters:

  • dim (int) – The number of dimensions in the parameter space.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior. Defaults to None.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior.

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to Beta likelihood.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time. Defaults to None.

  • optimizer_options (Optional[Dict[str, Any]]) –

outcome_type = 'percentage'

aepsych.models.monotonic_rejection_gp module

aepsych.models.inducing_points module

class aepsych.models.inducing_points.FixedAllocator(*args, **kwargs)[source]

Bases: BaseAllocator

Initialize the FixedAllocator with inducing points to use and bounds.

Parameters:

  • dim (int) – Dimensionality of the search space.

  • points (torch.Tensor) – Inducing points to use (should be n, d).

allocate_inducing_points(inputs=None, covar_module=None, num_inducing=100, input_batch_shape=torch.Size([]))[source]

Allocate inducing points by returning the fixed inducing points.

Parameters:

  • inputs (torch.Tensor) – Input tensor, not required for FixedAllocator.

  • covar_module (torch.nn.Module, optional) – Kernel covariance module; included for API compatibility, but not used here.

  • num_inducing (int, optional) – The number of inducing points to generate. Defaults to 10.

  • input_batch_shape (torch.Size, optional) – Batch shape, defaults to an empty size; included for API compatibility, but not used here.

Returns:

The fixed inducing points.

Return type:

torch.Tensor

classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the FixedAllocator.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the allocator, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the FixedAllocator.

Return type:

Dict[str, Any]

class aepsych.models.inducing_points.GreedyVarianceReduction(*args, **kwargs)[source]

Bases: GreedyVarianceReduction, BaseAllocator

Initialize the allocator with optional bounds.

Parameters:

  • dim (int) – Dimensionality of the search space.

  • *args (Any) – Other allocator specific arguments.

  • **kwargs (Any) –

    Other allocator specific arguments.

Return type:

Any

allocate_inducing_points(inputs=None, covar_module=None, num_inducing=100, input_batch_shape=torch.Size([]))[source]

Allocate inducing points using the GreedyVarianceReduction strategy. This is a thin wrapper around BoTorch’s GreedyVarianceRedution inducing point allocator.

Parameters:

  • inputs (torch.Tensor) – Input tensor, not required for GreedyVarianceReduction.

  • covar_module (torch.nn.Module, optional) – Kernel covariance module; included for API compatibility, but not used here.

  • num_inducing (int, optional) – The number of inducing points to generate. Defaults to 10.

  • input_batch_shape (torch.Size, optional) – Batch shape, defaults to an empty size; included for API compatibility, but not used here.

Returns:

The allocated inducing points.

Return type:

torch.Tensor

class aepsych.models.inducing_points.KMeansAllocator(*args, **kwargs)[source]

Bases: BaseAllocator

An inducing point allocator that uses k-means++ to allocate inducing points.

Initialize the allocator with optional bounds.

Parameters:

  • dim (int) – Dimensionality of the search space.

  • *args (Any) – Other allocator specific arguments.

  • **kwargs (Any) –

    Other allocator specific arguments.

Return type:

Any

allocate_inducing_points(inputs=None, covar_module=None, num_inducing=100, input_batch_shape=torch.Size([]))[source]

Generates num_inducing inducing points using k-means++ initialization on the input data.

Parameters:

  • inputs (torch.Tensor) – A tensor of shape (n, d) containing the input data.

  • covar_module (torch.nn.Module, optional) – Kernel covariance module; included for API compatibility, but not used here.

  • num_inducing (int, optional) – The number of inducing points to generate. Defaults to 100.

  • input_batch_shape (torch.Size, optional) – Batch shape, defaults to an empty size; included for API compatibility, but not used here.

Returns:

A (num_inducing, d)-dimensional tensor of inducing points selected via k-means++.

Return type:

torch.Tensor

class aepsych.models.inducing_points.SobolAllocator(*args, **kwargs)[source]

Bases: BaseAllocator

An inducing point allocator that uses Sobol sequences to allocate inducing points.

Initializes a Sobol Allocator. This allocator must have bounds.

Parameters:

  • dim (int) – Dimensionality of the search space.

  • bounds (torch.Tensor) – Bounds for allocating points. Should be of shape (2, d).

allocate_inducing_points(inputs=None, covar_module=None, num_inducing=100, input_batch_shape=torch.Size([]))[source]

Generates num_inducing inducing points within the specified bounds using Sobol sampling.

Parameters:

  • inputs (torch.Tensor) – Input tensor, ignored for Sobol points.

  • covar_module (torch.nn.Module, optional) – Kernel covariance module; included for API compatibility, but not used here.

  • num_inducing (int, optional) – The number of inducing points to generate. Defaults to 100.

  • input_batch_shape (torch.Size, optional) – Batch shape, defaults to an empty size; included for API compatibility, but not used here.

Returns:

A (num_inducing, d)-dimensional tensor of inducing points within the specified bounds.

Return type:

torch.Tensor

classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the FixedAllocator.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the allocator, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the FixedAllocator.

Return type:

Dict[str, Any]

Module contents

class aepsych.models.GPClassificationModel(*args, **kwargs)[source]

Bases: AEPsychModelDeviceMixin, ApproximateGP

Probit-GP model with variational inference.

From a conventional ML perspective this is a GP Classification model, though in the psychophysics context it can also be thought of as a nonlinear generalization of the standard linear model for 1AFC or yes/no trials.

For more on variational inference, see e.g. https://docs.gpytorch.ai/en/v1.1.1/examples/04_Variational_and_Approximate_GPs/

Initialize the GP Classification model

Parameters:

  • dim (int) – The number of dimensions in the parameter space.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior.

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to Bernouli likelihood.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time.

  • optimizer_options (Dict[str, Any], optional) – Optimizer options to pass to the SciPy optimizer during fitting. Assumes we are using L-BFGS-B.

stimuli_per_trial: int = 1
outcome_type = 'binary'
likelihood: Likelihood
classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the model.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the model, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the model.

Return type:

Dict[str, Any]

fit(train_x, train_y, warmstart_hyperparams=False, warmstart_induc=False, **kwargs)[source]

Fit underlying model.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.LongTensor) – Responses.

  • warmstart_hyperparams (bool) – Whether to reuse the previous hyperparameters (True) or fit from scratch (False). Defaults to False.

  • warmstart_induc (bool) – Whether to reuse the previous inducing points or fit from scratch (False). Defaults to False.

Return type:

None

sample(x, num_samples)[source]

Sample from underlying model.

Parameters:

  • x (torch.Tensor) – Points at which to sample.

  • num_samples (int) – Number of samples to return.

  • ignored (kwargs are) –

Returns:

Posterior samples [num_samples x dim]

Return type:

torch.Tensor

predict(x, probability_space=False)[source]

Query the model for posterior mean and variance.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

predict_probability(x)[source]

Query the model for posterior mean and variance in probability space.

Parameters:

x (torch.Tensor) – Points at which to predict from the model.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

update(train_x, train_y, **kwargs)[source]

Perform a warm-start update of the model from previous fit.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.Tensor) – Responses.

training: bool
class aepsych.models.GPRegressionModel(*args, **kwargs)[source]

Bases: AEPsychModelDeviceMixin, ExactGP

GP Regression model for continuous outcomes, using exact inference.

Initialize the GP regression model

Parameters:

  • dim (int) – The number of dimensions in the parameter space.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior.

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to Gaussian likelihood.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time.

  • optimizer_options (Dict[str, Any], optional) – Optimizer options to pass to the SciPy optimizer during fitting. Assumes we are using L-BFGS-B.

stimuli_per_trial: int = 1
outcome_type = 'continuous'
fit(train_x, train_y, **kwargs)[source]

Fit underlying model.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.LongTensor) – Responses.

Return type:

None

sample(x, num_samples)[source]

Sample from underlying model.

Parameters:

  • x (torch.Tensor) – Points at which to sample.

  • num_samples (int) – Number of samples to return.

Returns:

Posterior samples [num_samples x dim]

Return type:

torch.Tensor

update(train_x, train_y, **kwargs)[source]

Perform a warm-start update of the model from previous fit.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.Tensor) – Responses.

predict(x, **kwargs)[source]

Query the model for posterior mean and variance.

Parameters:

x (torch.Tensor) – Points at which to predict from the model.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

class aepsych.models.OrdinalGPModel(*args, **kwargs)[source]

Bases: GPClassificationModel

Convenience wrapper for GPClassificationModel that hardcodes an ordinal likelihood, better priors for this setting, and adds a convenience method for computing outcome probabilities.

TODO: at some point we should refactor posteriors so that things like OrdinalPosterior and MonotonicPosterior don’t have to have their own model classes.

Initialize the OrdinalGPModel

Parameters:

likelihood (Likelihood) – The likelihood function to use. If None defaults to Ordinal likelihood.

outcome_type = 'ordinal'
predict_probs(xgrid)[source]

Predict probabilities of each ordinal level at xgrid

Parameters:

xgrid (torch.Tensor) – Tensor of input points to predict at

Returns:

Tensor of probabilities of each ordinal level at xgrid

Return type:

torch.Tensor

calculate_probs(fmean, fvar)[source]

Calculate probabilities of each ordinal level given a mean and variance

Parameters:

  • fmean (torch.Tensor) – Mean of the latent function

  • fvar (torch.Tensor) – Variance of the latent function

Returns:

Tensor of probabilities of each ordinal level

Return type:

torch.Tensor

class aepsych.models.MonotonicProjectionGP(*args, **kwargs)[source]

Bases: GPClassificationModel

A monotonic GP based on posterior projection

NOTE: This model does not currently support backprop and so cannot be used with gradient optimization for active learning.

This model produces predictions that are monotonic in any number of specified monotonic dimensions. It follows the intuition of the paper

Lin L, Dunson DB (2014) Bayesian monotone regression using Gaussian process projection, Biometrika 101(2): 303-317.

but makes significant departures by using heuristics for a lot of what is done in a more principled way in the paper. The reason for the move to heuristics is to improve scaling, especially with multiple monotonic dimensions.

The method in the paper applies PAVA projection at the sample level, which requires a significant amount of costly GP posterior sampling. The approach taken here applies rolling-max projection to quantiles of the distribution, and so requires only marginal posterior evaluation. There is also a significant departure in the way multiple monotonic dimensions are handled, since in the paper computation scales exponentially with the number of monotonic dimensions and the heuristic approach taken here scales linearly in the number of dimensions.

The cost of these changes is that the convergence guarantees proven in the paper no longer hold. The method implemented here is a heuristic, and it may be useful in some problems.

The principle behind the method given here is that sample-level monotonicity implies monotonicity in the quantiles. We enforce monotonicity in several quantiles, and use that as an approximation for the true projected posterior distribution.

The approach here also supports specifying a minimum value of f. That minimum will be enforced on mu, but not necessarily on the lower bound of the projected posterior since we keep the projected posterior normal. The min f value will also be enforced on samples drawn from the model, while monotonicity will not be enforced at the sample level.

The procedure for computing the monotonic projected posterior at x is: 1. Separately for each monotonic dimension, create a grid of s points that differ only in that dimension, and sweep from the lower bound up to x. 2. Evaluate the marginal distribution, mu and sigma, on the full set of points (x and the s grid points for each monotonic dimension). 3. Compute the mu +/- 2 * sigma quantiles. 4. Enforce monotonicity in the quantiles by taking mu_proj as the maximum mu across the set, and lb_proj as the maximum of mu - 2 * sigma across the set. ub_proj is left as mu(x) + 2 * sigma(x), but is clamped to mu_proj in case that project put it above the original ub. 5. Clamp mu and lb to the minimum value for f, if one was set. 6. Construct a new normal posterior given the projected quantiles by taking mu_proj as the mean, and (ub - lb) / 4 as the standard deviation. Adjust the covariance matrix to account for the change in the marginal variances.

The process above requires only marginal posterior evaluation on the grid of points used for the posterior projection, and the size of that grid scales linearly with the number of monotonic dimensions, not exponentially.

The args here are the same as for GPClassificationModel with the addition of:

Parameters:

  • monotonic_dims (List[int]) – A list of the dimensions on which monotonicity should be enforced.

  • monotonic_grid_size (int) – The size of the grid, s, in 1. above.

  • min_f_val (Optional[float]) – If provided, maintains this minimum in the projection in 5.

  • lb (torch.Tensor) –

  • ub (torch.Tensor) –

  • dim (int) –

  • mean_module (Optional[gpytorch.means.Mean]) –

  • covar_module (Optional[gpytorch.kernels.Kernel]) –

  • likelihood (Optional[Likelihood]) –

  • inducing_point_method (Optional[InducingPointAllocator]) –

  • inducing_size (int) –

  • max_fit_time (Optional[float]) –

  • optimizer_options (Optional[Dict[str, Any]]) –

Initialize the MonotonicProjectionGP model. Unlike other models, this model needs bounds.

Parameters:

  • lb (torch.Tensor) – Lower bounds of the parameters.

  • ub (torch.Tensor) – Upper bounds of the parameters.

  • dim (int, optional) – The number of dimensions in the parameter space.

  • monotonic_dims (List[int]) – A list of the dimensions on which monotonicity should be enforced.

  • monotonic_grid_size (int) – The size of the grid, s, in 1. above. Defaults to 20.

  • min_f_val (float, optional) – If provided, maintains this minimum in the projection in 5. Defaults to None.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior. Defaults to None.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior. Defaults to None.

  • likelihood (Likelihood, optional) – The likelihood function to use. If None defaults to Gaussian likelihood. Defaults to None.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – The number of inducing points to use. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time. Defaults to None.

  • optimizer_options (Optional[Dict[str, Any]]) –

posterior(X, observation_noise=False, **kwargs)[source]

Compute the posterior at X, projecting to enforce monotonicity.

Parameters:

  • X (torch.Tensor) – The input points at which to compute the posterior.

  • observation_noise (Union[bool, torch.Tensor]) – Whether or not to include the observation noise in the posterior. Defaults to False.

  • kwargs (Any) –

Returns:

The posterior at X.

Return type:

GPyTorchPosterior

sample(x, num_samples)[source]

Sample from the model.

Parameters:

  • x (torch.Tensor) – The input points at which to sample.

  • num_samples (int) – The number of samples to draw.

Returns:

The samples at x.

Return type:

torch.Tensor

classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the model.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the model, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the model.

Return type:

Dict[str, Any]

class aepsych.models.HadamardSemiPModel(*args, **kwargs)[source]

Bases: GPClassificationModel

Semiparametric GP model for psychophysics, with a MVN approximation to the elementwise product of GPs.

Implements a semi-parametric model with a functional form like \(k(x_c()x_i + c(x_c))\), for scalar intensity dimension \(x_i\) and vector-valued context dimensions \(x_c\), with k and c having a GP prior. In contrast to SemiParametricGPModel, this version approximates the product as a single multivariate normal, which should be faster (the approximation is exact if one of the GP’s variance goes to zero). Intended for use with a BernoulliObjectiveLikelihood with flexible link function such as Logistic or Gumbel nonlinearity with a floor.

Initialize HadamardSemiPModel. :param dim: The number of dimensions in the parameter space. :type dim: int :param stim_dim: Index of the intensity (monotonic) dimension. Defaults to 0. :type stim_dim: int :param slope_mean_module: Mean module to use (default: constant mean) for slope. :type slope_mean_module: gpytorch.means.Mean, optional :param slope_covar_module: Covariance kernel to use (default: scaled RBF) for slope. :type slope_covar_module: gpytorch.kernels.Kernel, optional :param offset_mean_module: Mean module to use (default: constant mean) for offset. :type offset_mean_module: gpytorch.means.Mean, optional :param offset_covar_module: Covariance kernel to use (default: scaled RBF) for offset. :type offset_covar_module: gpytorch.kernels.Kernel, optional :param likelihood: defaults to bernoulli with logistic input and a floor of .5 :type likelihood: gpytorch.likelihood.Likelihood, optional) :param slope_mean: The mean of the slope. Defaults to 2. :type slope_mean: float :param inducing_point_method: The method to use for selecting inducing points.

If not set, a GreedyVarianceReduction is made.

Parameters:

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time.

  • optimizer_options (Dict[str, Any], optional) – Optimizer options to pass to the SciPy optimizer during fitting. Assumes we are using L-BFGS-B.

  • dim (int) –

  • stim_dim (int) –

  • slope_mean_module (Optional[gpytorch.means.Mean]) –

  • slope_covar_module (Optional[gpytorch.kernels.Kernel]) –

  • offset_mean_module (Optional[gpytorch.means.Mean]) –

  • offset_covar_module (Optional[gpytorch.kernels.Kernel]) –

  • likelihood (Optional[Likelihood]) –

  • slope_mean (float) –

  • inducing_point_method (InducingPointAllocator, optional) –

stimuli_per_trial: int = 1
outcome_type = 'binary'
forward(x)[source]

Forward pass for HadamardSemiPModel GP.

generates a k(c + x[:,stim_dim]) = kc + kx[:,stim_dim] mvn object where k and c are slope and offset GPs and x[:,stim_dim] are the intensity stimulus (x) locations and thus acts as a constant offset to the k mvn. :param x: Points at which to sample. :type x: torch.Tensor

Returns:

MVN object evaluated at samples

Parameters:

x (Tensor) –

Return type:

MultivariateNormal

classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the model.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the model, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the model.

Return type:

Dict[str, Any]

predict(x, probability_space=False)[source]

Query the model for posterior mean and variance.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

class aepsych.models.SemiParametricGPModel(*args, **kwargs)[source]

Bases: GPClassificationModel

Semiparametric GP model for psychophysics.

Implements a semi-parametric model with a functional form like \(k(x_c()x_i + c(x_c))\), for scalar intensity dimension \(x_i\) and vector-valued context dimensions \(x_c\), with k and c having a GP prior. In contrast to HadamardSemiPModel, this version uses a batched GP directly, which is about 2-3x slower but does not use the MVN approximation.

Intended for use with a BernoulliObjectiveLikelihood with flexible link function such as Logistic or Gumbel nonlinearity with a floor.

Initialize SemiParametricGP. :param dim: The number of dimensions in the parameter space. :type dim: int, optional :param stim_dim: Index of the intensity (monotonic) dimension. Defaults to 0. :type stim_dim: int :param mean_module: GP mean class. Defaults to a constant with a normal prior. :type mean_module: gpytorch.means.Mean, optional :param covar_module: GP covariance kernel class. Defaults to scaled RBF with a

gamma prior.

Parameters:

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to linear-Bernouli likelihood with probit link.

  • slope_mean (float) – The mean of the slope. Defaults to 2.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time.

  • optimizer_options (Dict[str, Any], optional) – Optimizer options to pass to the SciPy optimizer during fitting. Assumes we are using L-BFGS-B.

  • dim (int) –

  • stim_dim (int) –

  • mean_module (Optional[gpytorch.means.Mean]) –

  • covar_module (gpytorch.kernels.Kernel, optional) –

stimuli_per_trial: int = 1
outcome_type = 'binary'
classmethod get_config_options(config, name=None, options=None)[source]

Get configuration options for the model.

Parameters:

  • config (Config) – Configuration object.

  • name (str, optional) – Name of the model, defaults to None.

  • options (Dict[str, Any], optional) – Additional options, defaults to None.

Returns:

Configuration options for the model.

Return type:

Dict[str, Any]

fit(train_x, train_y, warmstart_hyperparams=False, warmstart_induc=False, **kwargs)[source]

Fit underlying model.

Parameters:

  • train_x (torch.Tensor) – Inputs.

  • train_y (torch.LongTensor) – Responses.

  • warmstart_hyperparams (bool) – Whether to reuse the previous hyperparameters (True) or fit from scratch (False). Defaults to False.

  • warmstart_induc (bool) – Whether to reuse the previous inducing points or fit from scratch (False). Defaults to False.

  • kwargs – Keyword arguments passed to optimizer=fit_gpytorch_mll_scipy.

Return type:

None

sample(x, num_samples, probability_space=False)[source]

Sample from underlying model.

Parameters:

  • x (torch.Tensor) – n x d Points at which to sample.

  • num_samples (int) – Number of samples to return. Defaults to None.

  • probability_space (bool) – Whether to sample from the probability space (True) or the latent function. Defaults to False.

  • ignored (kwargs are) –

Returns:

Posterior samples

Return type:

(num_samples x n) torch.Tensor

predict(x, probability_space=False)[source]

Query the model for posterior mean and variance.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

Returns:

Posterior mean and variance at query points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

posterior(X, posterior_transform=None)[source]

Get the posterior distribution at the given points.

Parameters:

  • X (torch.Tensor) – Points at which to evaluate the posterior.

  • posterior_transform (PosteriorTransform, optional) – A transform to apply to the posterior. Defaults to None.

Returns:

The posterior distribution at the given points.

Return type:

SemiPPosterior

aepsych.models.semi_p_posterior_transform(posterior)[source]

Transform a posterior from a SemiP model to a Hadamard model.

Parameters:

posterior (GPyTorchPosterior) – The posterior to transform

Returns:

The transformed posterior.

Return type:

GPyTorchPosterior

class aepsych.models.GPBetaRegressionModel(*args, **kwargs)[source]

Bases: GPClassificationModel

Initialize the GP Beta Regression model

Parameters:

  • dim (int) – The number of dimensions in the parameter space.

  • mean_module (gpytorch.means.Mean, optional) – GP mean class. Defaults to a constant with a normal prior. Defaults to None.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior.

  • likelihood (gpytorch.likelihood.Likelihood, optional) – The likelihood function to use. If None defaults to Beta likelihood.

  • inducing_point_method (InducingPointAllocator, optional) – The method to use for selecting inducing points. If not set, a GreedyVarianceReduction is made.

  • inducing_size (int) – Number of inducing points. Defaults to 100.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. If None, there is no limit to the fitting time. Defaults to None.

  • optimizer_options (Optional[Dict[str, Any]]) –

outcome_type = 'percentage'
likelihood: Likelihood
training: bool
class aepsych.models.PairwiseProbitModel(*args, **kwargs)[source]

Bases: PairwiseGP, AEPsychMixin

Initialize the PairwiseProbitModel

Parameters:

  • lb (torch.Tensor) – Lower bounds of the parameters.

  • ub (torch.Tensor) – Upper bounds of the parameters.

  • dim (int, optional) – The number of dimensions in the parameter space. If None, it is inferred from the size of lb and ub. Defaults to None.

  • covar_module (gpytorch.kernels.Kernel, optional) – GP covariance kernel class. Defaults to scaled RBF with a gamma prior. Defaults to None.

  • max_fit_time (float, optional) – The maximum amount of time, in seconds, to spend fitting the model. Defaults to None.

  • optimizer_options (Optional[Dict[str, Any]]) –

stimuli_per_trial: int = 2
outcome_type = 'binary'
fit(train_x, train_y, optimizer_kwargs=None, **kwargs)[source]

Fit the model to the training data.

Parameters:

  • train_x (torch.Tensor) – Trainin x points.

  • train_y (torch.Tensor) – Training y points.

  • optimizer_kwargs (Dict[str, Any], optional) – Keyword arguments to pass to the optimizer. Defaults to None.

Return type:

None

update(train_x, train_y, warmstart=True)[source]

Perform a warm-start update of the model from previous fit.

Parameters:

  • train_x (torch.Tensor) – Train X.

  • train_y (torch.Tensor) – Train Y.

  • warmstart (bool) – If True, warm-start model fitting with current parameters. Defaults to True.

Return type:

None

predict(x, probability_space=False, num_samples=1000, rereference='x_min')[source]

Query the model for posterior mean and variance.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

  • num_samples (int) – Number of samples to return. Defaults to 1000.

  • rereference (str) – How to sample. Options are “x_min”, “x_max”, “f_min”, “f_max”. Defaults to “x_min”.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

predict_probability(x, probability_space=False, num_samples=1000, rereference='x_min')[source]

Query the model for posterior mean and variance in probability space.

Parameters:

  • x (torch.Tensor) – Points at which to predict from the model.

  • probability_space (bool) – Return outputs in units of response probability instead of latent function value. Defaults to False.

  • num_samples (int) – Number of samples to return. Defaults to 1000.

  • rereference (str) – How to sample. Options are “x_min”, “x_max”, “f_min”, “f_max”. Defaults to “x_min”.

Returns:

Posterior mean and variance at queries points.

Return type:

Tuple[torch.Tensor, torch.Tensor]

sample(x, num_samples, rereference='x_min')[source]

Sample from the model model posterior.

Parameters:

  • x (torch.Tensor) – Points at which to sample.

  • num_samples (int) – Number of samples to return.

  • rereference (str) – How to sample. Options are “x_min”, “x_max”, “f_min”, “f_max”. Defaults to “x_min”.

Returns:

Posterior samples [num_samples x dim]

Return type:

torch.Tensor

classmethod get_config_options(config, name=None, options=None)[source]

Alternate constructor for GPClassification model from a configuration.

This is used when we recursively build a full sampling strategy from a configuration. TODO: document how this works in some tutorial.

Parameters:

  • config (Config) – A configuration containing keys/values matching this class

  • name (Optional[str]) –

  • options (Optional[Dict[str, Any]]) –

Returns:

Configured class instance.

Return type:

GPClassificationModel