RLParams

Module containing commonly used reinforcement learning parameter objects.

Module Documentation

class fitr.rlparams.ChoiceRandomness(name='Choice Randomness', rng='pos', mean=4, sd=1)

An choice randomness parameter object

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.gamma distribution

Methods

sample(size=1)

Samples from the parameter’s distribution

class fitr.rlparams.EligibilityTrace(name='Eligibility Trace', rng='unit', mean=0.5, sd=0.27)

An eligibility trace parameter object.

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.beta distribution

Methods

sample(size=1)

Samples from the parameter’s distribution

class fitr.rlparams.LearningRate(name='Learning Rate', rng='unit', mean=0.5, sd=0.27)

A learning rate object.

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.beta distribution

Methods

sample(size=1)

Samples from the parameter’s distribution

class fitr.rlparams.MBMF_Balance(name='Model-Based Control Weight', rng='unit', mean=0.5, sd=0.27)

An object representing the parameter that balances model-based and model-free control.

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.beta distribution

Methods

sample(size=1)

Samples from the parameter’s distribution

class fitr.rlparams.Param(name=None, rng=None)

A base parameter object that can be used to generate new parameters.

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

Methods

sample(size=1)

Samples from the parameter’s distribution

convert_meansd(mean, sd, dist)

Converts mean and standard deviation to other distribution parameters.

Parameters:

mean : float

Mean value for distribution (must lie within region of support)

sd : float

Standard deviation for distribution

dist : {‘beta’, ‘gamma’}

Target distribution

Notes

Currently, only the gamma and beta distributions are supported for this function.

The Beta distribution has two shape parameters \(\lbrace \alpha, \beta \rbrace > 0\). Using the mean \(\mu\) and the standard deviation \(\sigma\), the \(\alpha\) parameter can be calculated as

\[\alpha = (\frac{1-\mu}{\sigma^2} - \frac{1}{\mu})\mu^2\]

and the \(\beta\) parameter as

\[\beta = \alpha (\frac{1}{\mu} - 1)\]

Note that for the Beta distribution to be defined this way, the following constraint must hold for the mean, \(0 < \mu < 1\), and the following for the variance, \(0 < \sigma^2 \leq \mu - \mu^2\).

For the Gamma distribution, we have a shape parameter \(\kappa > 0\) and a scale parameter \(\theta\). These can be calculated using the mean \(\mu\) and standard deviation \(\sigma\) as

\[\theta = \frac{\sigma^2}{\mu}\]

and

\[\kappa = \frac{\mu^2}{\sigma^2}\]

plot_pdf(xlim=None, figsize=None, save_figure=False, filename='parameter-pdf.pdf')

Plots the probability density function of this parameter

Parameters:

xlim : (optional) list of lower and upper bounds of x axis

figsize : (optional) list defining plot dimensions

save_figure : bool

Whether to save the figure at function call

filename : str

The name of the file at which to save the figure

sample(size=1)

Samples from the parameter’s distribution

Parameters:

size : int

Number of samples to draw

Returns:

ndarray

class fitr.rlparams.Perseveration(name='Perseveration', rng='unc', mean=0.0, sd=0.1)

An perseveration parameter object

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.norm distribution

Methods

sample(size=1)

Samples from the parameter’s distribution

class fitr.rlparams.RewardSensitivity(name='Reward Sensitivity', rng='unit', mean=0.5, sd=0.27)

A reward sensitivity object.

Attributes:

name : str

Name of the parameter. To be used for plots and so forth.

rng : {‘unit’, ‘pos’, ‘neg’, ‘unc’}

The domain over which the parameter lies (unit=[0,1], pos=[0,+Inf], neg=[-Inf,0], unc=[-Inf, +Inf])

dist : scipy.stats.beta distribution

Methods

sample(size=1)

Samples from the parameter’s distribution