aihwkit.simulator.parameters.inference module

Inference related parameters for resistive processing units.

class aihwkit.simulator.parameters.inference.DriftParameter(nu=0.0, t_0=1.0, reset_tol=1e-07, nu_dtod=0.0, nu_std=0.0, wg_ratio=1.0, g_offset=0.0, w_offset=0.0, nu_k=0.0, log_g0=0.0, w_noise_std=0.0)[source]

Bases: SimpleDriftParameter

Parameter for a power law drift.

The drift is based on the model described by Oh et al (2019).

It computes: .. math:

w_{ij}*\left(\frac{t + \Delta t}{t_0}\right)^(-\nu^\text{actual}_{ij})

where the drift coefficient is drawn once at the beginning and might depend on device. It also can depend on the actual weight value.

The actual drift coefficient is computed as: .. math:

\nu_{ij}^\text{actual} =  \nu_{ij} - \nu_k \log \frac{(w_{ij} - w_\text{off}) / r_\text{wg}
+ g_\text{off}}{G_0}  + \nu\sigma_\nu\xi

here \(w_{ij}\) is the actual weight and nu_{ij} fixed for each device given by the mean \(\nu\) and the device-to-device variation: \(\nu_{ij} = \nu + \nu_dtod\nu\xi\) and are only drawn once at the beginning (tile instantiation). xi is Gaussian noise.

Note

If the weight has changed from the last drift call (determined by the reset_tol parameter), for instance due to update, decay or noise, then the drift time \(t\) will be reset and start from new, however, the drift coefficients \(\nu_{ij}\) are not changed. On the other hand, if the weights has not changed since last call, \(t\) will accumulate the time.

Caution

Note that the drift coefficient does not depend on the initially programmed weight value at \(t=0\) in the current implementation (ie G0 is a constant for all devices), but instead on the actual weight. In some materials (e.g. phase changed materials), that might be not accurate.

Parameters:

  • nu (float) –

  • t_0 (float) –

  • reset_tol (float) –

  • nu_dtod (float) –

  • nu_std (float) –

  • wg_ratio (float) –

  • g_offset (float) –

  • w_offset (float) –

  • nu_k (float) –

  • log_g0 (float) –

  • w_noise_std (float) –

g_offset: float = 0.0

g_min to convert to physical units.

log_g0: float = 0.0

Log g0.

nu_dtod: float = 0.0

Device-to-device variation of the \(\nu\) values.

nu_k: float = 0.0

nu with \(W\).

That is \(\nu(R) = nu_0 - k \log(G/G_0)\). See Oh et al. for details.

Type:

Variation of math

nu_std: float = 0.0

Cycle-to-cycle variation of \(\nu\).

A more realistic way to add noise of the drift might be using w_noise_std.

w_noise_std: float = 0.0

Additional weight noise (Gaussian diffusion) added to the weights after the drift is applied.

w_offset: float = 0.0

w(g_min), i.e. to what value g_min is mapped to in w-space.

wg_ratio: float = 1.0

(w_max-w_min)/(g_max-g_min) to convert to physical units.

class aihwkit.simulator.parameters.inference.SimpleDriftParameter(nu=0.0, t_0=1.0, reset_tol=1e-07)[source]

Bases: _PrintableMixin

Parameter for a simple power law drift.

The drift as a simple power law drift without device-to-device variation or conductance dependence.

It computes: .. math:

w_{ij}*\left(\frac{t + \Delta t}{t_0}\right)^(-\nu)
Parameters:

  • nu (float) –

  • t_0 (float) –

  • reset_tol (float) –

bindings_class: ClassVar[str | Type | None] = 'DriftParameter'
bindings_module: ClassVar[str] = 'devices'
nu: float = 0.0

Average drift \(\nu\) value.

Need to non-zero to actually use the drift.

reset_tol: float = 1e-07

Reset tolerance.

This should a number smaller than the expected weight change as it is used to detect any changes in the weight from the last drift call. Every change to the weight above this tolerance will reset the drift time.

Caution

Any write noise or diffusion on the weight might thus interfere with the drift.

t_0: float = 1.0

Time between write and first read.

Usually assumed in milliseconds, however, it really determines the time units of time_since_last_call when calling the drift.

class aihwkit.simulator.parameters.inference.WeightClipParameter(fixed_value=-1.0, sigma=2.5, type=<factory>)[source]

Bases: _PrintableMixin

Parameter that clip the weights during hardware-aware training.

Important

A clipping type has to be set before any of the parameter changes take any effect.

Parameters:
bindings_class

alias of WeightClipParameter

fixed_value: float = -1.0

Clipping value in case of FixedValue type.

Caution

If fixed_value > 0 it will be also applied during other clipping types.

sigma: float = 2.5

Sigma value for clipping for the LayerGaussian type.

type: WeightClipType

Type of clipping.

class aihwkit.simulator.parameters.inference.WeightModifierParameter(std_dev=0.0, per_batch_sample=False, res=0.0, sto_round=False, dorefa_clip=0.6, pdrop=0.0, enable_during_test=False, rel_to_actual_wmax=True, assumed_wmax=1.0, copy_last_column=False, coeffs=<factory>, type=<factory>, g_max=25.0, pcm_zero_thres=0.0, pcm_t_inference=0.0, pcm_prob_at_reset=0.0, pcm_prob_at_gmax=0.0, pcm_prob_at_random=0.0, pcm_t0=20.0)[source]

Bases: _PrintableMixin

Parameter that modify the forward/backward weights during hardware-aware training.

Parameters:

  • std_dev (float) –

  • per_batch_sample (bool) –

  • res (float) –

  • sto_round (bool) –

  • dorefa_clip (float) –

  • pdrop (float) –

  • enable_during_test (bool) –

  • rel_to_actual_wmax (bool) –

  • assumed_wmax (float) –

  • copy_last_column (bool) –

  • coeffs (List[float]) –

  • type (WeightModifierType) –

  • g_max (float) –

  • pcm_zero_thres (float) –

  • pcm_t_inference (float) –

  • pcm_prob_at_reset (float) –

  • pcm_prob_at_gmax (float) –

  • pcm_prob_at_random (float) –

  • pcm_t0 (float) –

assumed_wmax: float = 1.0

Assumed weight value that is mapped to the maximal conductance.

This is typically 1.0. This parameter will be ignored if rel_to_actual_wmax is set.

bindings_class: ClassVar[str | Type | None] = 'WeightModifierParameter'
bindings_module: ClassVar[str] = 'tiles'
coeffs: List[float]

Coefficients for the POLY weight modifier type.

See WeightModifierType for details.

copy_last_column: bool = False

Whether to not apply noise to the last column (which usually contains the bias values).

dorefa_clip: float = 0.6

Parameter for DoReFa.

enable_during_test: bool = False

Whether to use the last modified weight matrix during testing.

Caution

This will not remove drop connect or any other noise during evaluation, and thus should only used with care.

g_max: float = 25.0

PCM_NOISE and PROG_NOISE parameter, \(g_\text{max}\) setting in \(\mu S\).

pcm_prob_at_gmax: float = 0.0

PCM_NOISE parameter, probability of devices being at \(g_\text{max}\).

pcm_prob_at_random: float = 0.0

PCM_NOISE parameter, probability of devices being at random value in the range.

pcm_prob_at_reset: float = 0.0

PCM_NOISE parameter, probability of reset.

pcm_t0: float = 20.0

PCM_NOISE parameter, programming conversion time in seconds.

pcm_t_inference: float = 0.0

PCM_NOISE parameter, time of inference.

pcm_zero_thres: float = 0.0

PCM_NOISE parameter

pdrop: float = 0.0

Drop connect probability.

Drop connect sets weights to zero with the given probability. This implements drop connect.

Important

Drop connect can be used with any other modifier type in combination.

per_batch_sample: bool = False

Should we resample noise for each sample in the batch.

This parameter only affects is used when using the TorchSimulatorTile. In case of RPUCudaTile it will throw an error.

rel_to_actual_wmax: bool = True

Whether to calculate the abs max of the weight and apply noise relative to this number.

If set to False, assumed_wmax is taken as relative units.

res: float = 0.0

Resolution of the discretization.

The invert of res gives the number of equal sized steps in \(-a_\text{max}\ldots,a_\text{max}\) where the \(a_\text{max}\) is either given by the abs max (if rel_to_actual_wmax is set) or assumed_wmax otherwise.

res is only used in the modifier types DoReFa, Discretize, and DiscretizeAddNormal.

std_dev: float = 0.0

Standard deviation of the added noise to the weight matrix.

This parameter affects the modifier types AddNormal, MultNormal and DiscretizeAddNormal.

Note

If the parameter rel_to_actual_wmax is set then the std_dev is computed in relative terms to the abs max of the given weight matrix, otherwise it in relative terms to the assumed max, which is set by assumed_wmax.

sto_round: bool = False

Whether the discretization is done with stochastic rounding enabled.

sto_round is only used in the modifier types DoReFa, Discretize, and DiscretizeAddNormal.

type: WeightModifierType

Type of the weight modification.

class aihwkit.simulator.parameters.inference.WeightRemapParameter(remapped_wmax=1.0, max_scale_range=0.0, max_scale_ref=0.0, type=<factory>)[source]

Bases: _PrintableMixin

Parameter that remap the weights during hardware-aware training.

Important

A remap type has to be set before any of the parameter changes take any effect.

Parameters:

  • remapped_wmax (float) –

  • max_scale_range (float) –

  • max_scale_ref (float) –

  • type (WeightRemapType) –

bindings_class

alias of WeightRemapParameter

max_scale_range: float = 0.0

Maximal range of scale values. Use zero to turn any restrictions off (default).

max_scale_ref: float = 0.0

Reference scale that use used as minimal scale for determining the scale range.

remapped_wmax: float = 1.0

Assumed max of weight, ie the value of the weight the maximal conductance is mapped to. Typically 1.0.

type: WeightRemapType

Type of clipping.