aihwkit.simulator.parameters.enums module

Utility enumerators for resistive processing units.

class aihwkit.simulator.parameters.enums.AnalogMVType(value)

Bases: Enum

Type of the analog matrix-vector product.

BIT_WISE = 'BitWise'

Bit-wise PWM input to speedup MAC and increase energy efficiency (may sacrifice some accuracy).

Caution

Only supported for TorchInferenceRPUConfigIRDropT

IDEAL = 'Ideal'

FP mat-vec without any non-idealities. Same as setting is_perfect=True.

ONE_PASS = 'OnePass'

One pass through the crossbar array for positive and negative inputs.

POS_NEG_SEPARATE = 'PosNegSeparate'

Two passes through the crossbar array for positive and negative inputs separately. The output of the two passes are added in analog and then passed once through ADC stage (which also applies the output noise, range clipping, output non-linearity etc.).

Caution

Only supported for RPUCuda tiles (based on C++ not pure torch)

POS_NEG_SEPARATE_DIGITAL_SUM = 'PosNegSeparateDigitalSum'

Two passes through the crossbar array for positive and negative inputs separately. The ADC output stage is applied to each pass separately and the results are summed in full precision (i.e. in digital).

Caution

Only supported for RPUCuda tiles (based on C++ not pure torch)

SPLIT_MODE = 'SplitMode'

Split PWM bits into two phases to speedup MAC and increase energy efficiency (may sacrifice some accuracy).

Caution

Only supported for TorchInferenceRPUConfigIRDropT

class aihwkit.simulator.parameters.enums.BoundManagementType(value)

Bases: Enum

Bound management type.

In the case Iterative the MAC is iteratively recomputed with inputs iteratively halved, when the output bound was hit.

Caution

Bound management is only available for the forward pass. It will be ignored when used for the backward pass.

ITERATIVE = 'Iterative'

Iteratively recomputes input scale set to \(\alpha\leftarrow\alpha/2\).

It iteratively recomputes the bounds up to limit of passes (given by max_bm_factor or max_bm_res).

ITERATIVE_WORST_CASE = 'IterativeWorstCase'

Worst case bound management.

Uses AbsMax noise management for the first pass and only when output bound is hit, the AbsMaxNPSum for the second. Thus, at most 2 passes are computed.

NONE = 'None'

No bound management.

SHIFT = 'Shift'

Shift bound management.

Shifts the output by adding the difference output_bound - max_output to the analog output value. This is only useful to increase the dynamic range before the softmax, where the max can be safely.

Note

Shifting needs hardware implementations.

class aihwkit.simulator.parameters.enums.CountLRFeedbackPolicy(value)

Bases: Enum

Depreciated. Legacy.

EXTERN = 'Extern'

NONE = 'None'

PAST_MOMENTUM = 'PastMomentum'

class aihwkit.simulator.parameters.enums.NoiseManagementType(value)

Bases: Enum

Noise management type.

Noise management determines a factor \(\alpha\) how the input is reduced:

\[\mathbf{y} = \alpha\;F_\text{analog-mac}\left(\mathbf{x}/\alpha\right)\]

ABS_MAX = 'AbsMax'

Use \(\alpha\equiv\max{|\mathbf{x}|}\).

ABS_MAX_NP_SUM = 'AbsMaxNPSum'

Assume weight value is constant and given by nm_assumed_wmax.

Takes a worst case scenario of the weight matrix to calculate the input scale to ensure that output is not clipping. Assumed weight value is constant and given by nm_assumed_wmax.

AVERAGE_ABS_MAX = 'AverageAbsMax'

Moment-based scale input scale estimation.

Computes the average abs max over the mini-batch and applies nm_decay to update the value with the history.

Note

nm_decay is 1-momentum and always given in mini-batches. However, the CUDA implementation does not discount values within mini-batches, whereas the CPU implementation does.

CONSTANT = 'Constant'

A constant value (given by parameter nm_thres).

MAX = 'Max'

Use \(\alpha\equiv\max{\mathbf{x}}\).

NONE = 'None'

No noise management.

class aihwkit.simulator.parameters.enums.PulseType(value)

Bases: Enum

Pulse type.

DETERMINISTIC_IMPLICIT = 'DeterministicImplicit'

Coincidences are computed in deterministic manner.

Coincidences are calculated by \(b_l x_q d_q\) where BL is the desired bit length (possibly subject to dynamic adjustments using update_bl_management) and \(x_q\) and \(d_q\) are the quantized input and error values, respectively, normalized to the range \(0,\ldots,1\). It can be shown that explicit bit lines exist that generate these coincidences.

MEAN_COUNT = 'MeanCount'

Coincidence based in prob (\(p_a p_b\)).

NONE = 'None'

Floating point update instead of pulses.

NONE_WITH_DEVICE = 'NoneWithDevice'

Floating point like None, but with analog devices (e.g. weight clipping).

STOCHASTIC = 'Stochastic'

Two passes for plus and minus (only CPU).

STOCHASTIC_COMPRESSED = 'StochasticCompressed'

Generates actual stochastic bit lines.

Plus and minus pulses are taken in the same pass.

class aihwkit.simulator.parameters.enums.RPUDataType(value)

Bases: Enum

Data type for the C++ simulation of the analog tiles.

Note

FLOAT is the default. Other data types need special compilation options to be enabled.

DOUBLE = 'double'

Float64 (double) precision format.

FLOAT = 'float'

Float32 (single) precision format.

HALF = 'half'

Float16 (half) precision format.

as_torch()

Returns corresponding torch dtype.

Return type:

dtype

class aihwkit.simulator.parameters.enums.VectorUnitCellUpdatePolicy(value)

Bases: Enum

Vector unit cell update policy.

ALL = 'All'

All devices updated simultaneously.

SINGLE_FIXED = 'SingleFixed'

Device index is not changed. Can be set initially and/or updated on the fly.

SINGLE_RANDOM = 'SingleRandom'

A single device is selected by random choice each mini-batch.

SINGLE_SEQUENTIAL = 'SingleSequential'

Each device one at a time in sequence.

class aihwkit.simulator.parameters.enums.WeightClipType(value)

Bases: Enum

Weight clipper type.

AVERAGE_CHANNEL_MAX = 'AverageChannelMax'

Calculates the abs max of each output channel (row of the weight matrix) and takes the average as clipping value for all.

FIXED_VALUE = 'FixedValue'

Clip to fixed value give, symmetrical around zero.

LAYER_GAUSSIAN = 'LayerGaussian'

Calculates the second moment of the whole weight matrix and clips at sigma times the result symmetrically around zero.

NONE = 'None'

None.

class aihwkit.simulator.parameters.enums.WeightModifierType(value)

Bases: Enum

Weight modifier type.

ADD_NORMAL = 'AddNormal'

Additive Gaussian noise.

COPY = 'Copy'

Legacy. No explicit weight modifier, however, pdrop is still observed.

Use DROP_CONNECT or NONE instead.

DISCRETIZE = 'Discretize'

Quantize the weights.

DISCRETIZE_ADD_NORMAL = 'DiscretizeAddNormal'

First discretize and then additive Gaussian noise.

DOREFA = 'DoReFa'

DoReFa discretization.

DROP_CONNECT = 'DropConnect'

Drop connect.

Note that if “pdrop > 0” this is applied. It drop connect can be applied in conjunction to other modifiers as well.

MULT_NORMAL = 'MultNormal'

Multiplicative Gaussian noise.

NONE = 'None'

No weight modifier. Nothing happens to the weight.

PCM_NOISE = 'PCMNoise'

PCM programming / drift noise model added to the weight matrix during training.

POLY = 'Poly'

Nth order Polynomial noise model (in terms of the weight value).

In detail, for the duration of a mini-batch, each weight will be added a Gaussian random number with the standard deviation of \(\sigma_\text{wnoise} (c_0 + c_1 w_{ij}/\omega + c_N w_{ij}^N/\omega^N)\), where \(omega\) is either the actual max weight (if rel_to_actual_wmax is set) or the value assumed_wmax.

PROG_NOISE = 'ProgNoise'

Programming noise model added to the weight matrix during training. Same as “POLY”, except that weights are ensured to keep the sign with noise mirrored at zero.

class aihwkit.simulator.parameters.enums.WeightNoiseType(value)

Bases: Enum

Output weight noise type.

The weight noise is applied for each MAC computation, while not touching the actual weight matrix but referring it to the output.

\[y_i = \sum_j w_{ij}+\xi_{ij}\]

ADDITIVE_CONSTANT = 'AdditiveConstant'

The \(\xi\sim{\cal N}(0,\sigma)\) thus all are Gaussian distributed.

\(\sigma\) is determined by w_noise.

NONE = 'None'

No weight noise.

PCM_READ = 'PCMRead'

Output-referred PCM-like read noise.

Output-referred PCM-like read noise that scales with the amount of current generated for each output line and thus scales with both conductance values and input strength.

The same general for is taken as for PCM-like statistical model of the 1/f noise during inference, see aihwkit.inference.noise.pcm.PCMLikeNoiseModel.

class aihwkit.simulator.parameters.enums.WeightRemapType(value)

Bases: Enum

Weight clipper type.

CHANNELWISE_SYMMETRIC = 'ChannelwiseSymmetric'

Remap each column (output channel) in respect to the absolute max.

LAYERWISE_SYMMETRIC = 'LayerwiseSymmetric'

Remap according to the absolute max of the full weight matrix.

NONE = 'None'

None.