aihwkit.simulator.presets.utils module

Utils for configurations presets for resistive processing units.

class aihwkit.simulator.presets.utils.PresetIOParameters(bm_test_negative_bound=True, bound_management=<BoundManagementType.ITERATIVE: 'Iterative'>, inp_bound=1.0, inp_noise=0.0, inp_res=0.007936507936507936, inp_sto_round=False, is_perfect=False, max_bm_factor=1000, max_bm_res=0.25, nm_thres=0.0, noise_management=<NoiseManagementType.ABS_MAX: 'AbsMax'>, out_bound=20.0, out_noise=0.1, out_res=0.00196078431372549, out_scale=1.0, out_sto_round=False, w_noise=0.0, w_noise_type=<WeightNoiseType.NONE: 'None'>, ir_drop=0.0, ir_drop_g_ratio=571428.5714285715)[source]

Bases: aihwkit.simulator.configs.utils.IOParameters

Preset for the forward and backward pass parameters.

This defines the default (peripheral) hardware configurations used for most presets.

Currently, we assume 7 bit DAC (without stochastic rounding) and 9 bit ADC (including the sign) and a fixed dynamic range ratio. The dynamic range ratio is defined by how many fully-on inputs would saturate the output ADC when all weights are set maximal conductances. This value is set to 20 here, as the weight range is normalized to \(-1,\ldots 1\).

Moreover, the output noise (additive Gaussian) is set to 0.1, which is on the order of 1 LSB of the ADC.

By default, we turned additional weight noise off, however, some presets might turn it on as required by the device specification.

Finally, we assume by default that the device is run with bound management (see BoundManagementType) and noise management (see NoiseManagementType) turned on to ITERATIVE and ABS_MAX, respectively.

Parameters
Return type

None

bm_test_negative_bound: bool = True
bound_management: aihwkit.simulator.configs.utils.BoundManagementType = 'Iterative'

Type of bound management, see BoundManagementType.

Caution

Bound management is only available for the forward pass. It will be ignored when used for the backward pass.

inp_bound: float = 1.0

Input bound and ranges for the digital-to-analog converter (DAC).

inp_noise: float = 0.0

Std deviation of Gaussian input noise (\(\sigma_\text{inp}\)).

i.e. noisiness of the analog input (at the stage after DAC and before the multiplication).

inp_res: float = 0.007936507936507936

Number of discretization steps for DAC (\(\le0\) means infinite steps) or resolution (1/steps).

inp_sto_round: bool = False

Whether to enable stochastic rounding of DAC.

ir_drop: float = 0.0

Scale of IR drop along the inputs (rows of the weight matrix).

The IR-drop is calculated assuming that the first input is farthest away from the output channel. The expected drop is approximating the steady-state voltage distributions and depends on the input current.

ir_drop_g_ratio: float = 571428.5714285715

Physical ratio of wire conductance from one cell to the next to physical max conductance of a device.

Default is compute with 5mS maximal conductance set state and 0.35 Ohm wire resistance.

is_perfect: bool = False

Short-cut to compute a perfect forward pass.

If True, it assumes an ideal forward pass (e.g. no bound, ADC etc…). Will disregard all other settings in this case.

max_bm_factor: int = 1000

Maximal bound management factor.

If this factor is reached then the iterative process is stopped.

max_bm_res: float = 0.25

Limit the maximal number of iterations of the bound management.

Another way to limit the maximal number of iterations of the bound management. The max effective resolution number of the inputs, e.g. use \(1/4\) for 2 bits.

nm_thres: float = 0.0

Constant noise management value for type Constant.

In other cases, this is a upper threshold \(\theta\) above which the noise management factor is saturated. E.g. for AbsMax:

\begin{equation*} \alpha=\begin{cases}\max_i|x_i|, & \text{if} \max_i|x_i|<\theta \\ \theta, & \text{otherwise}\end{cases} \end{equation*}

Caution

If nm_thres is set (and type is not Constant), the noise management will clip some large input values, in favor of having a better SNR for smaller input values.

noise_management: aihwkit.simulator.configs.utils.NoiseManagementType = 'AbsMax'

Type of noise management, see NoiseManagementType.

out_bound: float = 20.0

Output bound and ranges for analog-to-digital converter (ADC).

out_noise: float = 0.1

Std deviation of Gaussian output noise (\(\sigma_\text{out}\)).

i.e. noisiness of device summation at the output.

out_res: float = 0.00196078431372549

Number of discretization steps for ADC or resolution.

Number of discretization steps for ADC (\(<=0\) means infinite steps) or resolution (1/steps).

out_scale: float = 1.0

Additional fixed scalar factor.

out_sto_round: bool = False

Whether to enable stochastic rounding of ADC.

w_noise: float = 0.0

Scale of output referred weight noise (\(\sigma_w\)) for a given w_noise_type.

w_noise_type: aihwkit.simulator.configs.utils.WeightNoiseType = 'None'

Type as specified in OutputWeightNoiseType.

Note

This noise us applied each time anew as it is referred to the output. It will not change the conductance values of the weight matrix. For the latter one can apply diffuse_weights().

class aihwkit.simulator.presets.utils.PresetUpdateParameters(desired_bl=31, fixed_bl=True, pulse_type=<PulseType.STOCHASTIC_COMPRESSED: 'StochasticCompressed'>, res=0, x_res_implicit=0, d_res_implicit=0, sto_round=False, update_bl_management=True, update_management=True)[source]

Bases: aihwkit.simulator.configs.utils.UpdateParameters

Preset for the general update behavior.

This defines the default update configurations used for most presets. Presets might override this default behavior to implement other analog SGD optimizers.

Parallel analog update is the default. We assume stochastic pulse to do the parallel update in analog, as described in Gokmen & Vlasov, Front. Neurosci. 2016.

Moreover, we assume that the pulse length is dynamically adjusted with a maximal pulse length of 31 pulses.

Parameters

  • desired_bl (int) –

  • fixed_bl (bool) –

  • pulse_type (aihwkit.simulator.configs.utils.PulseType) –

  • res (float) –

  • x_res_implicit (float) –

  • d_res_implicit (float) –

  • sto_round (bool) –

  • update_bl_management (bool) –

  • update_management (bool) –

Return type

None

d_res_implicit: float = 0

Resolution of each quantization step for the error d.

Resolution (ie. bin width) of each quantization step for the error d in case of DeterministicImplicit pulse trains. See PulseTypeMap for details.

desired_bl: int = 31

Desired length of the pulse trains.

For update BL management, it is the maximal pulse train length.

fixed_bl: bool = True

Whether to fix the length of the pulse trains.

See also update_bl_management.

In case of True (where dw_min is the mean minimal weight change step size) it is:

BL = desired_BL
A = B =  sqrt(learning_rate / (dw_min * BL));

In case of False:

if dw_min * desired_BL < learning_rate:
    A = B = 1;
    BL = ceil(learning_rate / dw_min;
else:
    # same as for fixed_BL=True
pulse_type: aihwkit.simulator.configs.utils.PulseType = 'StochasticCompressed'

Switching between different pulse types.

See also PulseTypeMap for details.

Important

Pulsing can also be turned off in which case the update is done as if in floating point and all other update related parameter are ignored.

res: float = 0

Resolution of the update probability for the stochastic bit line generation.

Resolution ie. bin width in 0..1) of the update probability for the stochastic bit line generation. Use -1 for turning discretization off. Can be given as number of steps as well.

sto_round: bool = False

Whether to enable stochastic rounding.

update_bl_management: bool = True

Whether to enable dynamical adjustment of A,``B``,and BL:

BL = ceil(learning_rate * abs(x_j) * abs(d_i) / weight_granularity);
BL  = min(BL,desired_BL);
A = B = sqrt(learning_rate / (weight_granularity * BL));

The weight_granularity is usually equal to dw_min.

update_management: bool = True

Whether to apply additional scaling.

After the above setting an additional scaling (always on when using update_bl_management`) is applied to account for the different input strengths. If

\[\gamma \equiv \max_i |x_i| / \max_j |d_j|\]

is the ratio between the two maximal inputs, then A is additionally scaled by \(\gamma\) and B is scaled by \(1/\gamma\).

x_res_implicit: float = 0

Resolution of each quantization step for the inputs x.

Resolution (ie. bin width) of each quantization step for the inputs x in case of DeterministicImplicit pulse trains. See PulseTypeMap for details.