aihwkit.simulator.presets.utils module

Utils for configurations presets for resistive processing units.

class aihwkit.simulator.presets.utils.PresetIOParameters(is_perfect=False, mv_type=AnalogMVType.ONE_PASS, inp_bound=1.0, inp_noise=0.0, inp_res=0.007936507936507936, inp_sto_round=False, inp_asymmetry=0.0, out_bound=20.0, out_noise=0.1, out_noise_std=0.0, out_res=0.00196078431372549, out_sto_round=False, out_scale=1.0, out_asymmetry=0.0, bound_management=BoundManagementType.ITERATIVE, noise_management=NoiseManagementType.ABS_MAX, w_noise=0.0, w_noise_type=WeightNoiseType.NONE, ir_drop=0.0, ir_drop_g_ratio=571428.5714285714, out_nonlinearity=0.0, out_nonlinearity_std=0.0, slope_calibration=0.0, v_offset_std=0.0, v_offset_w_min=-1.0, r_series=0.0, w_read_asymmetry_dtod=0.0, max_bm_factor=1000, max_bm_res=0.25, bm_test_negative_bound=True, nm_thres=0.0)

Bases: IOParameters

Preset for the forward and backward pass parameters.

This defines the default (peripheral) hardware configurations used for most presets.

Currently, we assume 7 bit DAC (without stochastic rounding) and 9 bit ADC (including the sign) and a fixed dynamic range ratio. The dynamic range ratio is defined by how many fully-on inputs would saturate the output ADC when all weights are set maximal conductances. This value is set to 20 here, as the weight range is normalized to \(-1,\ldots 1\).

Moreover, the output noise (additive Gaussian) is set to 0.1, which is on the order of 1 LSB of the ADC.

By default, we turned additional weight noise off, however, some presets might turn it on as required by the device specification.

Finally, we assume by default that the device is run with bound management (see BoundManagementType) and noise management (see NoiseManagementType) turned on to ITERATIVE and ABS_MAX, respectively.

Parameters:

• is_perfect (bool) –

• mv_type (AnalogMVType) –

• inp_bound (float) –

• inp_noise (float) –

• inp_res (float) –

• inp_sto_round (bool) –

• inp_asymmetry (float) –

• out_bound (float) –

• out_noise (float) –

• out_noise_std (float) –

• out_res (float) –

• out_sto_round (bool) –

• out_scale (float) –

• out_asymmetry (float) –

• bound_management (BoundManagementType) –

• noise_management (NoiseManagementType) –

• w_noise (float) –

• w_noise_type (WeightNoiseType) –

• ir_drop (float) –

• ir_drop_g_ratio (float) –

• out_nonlinearity (float) –

• out_nonlinearity_std (float) –

• slope_calibration (float) –

• v_offset_std (float) –

• v_offset_w_min (float) –

• r_series (float) –

• w_read_asymmetry_dtod (float) –

• max_bm_factor (int) –

• max_bm_res (float) –

• bm_test_negative_bound (bool) –

• nm_thres (float) –

bound_management: BoundManagementType = 'Iterative'

Type of bound management, see BoundManagementType.

Caution

Bound management is only available for the forward pass. It will be ignored when used for the backward pass.

inp_res: float = 0.007936507936507936

Number of discretization steps for DAC (\(\le0\) means infinite steps) or resolution (1/steps).

inp_sto_round: bool = False

Whether to enable stochastic rounding of DAC.

noise_management: NoiseManagementType = 'AbsMax'

Type of noise management, see NoiseManagementType.

out_bound: float = 20.0

Output bound and ranges for analog-to-digital converter (ADC).

out_noise: float = 0.1

Output noise strength at each output of a tile.

This sets the std-deviation of the Gaussian output noise (\(\sigma_\text{out}\)) at each output, i.e. noisiness of device summation at the output.

out_res: float = 0.00196078431372549

Number of discretization steps for ADC or resolution.

Number of discretization steps for ADC (\(<=0\) means infinite steps) or resolution (1/steps).

w_noise: float = 0.0

Scale of output referred weight noise (\(\sigma_w\)) for a given w_noise_type.

w_noise_type: WeightNoiseType = 'None'

Type as specified in OutputWeightNoiseType.

Note

This noise us applied each time anew as it is referred to the output. It will not change the conductance values of the weight matrix. For the latter one can apply diffuse_weights().

class aihwkit.simulator.presets.utils.PresetUpdateParameters(desired_bl=31, fixed_bl=True, pulse_type=PulseType.STOCHASTIC_COMPRESSED, res=0, x_res_implicit=0, d_res_implicit=0, d_sparsity=False, sto_round=False, update_bl_management=True, update_management=True, um_grad_scale=1.0)

Bases: UpdateParameters

Preset for the general update behavior.

This defines the default update configurations used for most presets. Presets might override this default behavior to implement other analog SGD optimizers.

Parallel analog update is the default. We assume stochastic pulse to do the parallel update in analog, as described in Gokmen & Vlasov, Front. Neurosci. 2016.

Moreover, we assume that the pulse length is dynamically adjusted with a maximal pulse length of 31 pulses.

Parameters:

• desired_bl (int) –

• fixed_bl (bool) –

• pulse_type (PulseType) –

• res (float) –

• x_res_implicit (float) –

• d_res_implicit (float) –

• d_sparsity (bool) –

• sto_round (bool) –

• update_bl_management (bool) –

• update_management (bool) –

• um_grad_scale (float) –

desired_bl: int = 31

Desired length of the pulse trains.

For update BL management, it is the maximal pulse train length.

pulse_type: PulseType = 'StochasticCompressed'

Switching between different pulse types.

See also PulseTypeMap for details.

Important

Pulsing can also be turned off in which case the update is done as if in floating point and all other update related parameter are ignored.

update_bl_management: bool = True

Whether to enable dynamical adjustment of A,``B``,and BL:

BL = ceil(learning_rate * abs(x_j) * abs(d_i) / weight_granularity);
BL  = min(BL,desired_BL);
A = B = sqrt(learning_rate / (weight_granularity * BL));

The weight_granularity is usually equal to dw_min.

update_management: bool = True

Whether to apply additional scaling.

After the above setting an additional scaling (always on when using update_bl_management`) is applied to account for the different input strengths. If

\[\gamma \equiv \max_i |x_i| / (\alpha \max_j |d_j|)\]

is the ratio between the two maximal inputs, then A is additionally scaled by \(\gamma\) and B is scaled by \(1/\gamma\).

The gradient scale \(\alpha\) can be set with um_grad_scale

class aihwkit.simulator.presets.utils.StandardIOParameters(is_perfect=False, mv_type=AnalogMVType.ONE_PASS, inp_bound=1.0, inp_noise=0.0, inp_res=0.003937007874015748, inp_sto_round=False, inp_asymmetry=0.0, out_bound=10.0, out_noise=0.04, out_noise_std=0.0, out_res=0.003937007874015748, out_sto_round=False, out_scale=1.0, out_asymmetry=0.0, bound_management=BoundManagementType.ITERATIVE, noise_management=NoiseManagementType.ABS_MAX, w_noise=0.0, w_noise_type=WeightNoiseType.NONE, ir_drop=0.0, ir_drop_g_ratio=571428.5714285714, out_nonlinearity=0.0, out_nonlinearity_std=0.0, slope_calibration=0.0, v_offset_std=0.0, v_offset_w_min=-1.0, r_series=0.0, w_read_asymmetry_dtod=0.0, max_bm_factor=1000, max_bm_res=0.25, bm_test_negative_bound=True, nm_thres=0.0)

Bases: IOParameters

Preset for the forward and backward pass parameters.

Preset that is more aligned with the the forward pass of StandardHWATrainingPreset, as it assumes the same DAC/ ADC resolution, output bound and output noise (see also Rasch et al. ArXiv 2023 for a discussion)

However, here, noise and bound mangement is turned on by default, and IR-drop as well as short-term weight noise is set to 0 by default.

Parameters:

• is_perfect (bool) –

• mv_type (AnalogMVType) –

• inp_bound (float) –

• inp_noise (float) –

• inp_res (float) –

• inp_sto_round (bool) –

• inp_asymmetry (float) –

• out_bound (float) –

• out_noise (float) –

• out_noise_std (float) –

• out_res (float) –

• out_sto_round (bool) –

• out_scale (float) –

• out_asymmetry (float) –

• bound_management (BoundManagementType) –

• noise_management (NoiseManagementType) –

• w_noise (float) –

• w_noise_type (WeightNoiseType) –

• ir_drop (float) –

• ir_drop_g_ratio (float) –

• out_nonlinearity (float) –

• out_nonlinearity_std (float) –

• slope_calibration (float) –

• v_offset_std (float) –

• v_offset_w_min (float) –

• r_series (float) –

• w_read_asymmetry_dtod (float) –

• max_bm_factor (int) –

• max_bm_res (float) –

• bm_test_negative_bound (bool) –

• nm_thres (float) –

bound_management: BoundManagementType = 'Iterative'

Type of bound management, see BoundManagementType.

Caution

Bound management is only available for the forward pass. It will be ignored when used for the backward pass.

inp_res: float = 0.003937007874015748

Number of discretization steps for DAC (\(\le0\) means infinite steps) or resolution (1/steps).

inp_sto_round: bool = False

Whether to enable stochastic rounding of DAC.

noise_management: NoiseManagementType = 'AbsMax'

Type of noise management, see NoiseManagementType.

out_bound: float = 10.0

Output bound and ranges for analog-to-digital converter (ADC).

out_noise: float = 0.04

Output noise strength at each output of a tile.

This sets the std-deviation of the Gaussian output noise (\(\sigma_\text{out}\)) at each output, i.e. noisiness of device summation at the output.

out_res: float = 0.003937007874015748

Number of discretization steps for ADC or resolution.

Number of discretization steps for ADC (\(<=0\) means infinite steps) or resolution (1/steps).

w_noise: float = 0.0

Scale of output referred weight noise (\(\sigma_w\)) for a given w_noise_type.

w_noise_type: WeightNoiseType = 'None'

Type as specified in OutputWeightNoiseType.

Note

This noise us applied each time anew as it is referred to the output. It will not change the conductance values of the weight matrix. For the latter one can apply diffuse_weights().