aihwkit.simulator.configs.utils module¶
Utility parameters for resistive processing units.
- class aihwkit.simulator.configs.utils.DriftParameter(nu=0.0, t_0=1.0, reset_tol=1e-07, nu_dtod=0.0, nu_std=0.0, wg_ratio=1.0, g_offset=0.0, w_offset=0.0, nu_k=0.0, log_g0=0.0, w_noise_std=0.0)[source]¶
Bases:
aihwkit.simulator.configs.utils.SimpleDriftParameter
Parameter for a power law drift.
The drift is based on the model described by Oh et al (2019).
It computes: .. math:
w_{ij}*\left(\frac{t + \Delta t}{t_0}\right)^(-\nu^\text{actual}_{ij})
where the drift coefficient is drawn once at the beginning and might depend on device. It also can depend on the actual weight value.
The actual drift coefficient is computed as: .. math:
\nu_{ij}^\text{actual} = \nu_{ij} - \nu_k \log \frac{(w_{ij} - w_\text{off}) / r_\text{wg} + g_\text{off}}{G_0} + \nu\sigma_\nu\xi
here \(w_{ij}\) is the actual weight and nu_{ij} fixed for each device given by the mean \(\nu\) and the device-to-device variation: \(\nu_{ij} = \nu + \nu_dtod\nu\xi\) and are only drawn once at the beginning (tile instantiation). xi is Gaussian noise.
Note
If the weight has changed from the last drift call (determined by the
reset_tol
parameter), for instance due to update, decay or noise, then the drift time \(t\) will be reset and start from new, however, the drift coefficients \(\nu_{ij}\) are not changed. On the other hand, if the weights has not changed since last call, \(t\) will accumulate the time.Caution
Note that the drift coefficient does not depend on the initially programmed weight value at \(t=0\) in the current implementation (ie G0 is a constant for all devices), but instead on the actual weight. In some materials (e.g. phase changed materials), that might be not accurate.
- Parameters
nu (float) –
t_0 (float) –
reset_tol (float) –
nu_dtod (float) –
nu_std (float) –
wg_ratio (float) –
g_offset (float) –
w_offset (float) –
nu_k (float) –
log_g0 (float) –
w_noise_std (float) –
- Return type
None
- g_offset: float = 0.0¶
g_min
to convert to physical units.
- log_g0: float = 0.0¶
Log g0.
- nu_dtod: float = 0.0¶
Device-to-device variation of the \(\nu\) values.
- nu_k: float = 0.0¶
nu with \(W\).
That is \(\nu(R) = nu_0 - k \log(G/G_0)\). See Oh et al. for details.
- Type
Variation of math
- nu_std: float = 0.0¶
Cycle-to-cycle variation of \(\nu\).
A more realistic way to add noise of the drift might be using
w_noise_std
.
- w_noise_std: float = 0.0¶
Additional weight noise (Gaussian diffusion) added to the weights after the drift is applied.
- w_offset: float = 0.0¶
w(g_min)
, i.e. to what valueg_min
is mapped to in w-space.
- wg_ratio: float = 1.0¶
(w_max-w_min)/(g_max-g_min)
to convert to physical units.
- class aihwkit.simulator.configs.utils.IOParameters(is_perfect=False, mv_type=AnalogMVType.ONE_PASS, inp_bound=1.0, inp_noise=0.0, inp_res=0.007936507936507936, inp_sto_round=False, inp_asymmetry=0.0, out_bound=12.0, out_noise=0.06, out_res=0.00196078431372549, out_sto_round=False, out_scale=1.0, out_asymmetry=0.0, bound_management=BoundManagementType.ITERATIVE, noise_management=NoiseManagementType.ABS_MAX, w_noise=0.0, w_noise_type=WeightNoiseType.NONE, ir_drop=0.0, ir_drop_g_ratio=571428.5714285714, out_nonlinearity=0.0, out_nonlinearity_std=0.0, slope_calibration=0.0, v_offset_std=0.0, v_offset_w_min=- 1.0, r_series=0.0, w_read_asymmetry_dtod=0.0, max_bm_factor=1000, max_bm_res=0.25, bm_test_negative_bound=True, nm_thres=0.0)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter that define the analog-matvec (forward / backward) and peripheral digital input-output behavior.
Here one can enable analog-digital conversion, dynamic input scaling, and define the properties of the analog-matvec computations, such as noise and non-idealities (e.g. IR-drop).
- Parameters
is_perfect (bool) –
mv_type (aihwkit.simulator.configs.enums.AnalogMVType) –
inp_bound (float) –
inp_noise (float) –
inp_res (float) –
inp_sto_round (bool) –
inp_asymmetry (float) –
out_bound (float) –
out_noise (float) –
out_res (float) –
out_sto_round (bool) –
out_scale (float) –
out_asymmetry (float) –
bound_management (aihwkit.simulator.configs.enums.BoundManagementType) –
noise_management (aihwkit.simulator.configs.enums.NoiseManagementType) –
w_noise (float) –
w_noise_type (aihwkit.simulator.configs.enums.WeightNoiseType) –
ir_drop (float) –
ir_drop_g_ratio (float) –
out_nonlinearity (float) –
out_nonlinearity_std (float) –
slope_calibration (float) –
v_offset_std (float) –
v_offset_w_min (float) –
r_series (float) –
w_read_asymmetry_dtod (float) –
max_bm_factor (int) –
max_bm_res (float) –
bm_test_negative_bound (bool) –
nm_thres (float) –
- Return type
None
- bm_test_negative_bound: bool = True¶
- bound_management: aihwkit.simulator.configs.enums.BoundManagementType = 'Iterative'¶
Type of bound management, see
BoundManagementType
.Caution
Bound management is only available for the forward pass. It will be ignored when used for the backward pass.
- inp_asymmetry: float = 0.0¶
Input asymmetry \(a_ ext{input}\).
Input of the negative input pass is scaled by \((1 - a_ ext{input})\).
Note
This setting has only effect in case of and
AnalogMVType
that uses separate passes for positive and negative inputs.
- inp_bound: float = 1.0¶
Input bound and ranges for the digital-to-analog converter (DAC).
- inp_noise: float = 0.0¶
Std deviation of Gaussian input noise (\(\sigma_\text{inp}\)).
i.e. noisiness of the analog input (at the stage after DAC and before the multiplication).
- inp_res: float = 0.007936507936507936¶
Number of discretization steps for DAC (\(\le0\) means infinite steps) or resolution (1/steps).
- inp_sto_round: bool = False¶
Whether to enable stochastic rounding of DAC.
- ir_drop: float = 0.0¶
Scale of IR drop along the inputs (rows of the weight matrix).
The IR-drop is calculated assuming that the first input is farthest away from the output channel. The expected drop is approximating the steady-state voltage distributions and depends on the input current.
- ir_drop_g_ratio: float = 571428.5714285714¶
Physical ratio of wire conductance from one cell to the next to physical max conductance of a device.
Default is compute with 5mS maximal conductance set state and 0.35 Ohm wire resistance.
- is_perfect: bool = False¶
Short-cut to compute a perfect forward pass.
If
True
, it assumes an ideal forward pass (e.g. no bound, ADC etc…). Will disregard all other settings in this case.
- max_bm_factor: int = 1000¶
Maximal bound management factor.
If this factor is reached then the iterative process is stopped.
- max_bm_res: float = 0.25¶
Limit the maximal number of iterations of the bound management.
Another way to limit the maximal number of iterations of the bound management. The max effective resolution number of the inputs, e.g. use \(1/4\) for 2 bits.
- mv_type: aihwkit.simulator.configs.enums.AnalogMVType = 'OnePass'¶
Selects the type of analog mat-vec computation. See
AnalogMVType
for details.
- nm_thres: float = 0.0¶
Constant noise management value for
type
Constant
.In other cases, this is a upper threshold \(\theta\) above which the noise management factor is saturated. E.g. for AbsMax:
\begin{equation*} \alpha=\begin{cases}\max_i|x_i|, & \text{if} \max_i|x_i|<\theta \\ \theta, & \text{otherwise}\end{cases} \end{equation*}Caution
If
nm_thres
is set (and type is notConstant
), the noise management will clip some large input values, in favor of having a better SNR for smaller input values.
- noise_management: aihwkit.simulator.configs.enums.NoiseManagementType = 'AbsMax'¶
Type of noise management, see
NoiseManagementType
.
- out_asymmetry: float = 0.0¶
Output asymmetry \(a_ ext{output}\).
Output of the negative input pass is scaled by \((1 - a_ ext{output})\).
Note
This setting has only effect in case of and
AnalogMVType
that uses separate passes for positive and negative inputs.
- out_bound: float = 12.0¶
Output bound and ranges for analog-to-digital converter (ADC).
- out_noise: float = 0.06¶
Std deviation of Gaussian output noise (\(\sigma_\text{out}\)).
i.e. noisiness of device summation at the output.
- out_nonlinearity: float = 0.0¶
S-shaped non-linearity applied to the analog output.
Output non-linearity applies an S-shaped non-linearity to the analog output (before the ADC), i.e. :math:` rac{y_i}{1 + n_i*|y_i|}` where \(n_i\) is drawn at the instantiation time by:
out_nonlinearity / out_bound * (1 + out_nonlinearity_std * rand)
- out_nonlinearity_std: float = 0.0¶
Output-to-output non linearity variation.
- out_res: float = 0.00196078431372549¶
Number of discretization steps for ADC or resolution.
Number of discretization steps for ADC (\(<=0\) means infinite steps) or resolution (1/steps).
- out_scale: float = 1.0¶
Additional fixed scalar factor.
- out_sto_round: bool = False¶
Whether to enable stochastic rounding of ADC.
- r_series: float = 0.0¶
Series resistance in fraction of the total output current.
- slope_calibration: float = 0.0¶
Models a calibration process of the output non-linearity (and r-series).
This is the relative value in the output range where the slope of the non-linearity should have slope 1. E.g. 0.5 would be at half-out range.
- v_offset_std: float = 0.0¶
Voltage offset variation.
The output is multiplied by a systematic factor set for each output line at time of instantiation, e.g. \((1 - v_i)\) for the coding device and \((1 + v_i)\) for the reference device (assuming differential reads).
- v_offset_w_min: float = -1.0¶
Voltage offset for an implicit reference unit.
- w_noise: float = 0.0¶
Scale of output referred weight noise (\(\sigma_w\)) for a given
w_noise_type
.
- w_noise_type: aihwkit.simulator.configs.enums.WeightNoiseType = 'None'¶
Type as specified in
OutputWeightNoiseType
.Note
This noise us applied each time anew as it is referred to the output. It will not change the conductance values of the weight matrix. For the latter one can apply
diffuse_weights()
.
- w_read_asymmetry_dtod: float = 0.0¶
Device polarity read dependence.
The negative inputs perceive a slightly different weight (e.g. pcm polarity dependence). Each device has a different factor, and the spread of this device-to-device variability can be set with
w_read_asymmetry_dtod
. A weight (given negative input) will be then scaled by \(1 - f_{ij}\) where \(f_{ij}\) is drawn from a Gaussian distribution (with zero mean and standard deviationw_read_asymmetry_dtod
).
- class aihwkit.simulator.configs.utils.InputRangeParameter(enable=False, init_value=3.0, init_from_data=100, init_std_alpha=3.0, decay=0.001, input_min_percentage=0.95, manage_output_clipping=True, output_min_percentage=0.95, gradient_scale=1.0, gradient_relative=True)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter related to input range learning
- Parameters
enable (bool) –
init_value (float) –
init_from_data (int) –
init_std_alpha (float) –
decay (float) –
input_min_percentage (float) –
manage_output_clipping (bool) –
output_min_percentage (float) –
gradient_scale (float) –
gradient_relative (bool) –
- Return type
None
- decay: float = 0.001¶
Decay rate for input range learning.
- enable: bool = False¶
Whether to enable to learn the input range. Note that if enable is
False
then no clip is applied.Note
The input bound (
forward.inp_bound
) is assumed to be 1 if enabled as the input range already scales the input into to the range \((-1, 1)\) by dividing the input to the type by itself and multiplying the output accordingly.Typically, noise and bound management should be set to NONE for the input range learning as it replaces the dynamic managements with a static but learned input bound. However, in some exceptional experimental cases one might want to enable the management techniques on top of the input range learning, so that no error is raised if they are not set to NONE.
- gradient_relative: bool = True¶
Whether to make the gradient of the input range learning relative to the current range value.
- gradient_scale: float = 1.0¶
Scale of the gradient magnitude (learning rate) for the input range learning.
- init_from_data: int = 100¶
Number of batches to use for initialization from data. Set 0 to turn off.
- init_std_alpha: float = 3.0¶
Standard deviation multiplier for initialization from data.
- init_value: float = 3.0¶
Initial setting of the input range in case of input range learning.
- input_min_percentage: float = 0.95¶
Decay is only applied if percentage of non-clipped values is above this value.
Note
The added gradient is (in case of non-clipped input percentage
percentage > input_min_percentage
):grad += decay * input_range
- manage_output_clipping: bool = True¶
Whether to increase the input range when output clipping occurs.
Caution
The output bound is taken from the
forward.out_bound
value, which has to exist. Noise and bound management have to be set to NONE if this feature is enabled otherwise aConfigError
is raised.
- output_min_percentage: float = 0.95¶
Increase of the input range is only applied if percentage of non-clipped output values is below this value.
Note
The gradient subtracted from the input range is (in case of
output_percentage < output_min_percentage
):grad -= (1.0 - output_percentage) * input_range
- class aihwkit.simulator.configs.utils.MapableRPU(mapping=<factory>)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Defines the mapping parameters and utility factories
- Parameters
mapping (aihwkit.simulator.configs.utils.MappingParameter) –
- Return type
None
- get_linear()[source]¶
Returns a AnalogLinear module as specified
- Return type
Union[Type[AnalogLinear], Type[AnalogLinearMapped]]
- mapping: aihwkit.simulator.configs.utils.MappingParameter¶
Parameter related to mapping weights to tiles for supporting modules.
- class aihwkit.simulator.configs.utils.MappingParameter(digital_bias=True, weight_scaling_omega=0.0, weight_scaling_columnwise=False, learn_out_scaling=False, out_scaling_columnwise=False, max_input_size=512, max_output_size=512)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter related to hardware design and the mapping of logical weight matrices to physical tiles.
Caution
Some of these parameters have only an effect for modules that support tile mappings.
- Parameters
digital_bias (bool) –
weight_scaling_omega (float) –
weight_scaling_columnwise (bool) –
learn_out_scaling (bool) –
out_scaling_columnwise (bool) –
max_input_size (int) –
max_output_size (int) –
- Return type
None
- digital_bias: bool = True¶
Whether the bias term is handled by the analog tile or kept in digital.
Note
Default is having a digital bias so that bias values are not stored onto the analog crossbar. This needs to be supported by the chip design. Set to False if the analog bias is instead situated on the the crossbar itself (as an extra column)
Note
digital_bias
is supported by all analog modules.
- learn_out_scaling: bool = False¶
Define (additional) out scales that are learnable parameter used to scale the output.
- max_input_size: int = 512¶
Maximal input size (number of columns) of the weight matrix that is handled on a single analog tile.
If the logical weight matrix size exceeds this size it will be split and mapped onto multiple analog tiles.
Caution
Only relevant for
Mapped
modules such asaihwkit.nn.modules.linear_mapped.AnalogLinearMapped
.
- max_output_size: int = 512¶
Maximal output size (number of rows) of the weight matrix that is handled on a single analog tile.
If the logical weight matrix size exceeds this size it will be split and mapped onto multiple analog tiles.
Caution
Only relevant for
Mapped
modules such asaihwkit.nn.modules.linear_mapped.AnalogLinearMapped
.
- out_scaling_columnwise: bool = False¶
Whether the learnable out scaling parameter enabled by
learn_out_scaling
is a scalar (False
) or learned for each output (True
).
- weight_scaling_columnwise: bool = False¶
Whether the weight matrix will be remapped column-wise over the maximum device allowed value.
- weight_scaling_omega: float = 0.0¶
omega_scale is a user defined parameter used to scale the weights while remapping these to cover the full range of values allowed.
- class aihwkit.simulator.configs.utils.PrePostProcessingParameter(input_range=<factory>)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter related to digital input and output processing, such as input clip learning.
- Parameters
input_range (aihwkit.simulator.configs.utils.InputRangeParameter) –
- Return type
None
- input_range: aihwkit.simulator.configs.utils.InputRangeParameter¶
- class aihwkit.simulator.configs.utils.PrePostProcessingRPU(pre_post=<factory>)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Defines the pre-post parameters and utility factories
- Parameters
pre_post (aihwkit.simulator.configs.utils.PrePostProcessingParameter) –
- Return type
None
- pre_post: aihwkit.simulator.configs.utils.PrePostProcessingParameter¶
Parameter related digital pre and post processing.
- class aihwkit.simulator.configs.utils.SimpleDriftParameter(nu=0.0, t_0=1.0, reset_tol=1e-07)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter for a simple power law drift.
The drift as a simple power law drift without device-to-device variation or conductance dependence.
It computes: .. math:
w_{ij}*\left(\frac{t + \Delta t}{t_0}\right)^(-\nu)
- Parameters
nu (float) –
t_0 (float) –
reset_tol (float) –
- Return type
None
- nu: float = 0.0¶
Average drift \(\nu\) value.
Need to non-zero to actually use the drift.
- reset_tol: float = 1e-07¶
Reset tolerance.
This should a number smaller than the expected weight change as it is used to detect any changes in the weight from the last drift call. Every change to the weight above this tolerance will reset the drift time.
Caution
Any write noise or diffusion on the weight might thus interfere with the drift.
- t_0: float = 1.0¶
Time between write and first read.
Usually assumed in milliseconds, however, it really determines the time units of
time_since_last_call
when calling the drift.
- class aihwkit.simulator.configs.utils.UpdateParameters(desired_bl=31, fixed_bl=True, pulse_type=PulseType.STOCHASTIC_COMPRESSED, res=0, x_res_implicit=0, d_res_implicit=0, sto_round=False, update_bl_management=True, update_management=True, um_grad_scale=1.0)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter that modify the update behaviour of a pulsed device.
- Parameters
desired_bl (int) –
fixed_bl (bool) –
pulse_type (aihwkit.simulator.configs.enums.PulseType) –
res (float) –
x_res_implicit (float) –
d_res_implicit (float) –
sto_round (bool) –
update_bl_management (bool) –
update_management (bool) –
um_grad_scale (float) –
- Return type
None
- d_res_implicit: float = 0¶
Resolution of each quantization step for the error
d
.Resolution (ie. bin width) of each quantization step for the error
d
in case of DeterministicImplicit pulse trains. SeePulseTypeMap
for details.
- desired_bl: int = 31¶
Desired length of the pulse trains.
For update BL management, it is the maximal pulse train length.
- fixed_bl: bool = True¶
Whether to fix the length of the pulse trains.
See also
update_bl_management
.In case of
True
(wheredw_min
is the mean minimal weight change step size) it is:BL = desired_BL A = B = sqrt(learning_rate / (dw_min * BL))
In case of
False
:if dw_min * desired_BL < learning_rate: A = B = 1 BL = ceil(learning_rate / dw_min else: # same as for fixed_BL=True
- pulse_type: aihwkit.simulator.configs.enums.PulseType = 'StochasticCompressed'¶
Switching between different pulse types.
See also
PulseTypeMap
for details.Important
Pulsing can also be turned off in which case the update is done as if in floating point and all other update related parameter are ignored.
- res: float = 0¶
Resolution of the update probability for the stochastic bit line generation.
Resolution ie. bin width in
0..1
) of the update probability for the stochastic bit line generation. Use -1 for turning discretization off. Can be given as number of steps as well.
- sto_round: bool = False¶
Whether to enable stochastic rounding.
- um_grad_scale: float = 1.0¶
Scales the gradient for the update management.
The factor \(\alpha\) for the
update_management
. If smaller than 1 it means that the gradient will be earlier clipped when learning rate is too large (ie. exceeding the maximal pulse number times the weight granularity). If 1, both d and x inputs are clipped for the same learning rate.
- update_bl_management: bool = True¶
Whether to enable dynamical adjustment of
A
,``B``,andBL
:BL = ceil(learning_rate * abs(x_j) * abs(d_i) / weight_granularity); BL = min(BL,desired_BL); A = B = sqrt(learning_rate / (weight_granularity * BL));
The
weight_granularity
is usually equal todw_min
.
- update_management: bool = True¶
Whether to apply additional scaling.
After the above setting an additional scaling (always on when using update_bl_management`) is applied to account for the different input strengths. If
\[\gamma \equiv \max_i |x_i| / (\alpha \max_j |d_j|)\]is the ratio between the two maximal inputs, then
A
is additionally scaled by \(\gamma\) andB
is scaled by \(1/\gamma\).The gradient scale \(\alpha\) can be set with
um_grad_scale
- x_res_implicit: float = 0¶
Resolution of each quantization step for the inputs
x
.Resolution (ie. bin width) of each quantization step for the inputs
x
in case ofDeterministicImplicit
pulse trains. SeePulseTypeMap
for details.
- class aihwkit.simulator.configs.utils.WeightClipParameter(fixed_value=- 1.0, sigma=2.5, type=WeightClipType.NONE)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter that clip the weights during hardware-aware training.
Important
A clipping
type
has to be set before any of the parameter changes take any effect.- Parameters
fixed_value (float) –
sigma (float) –
type (aihwkit.simulator.configs.enums.WeightClipType) –
- Return type
None
- fixed_value: float = -1.0¶
Clipping value in case of
FixedValue
type.Caution
If
fixed_value > 0
it will be also applied during other clipping types.
- sigma: float = 2.5¶
Sigma value for clipping for the
LayerGaussian
type.
- type: aihwkit.simulator.configs.enums.WeightClipType = 'None'¶
Type of clipping.
- class aihwkit.simulator.configs.utils.WeightModifierParameter(std_dev=0.0, res=0.0, sto_round=False, dorefa_clip=0.6, pdrop=0.0, enable_during_test=False, rel_to_actual_wmax=True, assumed_wmax=1.0, copy_last_column=False, coeffs=<factory>, type=WeightModifierType.COPY)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter that modify the forward/backward weights during hardware-aware training.
- Parameters
std_dev (float) –
res (float) –
sto_round (bool) –
dorefa_clip (float) –
pdrop (float) –
enable_during_test (bool) –
rel_to_actual_wmax (bool) –
assumed_wmax (float) –
copy_last_column (bool) –
coeffs (List[float]) –
type (aihwkit.simulator.configs.enums.WeightModifierType) –
- Return type
None
- assumed_wmax: float = 1.0¶
Assumed weight value that is mapped to the maximal conductance.
This is typically 1.0. This parameter will be ignored if
rel_to_actual_wmax
is set.
- coeffs: List[float]¶
Coefficients for the
POLY
weight modifier type.See
WeightModifierType
for details.
- copy_last_column: bool = False¶
Whether to not apply noise to the last column (which usually contains the bias values).
- dorefa_clip: float = 0.6¶
Parameter for DoReFa.
- enable_during_test: bool = False¶
Whether to use the last modified weight matrix during testing.
Caution
This will not remove drop connect or any other noise during evaluation, and thus should only used with care.
- pdrop: float = 0.0¶
Drop connect probability.
Drop connect sets weights to zero with the given probability. This implements drop connect.
Important
Drop connect can be used with any other modifier type in combination.
- rel_to_actual_wmax: bool = True¶
Whether to calculate the abs max of the weight and apply noise relative to this number.
If set to False,
assumed_wmax
is taken as relative units.
- res: float = 0.0¶
Resolution of the discretization.
The invert of
res
gives the number of equal sized steps in \(-a_\text{max}\ldots,a_\text{max}\) where the \(a_\text{max}\) is either given by the abs max (ifrel_to_actual_wmax
is set) orassumed_wmax
otherwise.res
is only used in the modifier typesDoReFa
,Discretize
, andDiscretizeAddNormal
.
- std_dev: float = 0.0¶
Standard deviation of the added noise to the weight matrix.
This parameter affects the modifier types
AddNormal
,MultNormal
andDiscretizeAddNormal
.Note
If the parameter
rel_to_actual_wmax
is set then thestd_dev
is computed in relative terms to the abs max of the given weight matrix, otherwise it in relative terms to the assumed max, which is set byassumed_wmax
.
- sto_round: bool = False¶
Whether the discretization is done with stochastic rounding enabled.
sto_round
is only used in the modifier typesDoReFa
,Discretize
, andDiscretizeAddNormal
.
- type: aihwkit.simulator.configs.enums.WeightModifierType = 'Copy'¶
Type of the weight modification.
- class aihwkit.simulator.configs.utils.WeightRemapParameter(remapped_wmax=1.0, max_scale_range=0.0, max_scale_ref=0.0, type=WeightRemapType.NONE)[source]¶
Bases:
aihwkit.simulator.configs.helpers._PrintableMixin
Parameter that remap the weights during hardware-aware training.
Important
A remap
type
has to be set before any of the parameter changes take any effect.- Parameters
remapped_wmax (float) –
max_scale_range (float) –
max_scale_ref (float) –
type (aihwkit.simulator.configs.enums.WeightRemapType) –
- Return type
None
- max_scale_range: float = 0.0¶
Maximal range of scale values. Use zero to turn any restrictions off (default).
- max_scale_ref: float = 0.0¶
Reference scale that use used as minimal scale for determining the scale range.
- remapped_wmax: float = 1.0¶
Assumed max of weight, ie the value of the weight the maximal conductance is mapped to. Typically 1.0.
- type: aihwkit.simulator.configs.enums.WeightRemapType = 'None'¶
Type of clipping.