aihwkit.simulator.configs.utils module¶
Utility parameters for resistive processing units.
-
class
aihwkit.simulator.configs.utils.
BackwardIOParameters
(bm_test_negative_bound=True, bound_management=<BoundManagementType.NONE: 'None'>, inp_bound=1.0, inp_noise=0.0, inp_res=0.007936507936507936, inp_sto_round=False, is_perfect=False, max_bm_factor=1000, max_bm_res=0.25, nm_thres=0.0, noise_management=<NoiseManagementType.ABS_MAX: 'AbsMax'>, out_bound=12.0, out_noise=0.06, out_res=0.00196078431372549, out_scale=1.0, out_sto_round=False, w_noise=0.0, w_noise_type=<OutputWeightNoiseType.NONE: 'None'>)¶ Bases:
aihwkit.simulator.configs.utils.IOParameters
Parameters that modify the backward IO behavior.
This class contains the same parameters as
AnalogTileInputOutputParameters
, specializing the default value ofbound_management
(as backward does not support bound management).-
bound_management
: aihwkit.simulator.configs.utils.BoundManagementType = 'None'¶ Type of noise management, see
NoiseManagementType
.
-
-
class
aihwkit.simulator.configs.utils.
BoundManagementType
(value)¶ Bases:
enum.Enum
Bound management type.
In the case
Iterative
the MAC is iteratively recomputed with inputs iteratively halved, when the output bound was hit.-
ITERATIVE
= 'Iterative'¶ Iteratively recomputes input scale set to \(\alpha\leftarrow\alpha/2\).
-
NONE
= 'None'¶ No bound management.
-
-
class
aihwkit.simulator.configs.utils.
IOParameters
(bm_test_negative_bound=True, bound_management=<BoundManagementType.ITERATIVE: 'Iterative'>, inp_bound=1.0, inp_noise=0.0, inp_res=0.007936507936507936, inp_sto_round=False, is_perfect=False, max_bm_factor=1000, max_bm_res=0.25, nm_thres=0.0, noise_management=<NoiseManagementType.ABS_MAX: 'AbsMax'>, out_bound=12.0, out_noise=0.06, out_res=0.00196078431372549, out_scale=1.0, out_sto_round=False, w_noise=0.0, w_noise_type=<OutputWeightNoiseType.NONE: 'None'>)¶ Bases:
object
Parameters that modify the IO behavior.
-
bm_test_negative_bound
: bool = True¶
-
bound_management
: aihwkit.simulator.configs.utils.BoundManagementType = 'Iterative'¶ Type of bound management, see
BoundManagementType
.
-
inp_bound
: float = 1.0¶ Input bound and ranges for the digital-to-analog converter (DAC).
-
inp_noise
: float = 0.0¶ Std deviation of Gaussian input noise (\(\sigma_\text{inp}\)), i.e. noisiness of the analog input (at the stage after DAC and before the multiplication).
-
inp_res
: float = 0.007936507936507936¶ Number of discretization steps for DAC (\(\le0\) means infinite steps) or resolution (1/steps).
-
inp_sto_round
: bool = False¶ Whether to enable stochastic rounding of DAC.
-
is_perfect
: bool = False¶ Short-cut to compute a perfect forward pass. If
True
, it assumes an ideal forward pass (e.g. no bound, ADC etc…). Will disregard all other settings in this case.
-
max_bm_factor
: int = 1000¶ Maximal bound management factor. If this factor is reached then the iterative process is stopped.
-
max_bm_res
: float = 0.25¶ Another way to limit the maximal number of iterations of the bound management. The max effective resolution number of the inputs, e.g. use \(1/4\) for 2 bits.
-
nm_thres
: float = 0.0¶ Constant noise management value for
type
Constant
.In other cases, this is a upper threshold \(\theta\) above which the noise management factor is saturated. E.g. for AbsMax:
\begin{equation*} \alpha=\begin{cases}\max_i|x_i|, & \text{if} \max_i|x_i|<\theta \\ \theta, & \text{otherwise}\end{cases} \end{equation*}Caution
If
nm_thres
is set (and type is notConstant
), the noise management will clip some large input values, in favor of having a better SNR for smaller input values.
-
noise_management
: aihwkit.simulator.configs.utils.NoiseManagementType = 'AbsMax'¶ Type of noise management, see
NoiseManagementType
.
-
out_bound
: float = 12.0¶ Output bound and ranges for analog-to-digital converter (ADC).
-
out_noise
: float = 0.06¶ Std deviation of Gaussian output noise (\(\sigma_\text{out}\)), i.e. noisiness of device summation at the output.
-
out_res
: float = 0.00196078431372549¶ Number of discretization steps for ADC (\(<=0\) means infinite steps) or resolution (1/steps).
-
out_scale
: float = 1.0¶ Additional fixed scalar factor.
-
out_sto_round
: bool = False¶ Whether to enable stochastic rounding of ADC.
-
w_noise
: float = 0.0¶ Scale of output referred weight noise (\(\sigma_w\)) for a given
w_noise_type
.
-
w_noise_type
: aihwkit.simulator.configs.utils.OutputWeightNoiseType = 'None'¶ Type as specified in
OutputWeightNoiseType
.Note
This noise us applied each time anew as it is referred to the output. It will not change the conductance values of the weight matrix. For the latter one can apply
diffuse_weights()
.
-
-
class
aihwkit.simulator.configs.utils.
NoiseManagementType
(value)¶ Bases:
enum.Enum
Noise management type.
Noise management determines a factor \(\alpha\) how the input is reduced:
\[\mathbf{y} = \alpha\;F_\text{analog-mac}\left(\mathbf{x}/\alpha\right)\]-
ABS_MAX
= 'AbsMax'¶ Use \(\alpha\equiv\max{|\mathbf{x}|}\).
-
CONSTANT
= 'Constant'¶ A constant value (given by parameter
nm_thres
).
-
MAX
= 'Max'¶ Use \(\alpha\equiv\max{\mathbf{x}}\).
-
NONE
= 'None'¶ No noise management.
-
-
class
aihwkit.simulator.configs.utils.
OutputWeightNoiseType
(value)¶ Bases:
enum.Enum
Output weight noise type.
The weight noise is applied for each MAC computation, while not touching the actual weight matrix but referring it to the output.
\[y_i = \sum_j w_{ij}+\xi_{ij}\]-
ADDITIVE_CONSTANT
= 'AdditiveConstant'¶ The \(\xi\sim{\cal N}(0,\sigma)\) thus all are Gaussian distributed. \(\sigma\) is determined by
w_noise
.
-
NONE
= 'None'¶ No weight noise.
-
-
class
aihwkit.simulator.configs.utils.
PulseType
(value)¶ Bases:
enum.Enum
Pulse type.
-
MEAN_COUNT
= 'MeanCount'¶ Coincidence based in prob (\(p_a p_b\)).
-
NONE
= 'None'¶ Floating point update instead of pulses.
-
NONE_WITH_DEVICE
= 'NoneWithDevice'¶ Floating point like
None
, but with analog devices (e.g. weight clipping).
-
STOCHASTIC
= 'Stochastic'¶ Two passes for plus and minus (only CPU).
-
STOCHASTIC_COMPRESSED
= 'StochasticCompressed'¶ Generates actual stochastic bit lines. Plus and minus pulses are taken in the same pass.
-
-
class
aihwkit.simulator.configs.utils.
UpdateParameters
(desired_bl=31, fixed_bl=True, pulse_type=<PulseType.STOCHASTIC_COMPRESSED: 'StochasticCompressed'>, res=0, sto_round=False, update_bl_management=True, update_management=True)¶ Bases:
object
Parameter that modify the update behaviour of a pulsed device.
-
desired_bl
: int = 31¶ Desired length of the pulse trains. For update BL management, it is the maximal pulse train length.
-
fixed_bl
: bool = True¶ Whether to fix the length of the pulse trains (however, see
update_bl_management
).In case of
True
(wheredw_min
is the mean minimal weight change step size) it is:BL = desired_BL A = B = sqrt(learning_rate / (dw_min * BL));
In case of
False
:if dw_min * desired_BL < learning_rate: A = B = 1; BL = ceil(learning_rate / dw_min; else: # same as for fixed_BL=True
-
pulse_type
: aihwkit.simulator.configs.utils.PulseType = 'StochasticCompressed'¶ Switching between different pulse types. See
PulseTypeMap
for details.Important
Pulsing can also be turned off in which case the update is done as if in floating point and all other update related parameter are ignored.
-
res
: float = 0¶ Number of discretization steps of the probability in
0..1
. Use -1 for turning discretization off. Can be \(1/n_ ext{steps}\) as well.
-
sto_round
: bool = False¶ Whether to enable stochastic rounding.
-
update_bl_management
: bool = True¶ Whether to enable dynamical adjustment of
A
,``B``,andBL
:BL = ceil(learning_rate * abs(x_j) * abs(d_i) / dw_min); BL = min(BL,desired_BL); A = B = sqrt(learning_rate / (dw_min * BL));
-
update_management
: bool = True¶ After the above setting an additional scaling (always on when using update_bl_management`) is applied to account for the different input strengths. If
\[\gamma \equiv \max_i |x_i| / \max_j |d_j|\]is the ratio between the two maximal inputs, then
A
is additionally scaled by \(\gamma\) andB
is scaled by \(1/\gamma\).
-
-
class
aihwkit.simulator.configs.utils.
WeightClipParameter
(fixed_value=1.0, sigma=2.5, type=<WeightClipType.NONE: 'None'>)¶ Bases:
object
Parameter that clip the weights during hardware-aware training.
-
fixed_value
: float = 1.0¶ Clipping value in case of
FixedValue
type.
-
sigma
: float = 2.5¶ Sigma value for clipping for the
LayerGaussian
type.
-
type
: aihwkit.simulator.configs.utils.WeightClipType = 'None'¶ Type of clipping.
-
-
class
aihwkit.simulator.configs.utils.
WeightClipType
(value)¶ Bases:
enum.Enum
Weight clipper type.
-
AVERAGE_CHANNEL_MAX
= 'AverageChannelMax'¶ Calculates the abs max of each output channel (row of the weight matrix) and takes the average as clipping value for all.
-
FIXED_VALUE
= 'FixedValue'¶ Clip to fixed value give, symmetrical around zero.
-
LAYER_GAUSSIAN
= 'LayerGaussian'¶ Calculates the second moment of the whole weight matrix and clips at
sigma
times the result symmetrically around zero.
-
NONE
= 'None'¶ None.
-
-
class
aihwkit.simulator.configs.utils.
WeightModifierParameter
(std_dev=0.0, res=0.0, sto_round=False, dorefa_clip=0.6, pdrop=0.0, enable_during_test=False, rel_to_actual_wmax=True, assumed_wmax=1.0, type=<WeightModifierType.COPY: 'Copy'>)¶ Bases:
object
Parameter that modify the forward/backward weights during hardware-aware training.
-
assumed_wmax
: float = 1.0¶ Assumed weight value that is mapped to the maximal conductance.
This is typically 1.0. This parameter will be ignored if
rel_to_actual_wmax
is set.
-
dorefa_clip
: float = 0.6¶ Parameter for DoReFa.
-
enable_during_test
: bool = False¶ Whether to use the last modified weight matrix during testing.
Caution
This will NOT remove drop connect or any other noise during evaluation, and thus should only used with care.
-
pdrop
: float = 0.0¶ Drop connect probability.
Drop connect sets weights to zero with the given probability. This implements drop connect.
Important
Drop connect can be used with any other modifier type in combination.
-
rel_to_actual_wmax
: bool = True¶ Whether to calculate the abs max of the weight and apply noise relative to this number.
If set to False,
assumed_wmax
is taken as relative units.
-
res
: float = 0.0¶ Resolution of the discretization.
The invert of
res
gives the number of equal sized steps in \(-a_\text{max}\ldots,a_\text{max}\) where the \(a_\text{max}\) is either given by the abs max (ifrel_to_actual_wmax
is set) orassumed_wmax
otherwise.res
is only used in the modifier typesDoReFa
,Discretize
, andDiscretizeAddNormal
.
-
std_dev
: float = 0.0¶ Standard deviation of the added noise to the weight matrix.
This parameter affects the modifier types
AddNormal
,MultNormal
andDiscretizeAddNormal
.Note
If the parameter
rel_to_actual_wmax
is set then thestd_dev
is computed in relative terms to the abs max of the given weight matrix, otherwise it in relative terms to the assumed max, which is set byassumed_wmax
.
-
sto_round
: bool = False¶ Whether the discretization is done with stochastic rounding enabled.
sto_round
is only used in the modifier typesDoReFa
,Discretize
, andDiscretizeAddNormal
.
-
type
: aihwkit.simulator.configs.utils.WeightModifierType = 'Copy'¶ Type of the weight modification.
-
-
class
aihwkit.simulator.configs.utils.
WeightModifierType
(value)¶ Bases:
enum.Enum
Weight modifier type.
-
ADD_NORMAL
= 'AddNormal'¶ Additive Gaussian noise.
-
COPY
= 'Copy'¶ Just copy, however, could also drop.
-
DISCRETIZE
= 'Discretize'¶ Quantize the weights.
-
DISCRETIZE_ADD_NORMAL
= 'DiscretizeAddNormal'¶ First discretize and then additive Gaussian noise.
-
DOREFA
= 'DoReFa'¶ DoReFa discretization.
-
MULT_NORMAL
= 'MultNormal'¶ Mutiplicative Gaussian noise.
-
-
aihwkit.simulator.configs.utils.
parameters_to_bindings
(params)¶ Convert a dataclass parameter into a bindings class.
- Parameters
params (Any) –
- Return type
Any
-
aihwkit.simulator.configs.utils.
tile_parameters_to_bindings
(params)¶ Convert a tile dataclass parameter into a bindings class.
- Parameters
params (Any) –
- Return type
Any