aihwkit.simulator.parameters.pre_post module
Pre-post processing related parameters for resistive processing units.
- class aihwkit.simulator.parameters.pre_post.InputRangeParameter(enable=<factory>, learn_input_range=True, init_value=3.0, init_from_data=100, init_std_alpha=3.0, decay=0.001, input_min_percentage=0.95, manage_output_clipping=False, output_min_percentage=0.95, gradient_scale=1.0, gradient_relative=True, calibration_info=None)[source]
Bases:
_PrintableMixin
Parameter related to input range learning
- Parameters:
enable (bool) –
learn_input_range (bool) –
init_value (float) –
init_from_data (int) –
init_std_alpha (float) –
decay (float) –
input_min_percentage (float) –
manage_output_clipping (bool) –
output_min_percentage (float) –
gradient_scale (float) –
gradient_relative (bool) –
calibration_info (str | None) –
- decay: float = 0.001
Decay rate for input range learning.
- enable: bool
Whether to enable to learn the input range. Note that if enable is
False
then no clip is applied.Note
The input bound (
forward.inp_bound
) is assumed to be 1 if enabled as the input range already scales the input into to the range \((-1, 1)\) by dividing the input to the type by itself and multiplying the output accordingly.Typically, noise and bound management should be set to NONE for the input range learning as it replaces the dynamic managements with a static but learned input bound. However, in some exceptional experimental cases one might want to enable the management techniques on top of the input range learning, so that no error is raised if they are not set to NONE.
- gradient_relative: bool = True
Whether to make the gradient of the input range learning relative to the current range value.
- gradient_scale: float = 1.0
Scale of the gradient magnitude (learning rate) for the input range learning.
- init_from_data: int = 100
Number of batches to use for initialization from data. Set 0 to turn off.
- init_std_alpha: float = 3.0
Standard deviation multiplier for initialization from data.
- init_value: float = 3.0
Initial setting of the input range in case of input range learning.
- input_min_percentage: float = 0.95
Decay is only applied if percentage of non-clipped values is above this value.
Note
The added gradient is (in case of non-clipped input percentage
percentage > input_min_percentage
):grad += decay * input_range
- learn_input_range: bool = True
Whether to learn the input range when enabled.
Note
If not learned, the input range should in general be set with some calibration method before training the DNN.
- manage_output_clipping: bool = False
Whether to increase the input range when output clipping occurs.
Caution
The output bound is taken from the
forward.out_bound
value, which has to exist. Noise and bound management have to be set to NONE if this feature is enabled otherwise aConfigError
is raised.
- output_min_percentage: float = 0.95
Increase of the input range is only applied if percentage of non-clipped output values is below this value.
Note
The gradient subtracted from the input range is (in case of
output_percentage < output_min_percentage
):grad -= (1.0 - output_percentage) * input_range
- class aihwkit.simulator.parameters.pre_post.PrePostProcessingParameter(input_range=<factory>)[source]
Bases:
_PrintableMixin
Parameter related to digital input and output processing, such as input clip learning.
- Parameters:
input_range (InputRangeParameter) –
- input_range: InputRangeParameter
- class aihwkit.simulator.parameters.pre_post.PrePostProcessingRPU(tile_class, runtime=<factory>, pre_post=<factory>)[source]
Bases:
RPUConfigBase
,_PrintableMixin
Defines the pre-post parameters and utility factories
- Parameters:
tile_class (Type) –
runtime (RuntimeParameter) –
pre_post (PrePostProcessingParameter) –
- pre_post: PrePostProcessingParameter
Parameter related digital pre and post processing.