aihwkit.simulator.presets.devices module

Device configurations presets for resistive processing units.

class aihwkit.simulator.presets.devices.CapacitorPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.005, dw_min_dtod=0.1, dw_min_std=0.3, enforce_consistency=True, lifetime=1000000.0, lifetime_dtod=0.3, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.06, w_max=1.0, w_max_dtod=0.07, w_min=-1.0, w_min_dtod=0.07, gamma_up=0.05, gamma_down=0.05, gamma_up_dtod=0.01, gamma_down_dtod=0.01, allow_increasing=False, mean_bound_reference=True, mult_noise=False, write_noise_std=0.0, reverse_up=False, reverse_down=False, reverse_offset=0.01)[source]

Bases: aihwkit.simulator.configs.devices.LinearStepDevice

Preset configuration for a single capacitor resistive processing unit based on linear step device.

Fit of the model LinearStepDevice to Li & al., VLSI, 2018

Here some capacitor leakage is assumed as well.

Caution

Capacitor leakage is applied only once per mini-batch and this the size of the leakage has to be adapted by the user as it depends not only on the size of the leak of the physical capacitor but also on the assumptions how much physical time is required for a full forward and backward cycle through the network (which depends on whether one assumes pipelining or not).

The parameter lifetime needs to be adjusted accordingly.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • gamma_up (float) –

  • gamma_down (float) –

  • gamma_up_dtod (float) –

  • gamma_down_dtod (float) –

  • allow_increasing (bool) –

  • mean_bound_reference (bool) –

  • mult_noise (bool) –

  • write_noise_std (float) –

  • reverse_up (bool) –

  • reverse_down (bool) –

  • reverse_offset (float) –

Return type

None

allow_increasing: bool = False

Whether to allow increasing of update sizes.

Whether to allow the situation where update sizes increase towards the bound instead of saturating (and thus becoming smaller).

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.005

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.1

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 0.3

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

gamma_down: float = 0.05

The value of \(\gamma^-\).

gamma_down_dtod: float = 0.01

Device-to-device variation for \(\gamma^-\), i.e. the value of \(\gamma_\text{d-to-d}^-\).

gamma_up: float = 0.05

The value of \(\gamma^+\).

Intuitively, a value of 0.1 means that the update step size in up direction at the weight bounds is 10% decreased relative to that origin \(w=0\).

Note

In principle one could fix \(\gamma=\gamma^-=\gamma^+\) since up/down variation can be given by up_down_dtod, see PulsedDevice.

Note

The hard-bounds are still observed, so that the weight cannot grow beyond its bounds.

gamma_up_dtod: float = 0.01

Device-to-device variation for \(\gamma^+\), i.e. the value of \(\gamma_\text{d-to-d}^+\).

lifetime: float = 1000000.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.3

Device-to-device variation in the decay rate (in relative units).

mean_bound_reference: bool = True

Whether to use instead of the above:

\[ \begin{align}\begin{aligned}\gamma_{ij}^+ &=& - |\gamma^+ + \gamma_\text{d-to-d}^+ \xi|/b^\text{max}\\\gamma_{ij}^- &=& - |\gamma^- + \gamma_\text{d-to-d}^- \xi|/b^\text{min}\end{aligned}\end{align} \]

where \(b^\text{max}\) and \(b^\text{max}\) are the values given by w_max and w_min, see PulsedDevice.

mult_noise: bool = False

Whether to use multiplicative noise instead of additive cycle-to-cycle noise.

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

reverse_down: bool = False

Whether to increase the step size in down direction with decreasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

reverse_offset: float = 0.01

Offset to add to the step size for reverse up or down to avoid zero step size at weight min or max.

reverse_up: bool = False

Whether to increase the step size in up direction with increasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.06

See up_down.

w_max: float = 1.0

See w_min.

w_max_dtod: float = 0.07

See w_min_dtod.

w_min: float = -1.0

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.07

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 0.0

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled anew when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise} \Delta w_\text{min}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).

class aihwkit.simulator.presets.devices.EcRamMOPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.00028214, dw_min_dtod=0.1, dw_min_std=2.0, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.01, w_max=1.1714, w_max_dtod=0.05, w_min=-0.8286, w_min_dtod=0.05, gamma_up=0.4152, gamma_down=0.7342, gamma_up_dtod=0.05, gamma_down_dtod=0.05, allow_increasing=False, mean_bound_reference=True, mult_noise=True, write_noise_std=0.0, reverse_up=False, reverse_down=False, reverse_offset=0.01)[source]

Bases: aihwkit.simulator.configs.devices.LinearStepDevice

Preset configuration for a single metal-oxide ECRAM resistive processing unit based on linear step device.

Based on data from Kim et al. IEDM, 2019

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • gamma_up (float) –

  • gamma_down (float) –

  • gamma_up_dtod (float) –

  • gamma_down_dtod (float) –

  • allow_increasing (bool) –

  • mean_bound_reference (bool) –

  • mult_noise (bool) –

  • write_noise_std (float) –

  • reverse_up (bool) –

  • reverse_down (bool) –

  • reverse_offset (float) –

Return type

None

allow_increasing: bool = False

Whether to allow increasing of update sizes.

Whether to allow the situation where update sizes increase towards the bound instead of saturating (and thus becoming smaller).

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.00028214

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.1

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 2.0

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

gamma_down: float = 0.7342

The value of \(\gamma^-\).

gamma_down_dtod: float = 0.05

Device-to-device variation for \(\gamma^-\), i.e. the value of \(\gamma_\text{d-to-d}^-\).

gamma_up: float = 0.4152

The value of \(\gamma^+\).

Intuitively, a value of 0.1 means that the update step size in up direction at the weight bounds is 10% decreased relative to that origin \(w=0\).

Note

In principle one could fix \(\gamma=\gamma^-=\gamma^+\) since up/down variation can be given by up_down_dtod, see PulsedDevice.

Note

The hard-bounds are still observed, so that the weight cannot grow beyond its bounds.

gamma_up_dtod: float = 0.05

Device-to-device variation for \(\gamma^+\), i.e. the value of \(\gamma_\text{d-to-d}^+\).

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

mean_bound_reference: bool = True

Whether to use instead of the above:

\[ \begin{align}\begin{aligned}\gamma_{ij}^+ &=& - |\gamma^+ + \gamma_\text{d-to-d}^+ \xi|/b^\text{max}\\\gamma_{ij}^- &=& - |\gamma^- + \gamma_\text{d-to-d}^- \xi|/b^\text{min}\end{aligned}\end{align} \]

where \(b^\text{max}\) and \(b^\text{max}\) are the values given by w_max and w_min, see PulsedDevice.

mult_noise: bool = True

Whether to use multiplicative noise instead of additive cycle-to-cycle noise.

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

reverse_down: bool = False

Whether to increase the step size in down direction with decreasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

reverse_offset: float = 0.01

Offset to add to the step size for reverse up or down to avoid zero step size at weight min or max.

reverse_up: bool = False

Whether to increase the step size in up direction with increasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.01

See up_down.

w_max: float = 1.1714

See w_min.

w_max_dtod: float = 0.05

See w_min_dtod.

w_min: float = -0.8286

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.05

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 0.0

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled anew when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise} \Delta w_\text{min}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).

class aihwkit.simulator.presets.devices.EcRamPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.002, dw_min_dtod=0.1, dw_min_std=0.3, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.01, w_max=1.1724, w_max_dtod=0.05, w_min=-0.8276, w_min_dtod=0.05, gamma_up=0.1153, gamma_down=0.5085, gamma_up_dtod=0.05, gamma_down_dtod=0.05, allow_increasing=False, mean_bound_reference=True, mult_noise=True, write_noise_std=0.0, reverse_up=False, reverse_down=False, reverse_offset=0.01)[source]

Bases: aihwkit.simulator.configs.devices.LinearStepDevice

Preset configuration for a single Lithium-based ECRAM resistive processing unit based on linear step device.

Fit of the model LinearStepDevice to Tang & al., IEDM, 2018

The range is shifted, so that the symmetry point is at zero.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • gamma_up (float) –

  • gamma_down (float) –

  • gamma_up_dtod (float) –

  • gamma_down_dtod (float) –

  • allow_increasing (bool) –

  • mean_bound_reference (bool) –

  • mult_noise (bool) –

  • write_noise_std (float) –

  • reverse_up (bool) –

  • reverse_down (bool) –

  • reverse_offset (float) –

Return type

None

allow_increasing: bool = False

Whether to allow increasing of update sizes.

Whether to allow the situation where update sizes increase towards the bound instead of saturating (and thus becoming smaller).

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.002

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.1

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 0.3

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

gamma_down: float = 0.5085

The value of \(\gamma^-\).

gamma_down_dtod: float = 0.05

Device-to-device variation for \(\gamma^-\), i.e. the value of \(\gamma_\text{d-to-d}^-\).

gamma_up: float = 0.1153

The value of \(\gamma^+\).

Intuitively, a value of 0.1 means that the update step size in up direction at the weight bounds is 10% decreased relative to that origin \(w=0\).

Note

In principle one could fix \(\gamma=\gamma^-=\gamma^+\) since up/down variation can be given by up_down_dtod, see PulsedDevice.

Note

The hard-bounds are still observed, so that the weight cannot grow beyond its bounds.

gamma_up_dtod: float = 0.05

Device-to-device variation for \(\gamma^+\), i.e. the value of \(\gamma_\text{d-to-d}^+\).

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

mean_bound_reference: bool = True

Whether to use instead of the above:

\[ \begin{align}\begin{aligned}\gamma_{ij}^+ &=& - |\gamma^+ + \gamma_\text{d-to-d}^+ \xi|/b^\text{max}\\\gamma_{ij}^- &=& - |\gamma^- + \gamma_\text{d-to-d}^- \xi|/b^\text{min}\end{aligned}\end{align} \]

where \(b^\text{max}\) and \(b^\text{max}\) are the values given by w_max and w_min, see PulsedDevice.

mult_noise: bool = True

Whether to use multiplicative noise instead of additive cycle-to-cycle noise.

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

reverse_down: bool = False

Whether to increase the step size in down direction with decreasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

reverse_offset: float = 0.01

Offset to add to the step size for reverse up or down to avoid zero step size at weight min or max.

reverse_up: bool = False

Whether to increase the step size in up direction with increasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.01

See up_down.

w_max: float = 1.1724

See w_min.

w_max_dtod: float = 0.05

See w_min_dtod.

w_min: float = -0.8276

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.05

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 0.0

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled anew when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise} \Delta w_\text{min}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).

class aihwkit.simulator.presets.devices.GokmenVlasovPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.0016, dw_min_dtod=0.3, dw_min_std=0.3, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.01, w_max=1.0, w_max_dtod=0.3, w_min=-1.0, w_min_dtod=0.3)[source]

Bases: aihwkit.simulator.configs.devices.ConstantStepDevice

Preset configuration using ConstantStepDevice.

Definitions are (largely) from the specifications listed in Gokmen & Vlasov, Front. Neurosci. 2016 (which includes a number of device-to-device and cycle-to-cycle variations, see PulsedDevice).

Note, however, that we use some of the algorithmic optimization of the follow-up papers as well and here scale everything to the weight range -1..1.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

Return type

None

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.0016

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.3

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 0.3

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.01

See up_down.

w_max: float = 1.0

See w_min.

w_max_dtod: float = 0.3

See w_min_dtod.

w_min: float = -1.0

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.3

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

class aihwkit.simulator.presets.devices.IdealizedPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.0002, dw_min_dtod=0.3, dw_min_std=0.3, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.0, w_max=1.0, w_max_dtod=0.3, w_min=-1.0, w_min_dtod=0.3)[source]

Bases: aihwkit.simulator.configs.devices.ConstantStepDevice

Preset configuration using an idealized device using ConstantStepDevice.

(On average) perfectly symmetric device with 10000 steps.

Definitions are from the specifications listed in Gokmen & Vlasov, Front. Neurosci. 2016 (which includes a number of device-to-device and cycle-to-cycle variations, see PulsedDevice), however, setting the device-to-device asymmetry term to zero and increasing the number of states by roughly 8 (to 10000 states).

This is the same device used for Rasch, Gokmen & Haensch, IEEE Design & Test, 2019.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

Return type

None

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.0002

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.3

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 0.3

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.0

See up_down.

w_max: float = 1.0

See w_min.

w_max_dtod: float = 0.3

See w_min_dtod.

w_min: float = -1.0

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.3

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

class aihwkit.simulator.presets.devices.PCMPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.01, dw_min_dtod=0.2, dw_min_std=0.6, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.01, reset_dtod=0.02, reset_std=0.01, up_down=0.0, up_down_dtod=0.05, w_max=2.0, w_max_dtod=0.1, w_min=0.0, w_min_dtod=0.0, A_up=-27.235, A_down=-2.235, gamma_up=2.5, gamma_down=2.5, a=-1.0, b=0.0, dw_min_std_add=0.042, dw_min_std_slope=0.108, write_noise_std=0.0)[source]

Bases: aihwkit.simulator.configs.devices.ExpStepDevice

Preset configuration for a single Phase change memory (PCM) analog resistive processing unit based on exponential step device model.

A PCM device based on \(Ge_2Sb_2Te_5\) described in Nandakumar et al., Front. Neurosci. 2020 by using the exponential device model with complex cycle-to-cycle noise (see ExpStepDevice)

Note

This is an uni-directional device and thus can only be used as a plus-minus pair with refresh (see PCMPresetUnitCell which is using this device combined as a pair based on OneSidedUnitCell).

When the device is reset, device-to-device and cycle-to-cycle variation is assumed.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • A_up (float) –

  • A_down (float) –

  • gamma_up (float) –

  • gamma_down (float) –

  • a (float) –

  • b (float) –

  • dw_min_std_add (float) –

  • dw_min_std_slope (float) –

  • write_noise_std (float) –

Return type

None

A_down: float = -2.235

Factor A for the down direction.

A_up: float = -27.235

Factor A for the up direction.

a: float = -1.0

Global slope parameter.

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

b: float = 0.0

Global offset parameter.

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.01

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.2

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 0.6

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

dw_min_std_add: float = 0.042

additive cycle-to-cycle noise of the update size (in units of dw_min_std, see above).

dw_min_std_slope: float = 0.108

cycle-to-cycle noise of the update size (in units of dw_min_std, see above).

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

gamma_down: float = 2.5

Exponent for the down direction.

gamma_up: float = 2.5

Exponent for the up direction.

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.01

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.02

See reset.

reset_std: float = 0.01

See reset.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.05

See up_down.

w_max: float = 2.0

See w_min.

w_max_dtod: float = 0.1

See w_min_dtod.

w_min: float = 0.0

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.0

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 0.0

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled a new when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).

class aihwkit.simulator.presets.devices.PCMPresetUnitCell(unit_cell_devices=<factory>, refresh_every=1, units_in_mbatch=True, refresh_upper_thres=0.75, refresh_lower_thres=0.25, refresh_forward=<factory>, refresh_update=<factory>, copy_inverted=False)[source]

Bases: aihwkit.simulator.configs.devices.OneSidedUnitCell

A unit cell that is comprised of two uni-directional PCM devices of opposite sign (see PCMPresetDevice).

Check for refresh is performed after each mini-batch update. See OneSidedUnitCell for details on the refresh implementation.

Parameters
Return type

None

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.OneSidedResistiveDeviceParameter

copy_inverted: bool = False

Whether the use the “down” update behavior of the first device for the negative updates instead of the positive half of the second device.

refresh_every: int = 1

How often a refresh is performed (in units of the number of vector updates).

Note

If a refresh is done, full reads of both positive and negative devices are performed. Additionally, if single devices deemed to be refreshed, an (open-loop) re-write is done (once per column). Thus, refresh might have considerable runtime impacts.

refresh_forward: aihwkit.simulator.configs.utils.IOParameters

Input-output parameters that define the read during a refresh event.

AnalogTileInputOutputParameters that define the read (forward) of an refresh event. For instance the amount of noise or whether refresh is done using a ADC/DAC etc.

refresh_lower_thres: float = 0.25

Lower threshold for determining the refresh, see above.

refresh_update: aihwkit.simulator.configs.utils.UpdateParameters

Update parameters that define the type of update used for each refresh event.

Update parameters AnalogTileUpdateParameters that define the type of update used for each refresh event.

refresh_upper_thres: float = 0.75

Upper threshold for determining the refresh, see above.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

unit_cell_devices: List

Devices that compose this unit cell.

units_in_mbatch: bool = True

If set, the refresh_every counter is given in m_batch which is the re-use factor. Smaller numbers are not possible.

Caution

For CUDA devices, refresh is always done in m_batch (ie the number of re-use per layer for a mini-batch). Smaller numbers will have no effect.

class aihwkit.simulator.presets.devices.ReRamESPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.00135, dw_min_dtod=0.2, dw_min_std=5.0, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.259359, up_down_dtod=0.05, w_max=1.0, w_max_dtod=0.3, w_min=-1.0, w_min_dtod=0.3, A_up=-1.18445, A_down=-0.081404, gamma_up=5.0, gamma_down=5.0, a=-0.5, b=-0.5, dw_min_std_add=0.0, dw_min_std_slope=0.0, write_noise_std=75.0)[source]

Bases: aihwkit.simulator.configs.devices.ExpStepDevice

Preset configuration for a single RRAM analog resistive processing unit based on exp. step device.

Fit of the model ExpStepDevice to Gong & al., Nat. Commun., 2018.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • A_up (float) –

  • A_down (float) –

  • gamma_up (float) –

  • gamma_down (float) –

  • a (float) –

  • b (float) –

  • dw_min_std_add (float) –

  • dw_min_std_slope (float) –

  • write_noise_std (float) –

Return type

None

A_down: float = -0.081404

Factor A for the down direction.

A_up: float = -1.18445

Factor A for the up direction.

a: float = -0.5

Global slope parameter.

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

b: float = -0.5

Global offset parameter.

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.00135

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.2

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 5.0

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

dw_min_std_add: float = 0.0

additive cycle-to-cycle noise of the update size (in units of dw_min_std, see above).

dw_min_std_slope: float = 0.0

cycle-to-cycle noise of the update size (in units of dw_min_std, see above).

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

gamma_down: float = 5.0

Exponent for the down direction.

gamma_up: float = 5.0

Exponent for the up direction.

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

up_down: float = 0.259359

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.05

See up_down.

w_max: float = 1.0

See w_min.

w_max_dtod: float = 0.3

See w_min_dtod.

w_min: float = -1.0

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.3

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 75.0

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled a new when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).

class aihwkit.simulator.presets.devices.ReRamSBPresetDevice(construction_seed=0, corrupt_devices_prob=0.0, corrupt_devices_range=1000, diffusion=0.0, diffusion_dtod=0.0, drift=<factory>, dw_min=0.002, dw_min_dtod=0.3, dw_min_std=3.75, enforce_consistency=True, lifetime=0.0, lifetime_dtod=0.0, perfect_bias=False, reset=0.0, reset_dtod=0.0, reset_std=0.01, up_down=0.0, up_down_dtod=0.01, w_max=1.25, w_max_dtod=0.24, w_min=-0.75, w_min_dtod=0.39999999999999997, mult_noise=False, write_noise_std=56, reverse_up=False, reverse_down=False, reverse_offset=0.01)[source]

Bases: aihwkit.simulator.configs.devices.SoftBoundsDevice

Preset configuration for a single ReRAM analog resistive processing unit based on soft bounds device.

Loose fit of the model SoftBoundsDevice to Gong & al., Nat. Commun., 2018.

Note

Here it is assumed that the devices have been calibrated to the symmetry point by subtracting a reference device (which is in this case not explicitly modeled). For a more accurate fit see ReRamESPresetDevice.

Parameters

  • construction_seed (int) –

  • corrupt_devices_prob (float) –

  • corrupt_devices_range (int) –

  • diffusion (float) –

  • diffusion_dtod (float) –

  • drift (aihwkit.simulator.configs.utils.DriftParameter) –

  • dw_min (float) –

  • dw_min_dtod (float) –

  • dw_min_std (float) –

  • enforce_consistency (bool) –

  • lifetime (float) –

  • lifetime_dtod (float) –

  • perfect_bias (bool) –

  • reset (float) –

  • reset_dtod (float) –

  • reset_std (float) –

  • up_down (float) –

  • up_down_dtod (float) –

  • w_max (float) –

  • w_max_dtod (float) –

  • w_min (float) –

  • w_min_dtod (float) –

  • mult_noise (bool) –

  • write_noise_std (float) –

  • reverse_up (bool) –

  • reverse_down (bool) –

  • reverse_offset (float) –

Return type

None

as_bindings()

Return a representation of this instance as a simulator bindings object.

Return type

aihwkit.simulator.rpu_base.devices.PulsedResistiveDeviceParameter

construction_seed: int = 0

If not equal 0, will set a unique seed for hidden parameters during construction.

corrupt_devices_prob: float = 0.0

Probability for devices to be corrupt (weights fixed to random value with hard bounds, that is min and max bounds are set to equal).

corrupt_devices_range: int = 1000

Range around zero for establishing corrupt devices.

diffusion: float = 0.0

Standard deviation of diffusion process.

diffusion_dtod: float = 0.0

Device-to device variation of diffusion rate in relative units.

drift: DriftParameter

Parameter governing a power-law drift.

dw_min: float = 0.002

Mean of the minimal update step sizes across devices and directions.

dw_min_dtod: float = 0.3

Device-to-device std deviation of dw_min (in relative units to dw_min).

dw_min_std: float = 3.75

Cycle-to-cycle variation size of the update step (related to \(\sigma_\text{c-to-c}\) above) in relative units to dw_min.

Note

Many spread (device-to-device variation) parameters are given in relative units. For instance e.g. a setting of dw_min_std of 0.1 would mean 10% spread around the mean and thus a resulting standard deviation (\(\sigma_\text{c-to-c}\)) of dw_min * dw_min_std.

enforce_consistency: bool = True

Whether to enforce weight bounds consistency during initialization.

Whether to enforce that max weight bounds cannot be smaller than min weight bounds, and up direction step size is positive and down negative. Switches the opposite values if encountered during init.

lifetime: float = 0.0

One over decay_rate, ie \(1/r_\text{decay}\).

lifetime_dtod: float = 0.0

Device-to-device variation in the decay rate (in relative units).

mult_noise: bool = False

Whether to use multiplicative noise instead of additive cycle-to-cycle noise.

perfect_bias: bool = False

No up-down differences and device-to-device variability in the bounds for the devices in the bias row.

requires_decay()

Return whether device has decay enabled.

Return type

bool

requires_diffusion()

Return whether device has diffusion enabled.

Return type

bool

reset: float = 0.0

The reset values and spread per cross-point ij when using reset functionality of the device.

reset_dtod: float = 0.0

See reset.

reset_std: float = 0.01

See reset.

reverse_down: bool = False

Whether to increase the step size in down direction with decreasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

reverse_offset: float = 0.01

Offset to add to the step size for reverse up or down to avoid zero step size at weight min or max.

reverse_up: bool = False

Whether to increase the step size in up direction with increasing weights (default decreases).

Note

If set, mult_noise needs to be also set.

up_down: float = 0.0

Up and down direction step sizes can be systematically different and also vary across devices.

\(\Delta w_{ij}^d\) is set during RPU initialization (for each cross-point \(ij\)):

\[\Delta w_{ij}^d = d\; \Delta w_\text{min}\, \left( 1 + d \beta_{ij} + \sigma_\text{d-to-d}\xi\right)\]

where \(\xi\) is again a standard Gaussian. \(\beta_{ij}\) is the directional up versus down bias. At initialization up_down_dtod and up_down defines this bias term:

\[\beta_{ij} = \beta_\text{up-down} + \xi \sigma_\text{up-down-dtod}\]

where \(\xi\) is again a standard Gaussian number and \(\beta_\text{up-down}\) corresponds to up_down. Note that up_down_dtod is again given in relative units to dw_min.

up_down_dtod: float = 0.01

See up_down.

w_max: float = 1.25

See w_min.

w_max_dtod: float = 0.24

See w_min_dtod.

w_min: float = -0.75

Mean of hard bounds across device cross-point ij.

The parameters w_min and w_max are used to set the min/max bounds independently.

Note

For this abstract device, we assume that weights can have positive and negative values and are symmetrically around zero. In physical circuit terms, this might be implemented as a difference of two resistive elements.

w_min_dtod: float = 0.39999999999999997

Device-to-device variation of the hard bounds.

Device-to-device variation of the hard bounds, of min and max value, respectively. All are given in relative units to w_min, or w_max, respectively.

write_noise_std: float = 56

Whether to use update write noise.

Whether to use update write noise that is added to the updated devices weight, while the update is done on a hidden persistent weight. The update write noise is then sampled anew when the device is touched again.

Thus it is:

\[w_\text{apparent}{ij} = w_{ij} + \sigma_\text{write_noise} \Delta w_\text{min}\xi\]

and the update is done on \(w_{ij}\) but the forward sees the \(w_\text{apparent}\).