aihwkit.simulator.tiles.floating_point module¶
High level analog tiles (floating point).
- class aihwkit.simulator.tiles.floating_point.CudaFloatingPointTile(source_tile)[source]¶
Bases:
Generic[aihwkit.simulator.tiles.base.RPUConfigGeneric]Floating point tile (CUDA).
Floating point tile that uses GPU for its operation. The instantiation is based on an existing non-cuda tile: all the source attributes are copied except for the simulator tile, which is recreated using a GPU tile.
Caution
Deprecated. Use
FloatingPointTile(..).cuda()instead.- Parameters
source_tile – tile to be used as the source of this tile
- is_cuda = True¶
- class aihwkit.simulator.tiles.floating_point.FloatingPointTile(out_size, in_size, rpu_config=None, bias=False, in_trans=False, out_trans=False)[source]¶
Bases:
Generic[aihwkit.simulator.tiles.base.RPUConfigGeneric]Floating point tile.
Implements a floating point or ideal analog tile.
A linear layer with this tile is perfectly linear, it just uses the RPUCuda library for execution.
Forward pass:
\[\mathbf{y} = W\mathbf{x}\]\(W\) are the weights, \(\mathbf{x}\) is the input vector. \(\mathbf{y}\) is output of the vector matrix multiplication. Note that if bias is used, \(\mathbf{x}\) is concatenated with 1 so that the last column of \(W\) are the biases.
Backward pass:
Typical backward pass with transposed weights:
\[\mathbf{d'} = W^T\mathbf{d}\]where \(\mathbf{d}\) is the error vector. \(\mathbf{d}_o\) is output of the backward matrix vector multiplication.
Weight update:
Usual learning rule for back-propagation:
\[w_{ij} \leftarrow w_{ij} + \lambda d_i\,x_j\]Decay:
\[w_{ij} \leftarrow w_{ij}(1-\alpha r_\text{decay})\]Weight decay can be called by calling the analog tile decay.
Note
life_timeparameter is set during initialization. alpha is a scaling factor that can be given during run-time.Diffusion:
\[w_{ij} \leftarrow w_{ij} + \xi\;r_\text{diffusion}\]Similar to the decay, diffusion is only done when explicitly called. However, the parameter of the diffusion process are set during initialization and are fixed for the remainder. \(\xi\) is a standard Gaussian process.
- Parameters
out_size – output vector size of the tile, ie. the dimension of \(\mathbf{y}\) in case of \(\mathbf{y} = W\mathbf{x}\) (or equivalently the dimension of the \(\boldsymbol{\delta}\) of the backward pass).
in_size – input vector size, ie. the dimension of the vector \(\mathbf{x}\) in case of \(\mathbf{y} = W\mathbf{x}\)).
rpu_config – resistive processing unit configuration.
bias – whether to add a bias column to the tile, ie. \(W\) has an extra column to code the biases. Internally, the input \(\mathbf{x}\) will be automatically expanded by an extra dimension which will be set to 1 always.
in_trans – Whether to assume an transposed input (batch first).
out_trans – Whether to assume an transposed output (batch first).
- cpu()[source]¶
Return a copy of this tile in CPU memory.
Note
CUDA tiles weight can be accessed by get_weights etc methods, there is no need to move them to CPU and it is currently not supported.
- Returns
self in case of CPU
- Raises
CudaError – if a CUDA tile is moved to CPU
- Return type
aihwkit.simulator.tiles.base.BaseTile
- cuda(device=None)[source]¶
Return a copy of this tile in CUDA memory.
- Parameters
device (Optional[Union[torch.device, str, int]]) – CUDA device
- Returns
Self with the underlying C++ tile moved to CUDA memory.
- Raises
CudaError – if the library has not been compiled with CUDA.
- Return type
aihwkit.simulator.tiles.base.BaseTile