eprop_readout – Current-based leaky integrate readout neuron model with delta-shaped postsynaptic currents for e-prop plasticity ================================================================================================================================ Description +++++++++++ ``eprop_readout`` is an implementation of an integrate-and-fire neuron model with delta-shaped postsynaptic currents used as readout neuron for eligibility propagation (e-prop) plasticity. E-prop plasticity was originally introduced and implemented in TensorFlow in [1]_. The membrane voltage time course :math:`v_j^t` of the neuron :math:`j` is given by: .. math:: v_j^t &= \kappa v_j^{t-1}+ \zeta \sum_{i \neq j} W_{ji}^\text{out} z_i^{t-1} \,, \\ \kappa &= e^{ -\frac{ \Delta t }{ \tau_\text{m} } } \,, \\ \zeta &= \begin{cases} 1 \\ 1 - \kappa \end{cases} \,, \\ where :math:`W_{ji}^\text{out}` is the output synaptic weight matrix and :math:`z_i^{t-1}` is the recurrent presynaptic spike state variable. Descriptions of further parameters and variables can be found in the table below. The spike state variable of a presynaptic neuron is expressed by a Heaviside function: .. math:: z_i^t = H \left( v_i^t - v_\text{th} \right) \,. \\ An additional state variable and the corresponding differential equation represents a piecewise constant external current. See the documentation on the :doc:`iaf_psc_delta<../models/iaf_psc_delta/>` neuron model for more information on the integration of the subthreshold dynamics. The change of the synaptic weight is calculated from the gradient :math:`g^t` of the loss :math:`E^t` with respect to the synaptic weight :math:`W_{ji}`: :math:`\frac{ \text{d} E^t }{ \text{d} W_{ij} }` which depends on the presynaptic spikes :math:`z_i^{t-1}` and the learning signal :math:`L_j^t` emitted by the readout neurons. In the interval between two presynaptic spikes, the gradient is calculated at each time step until the cutoff time point. This computation occurs over the time range: :math:`t \in \left[ t_\text{spk,prev}, \min \left( t_\text{spk,prev} + \Delta t_\text{c}, t_\text{spk,curr} \right) \right]`. Here, :math:`t_\text{spk,prev}` represents the time of the previous spike that passed the synapse, while :math:`t_\text{spk,curr}` is the time of the current spike, which triggers the application of the learning rule and the subsequent synaptic weight update. The cutoff :math:`\Delta t_\text{c}` defines the maximum allowable interval for integration between spikes. The expression for the gradient is given by: .. math:: \frac{ \text{d} E^t }{ \text{d} W_{ji} } = L_j^t \bar{z}_i^{t-1} \,. \\ The presynaptic spike trains are low-pass filtered with the following exponential kernel: .. math:: \bar{z}_i^t = \mathcal{F}_\kappa \left( z_{i}^t \right) = \kappa \bar{z}_i^{t-1} + \zeta z_i^t \,. \\ Since readout neurons are leaky integrators without a spiking mechanism, the formula for computing the gradient lacks the surrogate gradient / pseudo-derivative and a firing regularization term. As a last step for every round in the loop over the time steps :math:`t`, the new weight is retrieved by feeding the current gradient :math:`g^t` to the optimizer (see :doc:`weight_optimizer<../models/weight_optimizer/>` for more information on the available optimizers): .. math:: w^t = \text{optimizer} \left( t, g^t, w^{t-1} \right) \,. \\ After the loop has terminated, the filtered dynamic variables of e-prop are propagated from the end of the cutoff until the next spike: .. math:: p &= \text{max} \left( 0, t_\text{s}^{t} - \left( t_\text{s}^{t-1} + {\Delta t}_\text{c} \right) \right) \,, \\ \bar{z}_i^{t+p} &= \bar{z}_i^t \alpha^p \,. \\ The learning signal :math:`L_j^t` is given by the non-plastic feedback weight matrix :math:`B_{jk}` and the continuous error signal :math:`e_k^t` emitted by readout neuron :math:`k` and :math:`e_k^t` defined via a mean-squared error loss: .. math:: L_j^t = B_{jk} e_k^t = B_{jk} \left( y_k^t - y_k^{*,t} \right) \,. \\ where the readout signal :math:`y_k^t` corresponds to the membrane voltage of readout neuron :math:`k` and :math:`y_k^{*,t}` is the real-valued target signal. Furthermore, the readout and target signal are multiplied by a learning window signal, which has a value of 1.0 within the learning window and 0.0 outside. For more information on e-prop plasticity, see the documentation on the other e-prop models: * :doc:`eprop_iaf<../models/eprop_iaf/>` * :doc:`eprop_iaf_adapt<../models/eprop_iaf_adapt/>` * :doc:`eprop_synapse<../models/eprop_synapse/>` * :doc:`eprop_learning_signal_connection<../models/eprop_learning_signal_connection/>` Details on the event-based NEST implementation of e-prop can be found in [2]_. Parameters ++++++++++ The following parameters can be set in the status dictionary. ========================= ======= ===================== ================== ===================================== **Neuron parameters** ---------------------------------------------------------------------------------------------------------------- Parameter Unit Math equivalent Default Description ========================= ======= ===================== ================== ===================================== ``C_m`` pF :math:`C_\text{m}` 250.0 Capacitance of the membrane ``E_L`` mV :math:`E_\text{L}` 0.0 Leak / resting membrane potential ``I_e`` pA :math:`I_\text{e}` 0.0 Constant external input current ``tau_m`` ms :math:`\tau_\text{m}` 10.0 Time constant of the membrane ``V_min`` mV :math:`v_\text{min}` negative maximum Absolute lower bound of the membrane value voltage representable by a ``double`` type in C++ ========================= ======= ===================== ================== ===================================== =========================== ======= =========================== ================ =============================== **E-prop parameters** ---------------------------------------------------------------------------------------------------------------- Parameter Unit Math equivalent Default Description =========================== ======= =========================== ================ =============================== ``eprop_isi_trace_cutoff`` ms :math:`{\Delta t}_\text{c}` maximum value Cutoff for integration of representable e-prop update between two by a ``long`` spikes type in C++ =========================== ======= =========================== ================ =============================== Recordables +++++++++++ The following state variables evolve during simulation and can be recorded. =============== ==== =============== ============= ================ **Neuron state variables and recordables** ------------------------------------------------------------------- State variable Unit Math equivalent Initial value Description =============== ==== =============== ============= ================ ``V_m`` mV :math:`v_j` 0.0 Membrane voltage =============== ==== =============== ============= ================ ========================= ==== =============== ============= ============== **E-prop state variables and recordables** --------------------------------------------------------------------------- State variable Unit Math equivalent Initial value Description ========================= ==== =============== ============= ============== ``error_signal`` mV :math:`L_j` 0.0 Error signal ``readout_signal`` mV :math:`y_j` 0.0 Readout signal ``target_signal`` mV :math:`y^*_j` 0.0 Target signal ========================= ==== =============== ============= ============== Usage +++++ This model can only be used in combination with the other e-prop models and the network architecture requires specific wiring, input, and output. The usage is demonstrated in several :doc:`supervised regression and classification tasks <../auto_examples/eprop_plasticity/index>` reproducing among others the original proof-of-concept tasks in [1]_. References ++++++++++ .. [1] Bellec G, Scherr F, Subramoney F, Hajek E, Salaj D, Legenstein R, Maass W (2020). A solution to the learning dilemma for recurrent networks of spiking neurons. Nature Communications, 11:3625. https://doi.org/10.1038/s41467-020-17236-y .. [2] Korcsak-Gorzo A, Stapmanns J, Espinoza Valverde JA, Plesser HE, Dahmen D, Bolten M, Van Albada SJ, Diesmann M. Event-based implementation of eligibility propagation (in preparation) Sends +++++ LearningSignalConnectionEvent, DelayedRateConnectionEvent Receives ++++++++ SpikeEvent, CurrentEvent, DelayedRateConnectionEvent, DataLoggingRequest See also ++++++++ Examples using this model +++++++++++++++++++++++++ .. listexamples:: eprop_readout