.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_interfaces/model_definition.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_interfaces_model_definition.py: Network Initialization ====================== `RectiPy` models are generally initialized via the :code:`rectipy.Network` class. The method `Network.from_yaml` serves as the main user interface for network model initialization. It allows the user to define the dynamic equations and default parameters of the units of an RNN via `YAML` templates. To this end, `RectiPy` makes use of the model definition capacities of `PyRates `_. `PyRates` provides a detailed documentation that covers `the mathematical syntax `_ that can be used to define neuron models, and how to use their `YAML template classes `_ to define neurodynamic models. We refer readers to this documentation to learn how to define neuron model templates themselves. Here, we will make use of the pre-implemented neuron models that come with `RectiPy` to explain how to initialize a `rectipy.Network` based on a neuron model template and how the `rectipy.Network` can be customized afterwards to construct a neural network model of choice. How to create a network with differential-equation-based nodes -------------------------------------------------------------- Let's begin with the `rectipy.Network.add_diffeq_node` method. The code below initializes a `rectipy.Network` instance and adds :math:`N = 5` rate-coupled LI neurons to the network via this method. LI rate neurons are defined via a single ordinary differential equation. For an introduction to the LI rate neuron model, see `the respective use example `_. .. GENERATED FROM PYTHON SOURCE LINES 25-42 .. code-block:: Python from rectipy import Network import numpy as np # define network parameters node = "neuron_model_templates.rate_neurons.leaky_integrator.tanh" N = 5 J = np.random.randn(N, N)*2.0 dt = 1e-3 # initialize network net = Network(dt=dt, device="cpu") # add a rate-neuron population to the network net.add_diffeq_node("tanh", node, weights=J, source_var="tanh_op/r", target_var="li_op/r_in", input_var="li_op/I_ext", output_var="li_op/v") .. GENERATED FROM PYTHON SOURCE LINES 43-67 In the code above, the variable :code:`node` is a path pointing to a YAML template that defines the dynamic equations and variables of a single LI neuron. Since `RectiPy` comes with a module that contains all its pre-implemented models (called `neuron_model_templates`), we can use the :code:`.` syntax to specify that path. If you define your own neuron model, you can use standard Python syntax to specify the path to that model, i.e. :code:`node = "/."`. As a second important piece of our network definition, we need to provide an :math:`N x N` coupling matrix, that defines the structure of the recurrent coupling in a network of :math:`N` neurons. Above, we call this matrix :code:`J`. During the call to `rectipy.Network.add_diffeq_node`, a `rectipy.nodes.RateNet` instance will be created, which can be accessed via `net["tanh"]` and contains a network of :math:`N` recurrently coupled neurons, with the neuron type given by :code:`node` and the coupling weights given by :code:`J`. The variables :code:`source_var` and :code:`target_var` specify which variables of the neuron model to use to establish the recurrent coupling. In the above example, we specify to project the variable :code:`r` of each neuron, defined in operator template :code:`tanh_op`, to the variable :code:`r_in` of each neuron, defined in operator template :code:`li_op`. The coupling weights provided via the keyword argument :code:`weights` will be used to scale the :code:`source_var`, and multiple sources projecting to the same target will be summed up. Thus, the above code implements the following definition of the target variable :code:`r_in` of each neuron: :math:`r_i^{in} = \sum_{j=1}^N J_{ij} r_j`. Furthermore, we specified that the variable :code:`I_ext` defined in operator template :code:`li_op` should serve as an input variable for any extrinsic inputs that we would like to apply to the RNN. We could use this variable, for example, to provide input from another network layer to the RNN. Finally, we defined that the variable :code:`u` defined in operator template :code:`li_op` serves as the output of the RNN layer and can thus be used to connect it to subsequent network layers. Below, we show how to access a couple of relevant attributes of the network. For a full documentation of the different attributes of `rectipy.network.Network` and `rectipy.nodes.RateNet` instances, see our `API `_. .. GENERATED FROM PYTHON SOURCE LINES 67-75 .. code-block:: Python net.compile() print(f"Number of neurons in network: {net.n_out}") print(f"Network nodes: {net.nodes}") print(f"State variables of the network: {net.state}") print(f"LI population parameters: {net['tanh']['node'].parameter_names}") .. GENERATED FROM PYTHON SOURCE LINES 76-81 How to create a spiking neural network -------------------------------------- in the above example, we initialized a network of rate neurons. To initialize a network of spiking neurons instead, we need to provide two additional keyword arguments to `Network.add_diffeq_node`: .. GENERATED FROM PYTHON SOURCE LINES 81-93 .. code-block:: Python # change neuron model to leaky integrate-and-fire neuron node = "neuron_model_templates.spiking_neurons.lif.lif" # initialize network net = Network(dt=dt, device="cpu") # initialize network net.add_diffeq_node("lif", node, weights=J, source_var="s", target_var="s_in", input_var="I_ext", output_var="s", op="lif_op", spike_var="spike", reset_var="v", spike_threshold=100.0, spike_reset=-100.0) .. GENERATED FROM PYTHON SOURCE LINES 94-118 First, note that we did not provide the operator template names with the variable names in this call to `Network.add_diffeq_node`. Instead, we provided the operator name as a separate keyword argument :code:`op`. This is possible in this case, since the LIF neuron template only consists of a single operator template. The keyword argument :code:`spike_var` specifies which variable to use to store spikes in, whereas the keyword argument :code:`spike_def` specifies which state variable to use to define the spiking condition of a neuron. When these two keyword arguments are provided, `rectipy.Network` is initialized with a `rectipy.nodes.SpikeNet` instance instead of a `rectipy.nodes.RateNet` instance. The class `rectipy.nodes.SpikeNet` implements the following spike condition: .. code-block:: if spike_def > spike_threshold: spike_var = 1.0 spike_def = spike_reset else: spike_var = 0.0 In more mathematic terms, and specific to our example model, this means that a spike is counted when :math:`v_i \geq \theta`, where :math:`\theta` is the spiking threshold. A spike is then counted via the variable :code:`spike_var` and the neuron's state variable is reset: :math:`v_i \rightarrow v_r`. The two control parameters of this spike reset conditions, :math:`\theta` and :math:`v_r`, can be controlled via two additional keyword arguments: :code:`spike_threshold` and :code:`spike_reset`. The default parameters of these two keyword arguments are as specified in the above call to `Network.add_diffeq_node`. Let's inspect the differences between the spiking network RNN layer and the rate-neuron RNN layer. .. GENERATED FROM PYTHON SOURCE LINES 118-126 .. code-block:: Python net.compile() print(f"Number of neurons in network: {net.n_out}") print(f"Network nodes: {net.nodes}") print(f"State variables of the network: {net.state}") print(f"LI population parameters: {net['lif']['node'].parameter_names}") .. GENERATED FROM PYTHON SOURCE LINES 127-132 As can be seen, a `SpikeNet` instance is now the only network node instead of a `RateNet` instance. Furthermore, the network node has a different state vector and a different number of parameters, owing to the differences in the LIF neuron as compared to the LI rate neuron. The state variable difference comes from the additional state-variable :math:`s_i`, whereas the difference in the number of model parameters is caused by the synaptic time constant :math:`\tau_s`. .. GENERATED FROM PYTHON SOURCE LINES 134-139 How to add a simple function node to the network ------------------------------------------------ In addition to nodes that are based on differential equation systems, it is also possible to add nodes with instantaneous activation functions. The code below shows how to add a node with the identity function (can be useful as input layers): .. GENERATED FROM PYTHON SOURCE LINES 139-143 .. code-block:: Python m = 3 net.add_func_node("input", m, activation_function="identity") .. GENERATED FROM PYTHON SOURCE LINES 144-145 To connect this node to an existing node, we can use the `Network.add_edge` method: .. GENERATED FROM PYTHON SOURCE LINES 145-148 .. code-block:: Python net.add_edge(source="input", target="lif") .. GENERATED FROM PYTHON SOURCE LINES 149-152 In this most simple case, a random set of weights will be drawn to map from the :math:`m = 3` input nodes to the :math:`N = 5` LIF neurons. You can also specify weights yourself and pass them via the keyword argument :code:`weights`. Let's add another function node and specify the weights explicitly: .. GENERATED FROM PYTHON SOURCE LINES 152-159 .. code-block:: Python # add a node of sigmoid units k = 2 net.add_func_node("output", k, activation_function="sigmoid") # add an edge from the LIF population to the sigmoid node net.add_edge(source="lif", target="output", weights=np.random.randn(N, k)) .. _sphx_glr_download_auto_interfaces_model_definition.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: model_definition.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: model_definition.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: model_definition.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_