.. _embedding:

Embedding the interpreter
#########################

PyStim mainly focuses on extending SystemVerilog using Python, by embedding the
Python interpreter into a SystemVerilog program. This allows the SystemVerilog
program to execute Python code and interact with Python objects. All of the
other documentation pages still apply here, so refer to them for general PyStim usage.


Getting started
===============

A basic simulation with an embedded interpreter can be created with just a few
lines of bash code and the ``pystim_pkg`` package methods, as shown below. For more
information, see :doc:`/compiling`.

.. code-block:: bash

    source ../../scripts/environment.sh
    ./run.sh questa


The essential structure of the ``hello_world.sv`` file looks like this:

.. code-block:: systemverilog

    import pystim_pkg::*;

    module hello_world();
        typedef pystim py;

        initial begin
            py::initialize_interpreter();
            py::print(py::str_("Hello world."));
            py::finalize_interpreter();
        end
    endmodule


The interpreter must be initialized before using any Python, which includes
all the functions and classes in PyStim. The methods class ``initialize_interpreter()``, and
``finalize_interpreter()``  are used to manage interpreter lifetime. After the ``finalize_interpreter()`` call,
the Python interpreter shuts down and clears its memory. No Python functions can be called after this.


Importing modules
=================

Python modules can be imported using ``py_module::import_()``:

.. code-block:: systemverilog

    py_module sys = py_module::import_("sys");
    py::print(sys.attr("path").obtain());

For convenience, the current working directory is included in ``sys.path`` when
embedding the interpreter. This makes it easy to import local Python files:

.. code-block:: python

    """calc.py located in the working directory"""


    def add(i, j):
        return i + j


.. code-block:: systemverilog

        begin
            py_module calc = py_module::import_("add_numbers");
            py_object res = calc.attr("add").call(py::int_(1), py::float_(2.2));
            int n = res.cast_int().get_value();
            assert (n == 3.2) 
        end



Interpreter lifetime
====================

The Python interpreter starts up with ``initialize_interpreter`` and shuts down
when ``finalize_interpreter`` is called. HDL simulation permits only one pair of 
``initialize_interpreter`` and ``finalize_interpreter`` function calls. After this, 
creating a new instance of interpreter not possible. The ``initialize_interpreter``
/ ``finalize_interpreter`` methods can be used anywhere in the SystemVerilog code 
to set the state of the interpreter at any time.

.. warning::

    Starting two python interpreters is a fatal error. So is
    calling ``initialize_interpreter`` for a second time after the interpreter
    has already been initialized will case a fatal error. The same applies to
    calling ``finalize_interpreter`` for a second time after the interpreter has
    already been finalized.

    Do not use the raw CPython API functions ``initialize_interpreter`` and
    ``finalize_interpreter`` as these do not properly handle the lifetime of
    PyStim's internal data.



Restarting the interpter
========================

When using the HDL simulator GUI (intractive mode) for mixed-language (HDL and C/Python)
simulations, it's essential to understand how the simulation state for HDL and C/Python
code are managed during a simulation restart in the already running context.

In HDL simulators, restarting the simulation in the GUI primarily affects the HDL side
of the simulation. It resets the state of the HDL simulation, including all signal
values, registers, and memories to their initial state as defined at simulation zero time. 
However, the state of the C/Python code used in the simulation (for instance, through DPI,
VPI, or PLI) is not automatically reset by this action.

While the HDL simulation is restarted in already running context, PyStim library
does automatically restart the Python interpreter. However, this may not apply to
third-party extension Python modules.The issue is that Python itself cannot completely
unload extension modules and there are several caveats with regard to interpreter
restarting. In short, not all memory may be freed, either due to Python reference 
cycles or user-created global data. All the details can be found in the CPython 
documentation.

Recommendation for Developers:
 - Plan Initialization and Reset: Ensure your Python code includes well-defined 
initialization and reset functions that can be called from HDL.
 - Simulation Hooks: Use simulator-specific mechanisms (e.g., callbacks or $restart commands) to integrate the reset process.
 - Testing: Regularly test the behavior of your mixed-language simulation during restarts to verify proper synchronization between HDL and Python states.

By understanding this distinction and explicitly managing the Python code state during
HDL simulation restarts, you can maintain consistency and avoid unintended 
simulation behavior.

Sub-interpreter support
=======================

Creating multiple copies of interpreter is not possible because it
represents the main Python interpreter. Sub-interpreters are something different
and they do permit the existence of multiple interpreters. This is an advanced
feature of the CPython API and should be handled with care. PyStim does not
currently offer a interface for sub-interpreters, so refer to the CPython
documentation for all the details regarding this feature.