.. index:: pair: page; Troubleshooting guide
.. _doxid-guide_troubleshooting:

Troubleshooting guide
=====================



.. _doxid-guide_troubleshooting_1guide_troubleshooting_gazebo_headless:

Running gazebo experiments on a headless machine
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you are trying to run a gazebo experiment on a headless machine (e.g. a virtual machine), you are likely to encounter the following error:

.. ref-code-block:: cpp

	[Err] [CameraSensor.cc:125] Unable to create CameraSensor. Rendering is disabled.

Gazebo needs a defined display to render images. To run on a headless machine, you can use ``Xvfb``, which runs all graphical operations in virtual memory, without displaying anything on a screen.

To install Xvfb on your machine:

.. ref-code-block:: cpp

	sudo apt install xvfb

To run an experiment using Xvfb:

.. ref-code-block:: cpp

	xvfb-run NRPCoreSim -c simulation_config.json

.. note::

	When, in an experiment run in a headless environment (e.g. a docker container), a gazebo Engine is launched by the (default) :ref:`BasicFork <doxid-class_basic_fork>` launcher, a customized version of ``xvfb-run``, named ``xvfb-run-nrp``, shall be used. It takes care of correctly stopping the launched processes.





.. _doxid-guide_troubleshooting_1guide_troubleshooting_gunicorn_timeout:

Worker timeout when using Python JSON Engine
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The problem will be indicated by logs that look like this:

.. ref-code-block:: cpp

	[2022-08-03 15:25:52 +0000] [6908] [CRITICAL] WORKER TIMEOUT (pid:6919)
	[2022-08-03 15:25:52 +0000] [6896] [CRITICAL] WORKER TIMEOUT (pid:6907)
	[2022-08-03 15:26:03 +0000] [6896] [WARNING] Worker with pid 6907 was terminated due to signal 9
	[2022-08-03 15:26:03 +0000] [6908] [WARNING] Worker with pid 6919 was terminated due to signal 9

Increase the timeout by adding the following option to the engine configuration:

.. ref-code-block:: cpp

	"ServerOptions": "{'timeout':600}"

or disable the timeouts:

.. ref-code-block:: cpp

	"ServerOptions": "{'timeout':0}"





.. _doxid-guide_troubleshooting_1guide_troubleshooting_test_symbol_lookup_error:

Symbol Lookup Error in Tests While Building NRP Core
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

It is possible that after making changes to NRP Core source code, shifting to another branch or pulling from the NRP Core git repository upstream, you see a "symbol lookup error" when building NRP Core next time. This happens after the signature of a function which is used in tests changes, even though tests have been updated correctly to reflex the changes in the source code. Concretely, the resulting error is of the form:

.. ref-code-block:: cpp

	<path>/<test library which is failing>: symbol lookup error: <path>/<test library which is failing>: undefined symbol: <some symbol>
	CMake Error at <another path>/GoogleTestAddTests.cmake:112 (message):
	  Error running test executable.
	
	    Path: '<path>/<test library which is failing>'
	    Result: 127
	    Output:
	      
	
	Call Stack (most recent call first):
	  <another path>/GoogleTestAddTests.cmake:225 (gtest_discover_tests_impl)

The error is caused by GoogleTest looking up installed NRP Core libraries instead of the freshly built ones. The easiest way to overcome this error is to **delete the NRP Core install folder**. If you have for example installed NRPCore at "/home/${USER}/.local/nrp" as recommended, just delete this folder and build NRP Core again. The error should disappear.