Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleanup the Windows instructions for using conda/pixi. #4989

Open
wants to merge 2 commits into
base: rolling
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 0 additions & 15 deletions source/How-To-Guides/Installation-Troubleshooting.rst
Original file line number Diff line number Diff line change
Expand Up @@ -312,24 +312,9 @@ In the dialog, select Enabled and click OK.

Close and open your terminal to reset the environment and try building again.

CMake packages unable to find asio, tinyxml2, tinyxml, or eigen
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

We've seen that sometimes the chocolatey packages for ``asio``, ``tinyxml2``, etc. do not add important registry entries and CMake will be unable to find them when building ROS 2.
We've not yet been able to identify the root cause, but uninstalling the chocolatey packages (with ``-n`` if the uninstall fails the first time), and then reinstalling them will fix the issue.

patch.exe opens a new command window and asks for administrator
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This will also cause the build of packages which need to use patch to fail, even you allow it to use administrator rights.

- ``choco uninstall patch; colcon build --cmake-clean-cache`` - This is a bug in the `GNU Patch For Windows package <https://chocolatey.org/packages/patch>`_. If this package is not installed, the build process will instead use the version of Patch distributed with git.

Failed to load Fast RTPS shared library
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

.. does not apply to Crystal
Fast RTPS requires ``msvcr20.dll``, which is part of the ``Visual C++ Redistributable Packages for Visual Studio 2013``.
Although it is usually installed by default in Windows 10, we know that some Windows 10-like versions don't have it installed by default (e.g.: Windows Server 2019).
In case you don't have it installed, you can download it from `here <https://www.microsoft.com/en-us/download/details.aspx?id=40784>`_.
Expand Down
201 changes: 96 additions & 105 deletions source/Installation/Alternatives/Windows-Development-Setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Windows (source)
:depth: 2
:local:

This guide is about how to setup a development environment for ROS 2 on Windows.
This page explains how to setup a development environment for ROS 2 on Windows.

System requirements
-------------------
Expand All @@ -22,36 +22,97 @@ Language support
Make sure you have a locale which supports ``UTF-8``.
For example, for a Chinese-language Windows 10 installation, you may need to install an `English language pack <https://support.microsoft.com/en-us/windows/language-packs-for-windows-a5094319-a92d-18de-5b53-1cfc697cfca8>`_.

.. include:: ../_Windows-Install-Prerequisites.rst
Create a location for the ROS 2 installation
--------------------------------------------

Install additional prerequisites from Chocolatey
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This location will contain both the installed binary packages, plus the ROS 2 installation itself.

.. code-block:: bash
Start a powershell session (usually by clicking on the start menu, then typing ``powershell``).

choco install -y cppcheck curl git winflexbison3
Then create a directory to store the installation.
Because of Windows path-length limitations, this should be as short as possible.
We'll use ``C:\dev`` for the rest of these instructions.

You will need to append the Git cmd folder ``C:\Program Files\Git\cmd`` to the PATH (you can do this by clicking the Windows icon, typing "Environment Variables", then clicking on "Edit the system environment variables".
In the resulting dialog, click "Environment Variables", the click "Path" on the bottom pane, then click "Edit" and add the path).
.. code-block:: console
Install Python prerequisites
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
md C:\dev
Install additional Python dependencies:
Install prerequisites
---------------------

.. code-block:: bash
ROS 2 uses `conda-forge <https://conda-forge.org/>`__ as a backend for packages, with `pixi <https://pixi.sh/latest/>`__ as the frontend.

Install MSVC
^^^^^^^^^^^^

In order to compile the ROS 2 code, the MSVC compiler must be installed.
Currently it is recommended to use MSVC 2019.

Continue using the previous powershell session, and run the following command to download it:

.. code-block:: console
irm https://aka.ms/vs/16/release/vs_buildtools.exe -OutFile vs_buildtools_2019.exe
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I first tried this, I got the following error:

irm : The remote name could not be resolved: 'aka.ms'
At line:1 char:1
+ irm https://aka.ms/vs/16/release/vs_buildtools.exe -OutFile vs_buildt ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest
   :HttpWebRequest) [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.Powe
   rShell.Commands.InvokeRestMethodCommand

I resolved this by going to a web browser, typing aka.ms, and logging in to a microsoft account. After that this command worked.

Now install MSVC 2019 (this will take some time):

.. code-block:: console
vs_buildtools_2019.exe --quiet --wait --norestart --add Microsoft.Component.MSBuild --add Microsoft.Net.Component.4.6.1.TargetingPack --add Microsoft.Net.Component.4.8.SDK --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CLI.Support --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.Windows10SDK.19041 --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools
clalancette marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
vs_buildtools_2019.exe --quiet --wait --norestart --add Microsoft.Component.MSBuild --add Microsoft.Net.Component.4.6.1.TargetingPack --add Microsoft.Net.Component.4.8.SDK --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CLI.Support --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.Windows10SDK.19041 --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools
.\vs_buildtools_2019.exe --quiet --wait --norestart --add Microsoft.Component.MSBuild --add Microsoft.Net.Component.4.6.1.TargetingPack --add Microsoft.Net.Component.4.8.SDK --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CLI.Support --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.Windows10SDK.19041 --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools

Need to start command with .\ or else I get the following error:

PS C:\Users\slore> vs_buildtools_2019.exe --quiet --wait --norestart --add Microsoft.Component.MSBuild --add Microsoft.Net.Component.4.6.1.TargetingPack --add Microsoft.Net.Component.4.8.SDK --add Microsoft.VisualStudio.Component.CoreBuildTools --add Microsoft.VisualStudio.Component.Roslyn.Compiler --add Microsoft.VisualStudio.Component.TextTemplating --add Microsoft.VisualStudio.Component.VC.CLI.Support --add Microsoft.VisualStudio.Component.VC.CoreBuildTools --add Microsoft.VisualStudio.Component.VC.CoreIde --add Microsoft.VisualStudio.Component.VC.Redist.14.Latest --add Microsoft.VisualStudio.Component.VC.Tools.x86.x64 --add Microsoft.VisualStudio.Component.Windows10SDK --add Microsoft.VisualStudio.Component.Windows10SDK.19041 --add Microsoft.VisualStudio.ComponentGroup.NativeDesktop.Core --add Microsoft.VisualStudio.Workload.MSBuildTools --add Microsoft.VisualStudio.Workload.VCTools
vs_buildtools_2019.exe : The term 'vs_buildtools_2019.exe' is not
recognized as the name of a cmdlet, function, script file, or operable
program. Check the spelling of the name, or if a path was included, verify
that the path is correct and try again.
At line:1 char:1
+ vs_buildtools_2019.exe --quiet --wait --norestart --add Microsoft.Com ...
+ ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (vs_buildtools_2019.exe:Stri
   ng) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException
 
 
Suggestion [3,General]: The command vs_buildtools_2019.exe was not found, but does exist in the current location. Windows PowerShell does not load commands from the current location by default. If you trust this command, instead type: ".\vs_buildtools_2019.exe". See "get-help about_Command_Precedence" for more details.

pip install -U colcon-common-extensions coverage flake8 flake8-blind-except flake8-builtins flake8-class-newline flake8-comprehensions flake8-deprecated flake8-import-order flake8-quotes mock mypy==0.931 pytest pytest-cov pytest-mock pytest-repeat pytest-rerunfailures pytest-runner vcstool
Install pixi
^^^^^^^^^^^^

Continue using the previous powershell session, and use the instructions on https://pixi.sh/latest/ to install ``pixi``.

Install dependencies
^^^^^^^^^^^^^^^^^^^^

Download the pixi configuration file in the existing powershell session:

.. code-block:: console
cd C:\dev
irm https://raw.githubusercontent.com/ros2/ros2/refs/heads/{REPOS_FILE_BRANCH}/pixi.toml -OutFile pixi.toml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does REPOS_FILE_BRANCH get populated in the docs? I copied this from the github render, so I had to change this to rolling manually.

Install dependencies:

.. code-block:: console
pixi install
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to close and reopen powershell before this command worked. I think it's necessary because pixi isn't on PATH in the current shell after installing it.

PS C:\dev> pixi install
pixi : The term 'pixi' is not recognized as the name of a cmdlet,
function, script file, or operable program. Check the spelling of the
name, or if a path was included, verify that the path is correct and try
again.
At line:1 char:1
+ pixi install
+ ~~~~
    + CategoryInfo          : ObjectNotFound: (pixi:String) [], CommandNot
   FoundException
    + FullyQualifiedErrorId : CommandNotFoundException

You should now close the powershell session, as the rest of the instructions will use the Windows command prompt.

Build ROS 2
-----------

Start a new Windows command prompt, which will be used for the build.

Source the MSVC compiler
^^^^^^^^^^^^^^^^^^^^^^^^

This is required in the command prompt you'll use to compile ROS 2, but it is *not* required when running:

.. code-block:: console
call "C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\VC\Auxiliary\Build\vcvarsall.bat" x86_amd64
Source the pixi environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is required in every command prompt you open to set up paths to the dependencies:

.. code-block:: console
cd C:\dev
pixi shell
Get ROS 2 code
^^^^^^^^^^^^^^

Now that we have the development tools we can get the ROS 2 source code.

First setup a development folder, for example ``C:\{DISTRO}``:
Setup a development folder, for example ``C:\dev\{DISTRO}``:

.. note::

Expand All @@ -60,8 +121,8 @@ First setup a development folder, for example ``C:\{DISTRO}``:

.. code-block:: bash
md \{DISTRO}\src
cd \{DISTRO}
md C:\dev\{DISTRO}\src
cd C:\dev\{DISTRO}
Get the ``ros2.repos`` file which defines the repositories to clone from:

Expand All @@ -80,8 +141,6 @@ Build the code in the workspace

.. _windows-dev-build-ros2:

To build ROS 2 you will need a Visual Studio Command Prompt ("x64 Native Tools Command Prompt for VS 2019") running as Administrator.

To build the ``\{DISTRO}`` folder tree:

.. code-block:: bash
Expand All @@ -93,19 +152,29 @@ To build the ``\{DISTRO}`` folder tree:
We're using ``--merge-install`` here to avoid a ``PATH`` variable that is too long at the end of the build.
If you're adapting these instructions to build a smaller workspace then you might be able to use the default behavior which is isolated install, i.e. where each package is installed to a different folder.

.. note::

If you are doing a debug build use ``python_d path\to\colcon_executable`` ``colcon``.
See `Extra stuff for debug mode`_ for more info on running Python code in debug builds on Windows.

Setup environment
-----------------

Start a command shell and source the ROS 2 setup file to set up the workspace:
Start a new Windows command prompt, which will be used in the examples.

Source the pixi environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is required in every command prompt you open to set up paths to the dependencies:

.. code-block:: console
cd C:\dev
pixi shell
Source the ROS 2 environment
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

This is required in every command prompt you open to setup the ROS 2 workspace:

.. code-block:: bash
call C:\{DISTRO}\install\local_setup.bat
call C:\dev\{DISTRO}\install\local_setup.bat
This will automatically set up the environment for any DDS vendors that support was built for.

Expand Down Expand Up @@ -140,7 +209,7 @@ Then, run a C++ ``talker``\ :
call install\local_setup.bat
ros2 run demo_nodes_cpp talker
In a separate shell you can do the same, but instead run a Python ``listener``\ :
In a separate command prompt you can do the same, but instead run a Python ``listener``\ :

.. code-block:: bash
Expand All @@ -160,84 +229,6 @@ Next steps

Continue with the :doc:`tutorials and demos <../../Tutorials>` to configure your environment, create your own workspace and packages, and learn ROS 2 core concepts.

Extra stuff for Debug mode
--------------------------

If you want to be able to run all the tests in Debug mode, you'll need to install a few more things:

* To be able to extract the Python source tarball, you can use PeaZip:

.. code-block:: bash
choco install -y peazip
* You'll also need SVN, since some of the Python source-build dependencies are checked out via SVN:

.. code-block:: bash
choco install -y svn hg
* You'll need to quit and restart the command prompt after installing the above.
* Get and extract the Python 3.8.3 source from the ``tgz``:

* https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tgz
* To keep these instructions concise, please extract it to ``C:\dev\Python-3.8.3``

* Now, build the Python source in debug mode from a Visual Studio command prompt:

.. code-block:: bash
cd C:\dev\Python-3.8.3\PCbuild
get_externals.bat
build.bat -p x64 -d
* Finally, copy the build products into the Python38 installation directories, next to the Release-mode Python executable and DLL's:

.. code-block:: bash
cd C:\dev\Python-3.8.3\PCbuild\amd64
copy python_d.exe C:\Python38 /Y
copy python38_d.dll C:\Python38 /Y
copy python3_d.dll C:\Python38 /Y
copy python38_d.lib C:\Python38\libs /Y
copy python3_d.lib C:\Python38\libs /Y
copy sqlite3_d.dll C:\Python38\DLLs /Y
for %I in (*_d.pyd) do copy %I C:\Python38\DLLs /Y
* Now, from a fresh command prompt, make sure that ``python_d`` works:
.. code-block:: bash
python_d -c "import _ctypes ; import coverage"
* Once you have verified the operation of ``python_d``, it is necessary to reinstall a few dependencies with the debug-enabled libraries:
.. code-block:: bash
python_d -m pip install --force-reinstall https://github.com/ros2/ros2/releases/download/numpy-archives/numpy-1.18.4-cp38-cp38d-win_amd64.whl
python_d -m pip install --force-reinstall https://github.com/ros2/ros2/releases/download/lxml-archives/lxml-4.5.1-cp38-cp38d-win_amd64.whl
* To verify the installation of these dependencies:
.. code-block:: bash
python_d -c "from lxml import etree ; import numpy"
* When you wish to return to building release binaries, it is necessary to uninstall the debug variants and use the release variants:
.. code-block:: bash
python -m pip uninstall numpy lxml
python -m pip install numpy lxml
* To create executables python scripts(.exe), python_d should be used to invoke colcon
.. code-block:: bash
python_d path\to\colcon_executable build
* Hooray, you're done!
Stay up to date
---------------

Expand All @@ -258,4 +249,4 @@ Uninstall

.. code-block:: bash
rmdir /s /q \ros2_{DISTRO}
rmdir /s /q C:\dev\ros2_{DISTRO}
Loading