====== NVIDIA Jetson TX1 ======

===== Users =====

^ User    ^ Password  ^
| ''jetson''  | ''jetson''    |

===== Running YOLOv5 with on-board CSI camera or external USB camera (Motion JPEG) =====

Arguments for NVIDIA's GStreamer camera pipeline depend on supported camera formats.

To start YOLOv5's detection, run the following command and provide the GStreamer camera pipeline as the source:

<file shell>
# On-board CSI camera
python3.8 detect.py --source "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)2592, height=(int)1458, framerate=(fraction)30/1, format=(string)NV12 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"

# External USB camera (Motion JPEG)
python3.8 detect.py --source "v4l2src device=/dev/video1 ! image/jpeg, width=(int)1920, height=(int)1080, framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink"
</file>

<WRAP important>
In the example above, the on-board CSI camera is represented by ''/dev/video0'' and the external USB camera by ''/dev/video1''. Depending on the configuration of connected cameras, those paths might differ.
</WRAP>

<WRAP info>
See this [[https://stackoverflow.com/questions/65638140/create-pipeline-for-gstreamer-for-usb-camera-mjpg-format|Stack Overflow post]].
</WRAP>

===== Current data =====

==== On-board camera ====

<file shell>
sudo apt install -y v4l-utils
</file>

<file shell>
v4l2-ctl -d /dev/video0 --list-formats-ext

ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'BG10'
        Name        : 10-bit Bayer BGBG/GRGR
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1458
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.008s (120.000 fps)
</file>

{{:projects:farmrobot:img_20210106_183252.jpg?400|}}
{{:projects:farmrobot:img_20210106_183318.jpg?400|}}
{{:projects:farmrobot:img_20210106_183359.jpg?400|}}
{{:projects:farmrobot:img_20210106_183426.jpg?400|}}
{{:projects:farmrobot:img_20210107_175211.jpg?400|}}

==== Measurements ====

YOLOv5 (street.mp4): 11/12 fps (~0.086s +- 0.005s per frame)

{{:projects:farmrobot:img_20201228_205724.jpg?600|}}

===== Setup =====

==== NVIDIA's JetPack SDK ====

The setup of the NVIDIA Jetson TX1 starts with a clean installation of [[https://developer.nvidia.com/embedded/jetpack|NVIDIA's JetPack SDK]].
To install the JetPack SDK on the Jetson TX1 you need to download and install [[https://developer.nvidia.com/nvsdk-manager|NVIDIA's SDK Manager]] onto a separate host machine with a 16.04 or 18.04 Ubuntu operating system. You will need to get an NVIDIA Developer Program Membership to download the SDK Manager.

After downloading the ''.deb'' file, you need to install it with ''apt'':

<file shell>
sudo apt install -y ./<path to file>.deb
</file>

<WRAP important>
If the installation fails because some dependencies couldn't be resolved, try enabling all common ''apt'' repositories:

<file shell>
sudo add-apt-repository main
sudo add-apt-repository universe
sudo add-apt-repository restricted
sudo add-apt-repository multiverse
</file>
</WRAP>

If the installation was successful, you should be able to run the SDK Manager, like so:

<file shell>
sdkmanager
</file>

When the SDK Manager starts, you first need to log in with your NVIDIA Developer Program Membership account. Next, you will be prompted to select a Jetson model. Select the NVIDIA Jetson TX1 and start connecting the Jetson TX1 to your host machine with the included USB A to USB micro B cable. It needs to be powered off when you connect it. After you have connected it, boot the Jetson TX1 in Force USB Recovery Mode. To do that press and hold down the //RECOVERY FORCE// button, then press the //POWER// button, after which you can release the //RECOVERY FORCE// button. Now when you hit //refresh// in the SDK Manager, it should say that it's connected to the Jetson TX1.

<WRAP info>
It's advised to have a monitor, keyboard and mouse connected to the Jetson TX1 at this point, since it will be necessary to finalize the JetPack SDK installation on the Jetson TX1 itself.
</WRAP>

You can now start the installation.

<WRAP important>
If the Jetson TX1 already has a working JetPack SDK installed, you should first uninstall it by selecting the //Uninstall & Repair// option on the bottom left.
</WRAP>

At some point it may ask you to put the Jetson TX1 in Force USB Recovery Mode again. Just repeat the steps described above, to put it in this mode again. When the OS image installation is finished, it will ask you to finalize the Ubuntu installation on the Jetson TX1, which includes creating a default user. The credentials of the default user then need to be entered in the popup of the SDK Manager, so that it can continue installing additional SDK tools.

After the installation finished, disconnect the Jetson TX1 from the host machine.

==== Mounting external storage to /home ====

Mount the partition on the SD-card to ''/mnt'':

<file shell>
sudo mount /dev/mmcblk2p1 /mnt
</file>

Change into the ''/home'' directory and copy all contents to ''/mnt'':

<file shell>
cd /home
cp -adpR * /mnt
</file>

Unmount the partition from ''/mnt'':

<file shell>
sudo umount /mnt
</file>

Then open the disk utility and go to the mounting settings of the partition on the SD-card.
Set it to automatically mount on ''/home'' on boot, save and reboot the Jetson TX1.

==== Updating apt packages ====

It's always recommended to keep your installed packages up-to-date by running:

<file shell>
sudo apt update && sudo apt upgrade -y
</file>

==== Installing Python 3.8 ====

To install Python 3.8 you first need to make sure that the ''software-properties-common'' package is installed:

<file shell>
sudo apt install -y software-properties-common
</file>

Then add the ''deadsnakes'' PPA to ''apt'', like so:

<file shell>
sudo add-apt-repository ppa:deadsnakes/ppa
</file>

Afterwards, you can install Python 3.8 via ''apt'':

<file shell>
sudo apt install -y python3.8
</file>

Also make sure to have ''pip'' for Python 3.X installed:

<file shell>
sudo apt install -y python3-pip
</file>


==== Adding temporary swap space ====

The next few operations can be very CPU and RAM intensive.

It's possible that the installed 4GB of RAM and pre-configured 2GB of swap space are not enough to handle these operations. That's why you may need to configure additional swap space.

First, check if any additional swap space is already configured by running:

<file shell>
sudo swapon --show
</file>

If there's no additional swap space configured apart from the default 2GB, proceed with the following steps.

Create a swap file that allocates enough space (Recommended are an additional 4 - 16GB):

<file shell>
sudo fallocate -l 8G ~/swapfile
</file>

Make sure that only superusers are able to read and write the swap file:

<file shell>
sudo chmod 600 ~/swapfile
</file>

Then set up a swap area on the swap file and activate it:

<file shell>
sudo mkswap ~/swapfile

sudo swapon ~/swapfile
</file>

==== Building OpenCV from source (with GStreamer support) ====

Because [[https://opencv.org/|OpenCV]] does not support NVIDIA's GStreamer software by default, it has to be built from source with GStreamer support enabled.

<WRAP info>
[[https://developer.download.nvidia.com/embedded/L4T/r32_Release_v1.0/Docs/Accelerated_GStreamer_User_Guide.pdf?9N7jupgUlde8JwGYrJQO434MhxI7ZlJf8jfNYE-eI9uMBECrQldp3897ijsNYjnxBWWiY8OtUVfEKr4cLQNUyoevaUfjyXksRnT__8EmmhKVHWYbRK7-6NdQOQ8jJtev1-0IhZKyQ1rxjQh2_35qOofKmyWN8kH58nZHAQyVRax9kaD5tXY|Accelerated GStreamer User Guide]]
</WRAP>

To get started, install the packages that OpenCV with GStreamer support depends on:

<file shell>
sudo apt install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev
</file>

Then, install [[https://numpy.org/|NumPy]] as OpenCV also depends on it:

<file shell>
python3.8 -m pip install --user numpy
</file>

After all dependencies are installed, go ahead and clone the [[https://github.com/opencv/opencv|OpenCV GitHub repository]], change into its directory and checkout the latest version branch:

<file shell>
git clone https://github.com/opencv/opencv.git

cd opencv

# General
git checkout <version>

# OpenCV 4.5.0
git checkout 4.5.0
</file>

Before you can start building OpenCV, create a ''/build`'' directory and change into it:

<file shell>
mkdir build

cd build
</file>

Then, use ''cmake'' to prepare the build with the correct settings:

<file shell>
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python3.8) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python3.8 -c “import sys; print(sys.prefix)”) \
-D PYTHON3_EXECUTABLE=$(which python3.8) \
-D PYTHON3_INCLUDE_DIR=$(python3.8 -c “from distutils.sysconfig import get_python_inc; print(get_python_inc())”) \
-D PYTHON3_PACKAGES_PATH=$(python3.8 -c “from site import getsitepackages; print(getsitepackages())”) \
-D WITH_GSTREAMER=ON \
-D BUILD_EXAMPLES=ON ..
</file>

It is important to check the resulting output. Check if the output contains the following important bits:

<file shell>
...

GStreamer:         YES (<version>)

...

Python3:
  Interpreter:     <path>
  Libraries:       <path>
  numpy:           <path>
  install path:    <path>

...
</file>

If everything went well, you can go ahead and build it:

<file shell>
sudo make -j$(nproc)
</file>

When the build is finished, install it:

<file shell>
sudo make install
</file>

==== Building PyTorch and Torchvision from source ====

At the time of writing this documentation there are no pre-built wheels of [[https://pytorch.org/|PyTorch]] 1.7.0 or later and the associated version of [[https://pytorch.org/docs/stable/torchvision/index.html|Torchvision]] that run on ARM64-based systems with Python 3.8 or later.

<WRAP info>
Both Python 3.8 or later and [[https://pytorch.org/|PyTorch]] 1.7.0 or later are required for [[https://github.com/ultralytics/yolov5|YOLOv5]] to run properly.
</WRAP>

<WRAP important>
You should first check the [[https://forums.developer.nvidia.com/|NVIDIA Developer Forums]] for new releases of PyTorch 1.7.0 or later that run on ARM64-based systems with Python 3.8 or later. You don't have to build PyTorch and Torchvision from source if that is the case.

[[https://forums.developer.nvidia.com/t/pytorch-for-jetson-version-1-7-0-now-available/72048|This]] is the most up-to-date forum post at the time of writing.
</WRAP>

First, make sure that you have ''python3.8-dev'' installed:

<file shell>
sudo apt install -y python3.8-dev
</file>

Then clone the official PyTorch repository on GitHub with the branch of the version you want to build and change into the created directory:

<file shell>
# General
git clone --recursive --branch <version> https://github.com/pytorch/pytorch

cd pytorch

# PyTorch 1.7.0
git clone --recursive --branch 1.7 https://github.com/pytorch/pytorch
</file>

Before you can build PyTorch, you first need to install all its dependencies:

<file shell>
python3.8 -m pip install --user -r requirements.txt
</file>

If all dependencies are installed, you can go ahead and build PyTorch:

<file shell>
python3.8 setup.py install --user
</file>

Once PyTorch itself is installed, you can build Torchvision from source.

First, install all libraries that Torchvision needs:

<file shell>
sudo apt install -y libjpeg-dev zlib1g-dev libpython3-dev libavcodec-dev libavformat-dev libswscale-dev
</file>

The version of Torchvision that is needed depends on the version of PyTorch that is installed. At the time of writing Torchvision 0.8.1 is needed for PyTorch 1.7.0.

Then clone the official Torchvision repository on GitHub with the branch of the version you want to build and change into the created directory:

<file shell>
# General
git clone --branch <version> https://github.com/pytorch/vision torchvision

cd torchvision

# Torchvision 0.8.1
git clone --branch v0.8.1 https://github.com/pytorch/vision torchvision
</file>

To build Torchvision, export Torchvision's version with the environment variable ''BUILD_VERSION'' first:

<file shell>
# General
export BUILD_VERSION=<version>

# Torchvision 0.8.1
export BUILD_VERSION=0.8.1
</file>

Then build Torchvision:

<file shell>
python3.8 setup.py install --user
</file>

==== Installing YOLOv5 ====

Clone our implementation of YOLOv5 from the GitLab repository:

<file shell>
git clone https://gitlab.hsrw.eu/jason.theiler/farm-robot-yolov5.git
</file>

Then change the directory into the local repository and clone the official [[https://github.com/ultralytics/yolov5|YOLOv5 GitHub repository]]:

<file shell>
cd farm-robot-yolov5

git clone https://github.com/ultralytics/yolov5.git
</file>

To get ready, install all the dependencies one by one:

<file shell>
python3.8 -m pip install --user <package>
</file>

<WRAP important>
Installing all dependencies at once using

<file shell>
python3.8 -m pip install --user -r requirements.txt
</file>

can result in errors due to how Python dependencies are installed using the ''--user'' option and the very limited amount of disk space left on the ''/tmp'' directory.
</WRAP>