NVIDIA Jetson TX1

Arguments for NVIDIA's GStreamer camera pipeline depend on supported camera formats.

To start YOLOv5's detection, run the following command and provide the GStreamer camera pipeline as the source:

# On-board CSI camera
python3.8 detect.py --source "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)2592, height=(int)1458, framerate=(fraction)30/1, format=(string)NV12 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"
 
# External USB camera (Motion JPEG)
python3.8 detect.py --source "v4l2src device=/dev/video1 ! image/jpeg, width=(int)1920, height=(int)1080, framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink"

sudo apt install -y v4l-utils

v4l2-ctl -d /dev/video0 --list-formats-ext
 
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'BG10'
        Name        : 10-bit Bayer BGBG/GRGR
                Size: Discrete 2592x1944
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 2592x1458
                        Interval: Discrete 0.033s (30.000 fps)
                Size: Discrete 1280x720
                        Interval: Discrete 0.008s (120.000 fps)

YOLOv5 (street.mp4): 11/12 fps (~0.086s +- 0.005s per frame)

The setup of the NVIDIA Jetson TX1 starts with a clean installation of NVIDIA's JetPack SDK. To install the JetPack SDK on the Jetson TX1 you need to download and install NVIDIA's SDK Manager onto a separate host machine with a 16.04 or 18.04 Ubuntu operating system. You will need to get an NVIDIA Developer Program Membership to download the SDK Manager.

After downloading the .deb file, you need to install it with apt:

sudo apt install -y ./<path to file>.deb

sudo add-apt-repository main
sudo add-apt-repository universe
sudo add-apt-repository restricted
sudo add-apt-repository multiverse

If the installation was successful, you should be able to run the SDK Manager, like so:

sdkmanager

When the SDK Manager starts, you first need to log in with your NVIDIA Developer Program Membership account. Next, you will be prompted to select a Jetson model. Select the NVIDIA Jetson TX1 and start connecting the Jetson TX1 to your host machine with the included USB A to USB micro B cable. It needs to be powered off when you connect it. After you have connected it, boot the Jetson TX1 in Force USB Recovery Mode. To do that press and hold down the RECOVERY FORCE button, then press the POWER button, after which you can release the RECOVERY FORCE button. Now when you hit refresh in the SDK Manager, it should say that it's connected to the Jetson TX1.

You can now start the installation.

At some point it may ask you to put the Jetson TX1 in Force USB Recovery Mode again. Just repeat the steps described above, to put it in this mode again. When the OS image installation is finished, it will ask you to finalize the Ubuntu installation on the Jetson TX1, which includes creating a default user. The credentials of the default user then need to be entered in the popup of the SDK Manager, so that it can continue installing additional SDK tools.

After the installation finished, disconnect the Jetson TX1 from the host machine.

Mount the partition on the SD-card to /mnt:

sudo mount /dev/mmcblk2p1 /mnt

Change into the /home directory and copy all contents to /mnt:

cd /home
cp -adpR * /mnt

Unmount the partition from /mnt:

sudo umount /mnt

Then open the disk utility and go to the mounting settings of the partition on the SD-card. Set it to automatically mount on /home on boot, save and reboot the Jetson TX1.

It's always recommended to keep your installed packages up-to-date by running:

sudo apt update && sudo apt upgrade -y

To install Python 3.8 you first need to make sure that the software-properties-common package is installed:

sudo apt install -y software-properties-common

Then add the deadsnakes PPA to apt, like so:

sudo add-apt-repository ppa:deadsnakes/ppa

Afterwards, you can install Python 3.8 via apt:

sudo apt install -y python3.8

Also make sure to have pip for Python 3.X installed:

sudo apt install -y python3-pip

The next few operations can be very CPU and RAM intensive.

It's possible that the installed 4GB of RAM and pre-configured 2GB of swap space are not enough to handle these operations. That's why you may need to configure additional swap space.

First, check if any additional swap space is already configured by running:

sudo swapon --show

If there's no additional swap space configured apart from the default 2GB, proceed with the following steps.

Create a swap file that allocates enough space (Recommended are an additional 4 - 16GB):

sudo fallocate -l 8G ~/swapfile

Make sure that only superusers are able to read and write the swap file:

sudo chmod 600 ~/swapfile

Then set up a swap area on the swap file and activate it:

sudo mkswap ~/swapfile
 
sudo swapon ~/swapfile

Because OpenCV does not support NVIDIA's GStreamer software by default, it has to be built from source with GStreamer support enabled.

To get started, install the packages that OpenCV with GStreamer support depends on:

sudo apt install -y libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev

Then, install NumPy as OpenCV also depends on it:

python3.8 -m pip install --user numpy

After all dependencies are installed, go ahead and clone the OpenCV GitHub repository, change into its directory and checkout the latest version branch:

git clone https://github.com/opencv/opencv.git
 
cd opencv
 
# General
git checkout <version>
 
# OpenCV 4.5.0
git checkout 4.5.0

Before you can start building OpenCV, create a /build` directory and change into it:

mkdir build
 
cd build

Then, use cmake to prepare the build with the correct settings:

cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D INSTALL_C_EXAMPLES=OFF \
-D PYTHON_EXECUTABLE=$(which python3.8) \
-D BUILD_opencv_python2=OFF \
-D CMAKE_INSTALL_PREFIX=$(python3.8 -c “import sys; print(sys.prefix)”) \
-D PYTHON3_EXECUTABLE=$(which python3.8) \
-D PYTHON3_INCLUDE_DIR=$(python3.8 -c “from distutils.sysconfig import get_python_inc; print(get_python_inc())”) \
-D PYTHON3_PACKAGES_PATH=$(python3.8 -c “from site import getsitepackages; print(getsitepackages())”) \
-D WITH_GSTREAMER=ON \
-D BUILD_EXAMPLES=ON ..

It is important to check the resulting output. Check if the output contains the following important bits:

...
 
GStreamer:         YES (<version>)
 
...
 
Python3:
  Interpreter:     <path>
  Libraries:       <path>
  numpy:           <path>
  install path:    <path>
 
...

If everything went well, you can go ahead and build it:

sudo make -j$(nproc)

When the build is finished, install it:

sudo make install

At the time of writing this documentation there are no pre-built wheels of PyTorch 1.7.0 or later and the associated version of Torchvision that run on ARM64-based systems with Python 3.8 or later.

First, make sure that you have python3.8-dev installed:

sudo apt install -y python3.8-dev

Then clone the official PyTorch repository on GitHub with the branch of the version you want to build and change into the created directory:

# General
git clone --recursive --branch <version> https://github.com/pytorch/pytorch
 
cd pytorch
 
# PyTorch 1.7.0
git clone --recursive --branch 1.7 https://github.com/pytorch/pytorch

Before you can build PyTorch, you first need to install all its dependencies:

python3.8 -m pip install --user -r requirements.txt

If all dependencies are installed, you can go ahead and build PyTorch:

python3.8 setup.py install --user

Once PyTorch itself is installed, you can build Torchvision from source.

First, install all libraries that Torchvision needs:

sudo apt install -y libjpeg-dev zlib1g-dev libpython3-dev libavcodec-dev libavformat-dev libswscale-dev

The version of Torchvision that is needed depends on the version of PyTorch that is installed. At the time of writing Torchvision 0.8.1 is needed for PyTorch 1.7.0.

Then clone the official Torchvision repository on GitHub with the branch of the version you want to build and change into the created directory:

# General
git clone --branch <version> https://github.com/pytorch/vision torchvision
 
cd torchvision
 
# Torchvision 0.8.1
git clone --branch v0.8.1 https://github.com/pytorch/vision torchvision

To build Torchvision, export Torchvision's version with the environment variable BUILD_VERSION first:

# General
export BUILD_VERSION=<version>
 
# Torchvision 0.8.1
export BUILD_VERSION=0.8.1

Then build Torchvision:

python3.8 setup.py install --user

Clone our implementation of YOLOv5 from the GitLab repository:

git clone https://gitlab.hsrw.eu/jason.theiler/farm-robot-yolov5.git

Then change the directory into the local repository and clone the official YOLOv5 GitHub repository:

cd farm-robot-yolov5
 
git clone https://github.com/ultralytics/yolov5.git

To get ready, install all the dependencies one by one:

python3.8 -m pip install --user <package>

python3.8 -m pip install --user -r requirements.txt