Differences

This shows you the differences between two versions of the page.

--- projects:farmrobot:step-by-step-example [2021/02/27 14:25]
jason created
+++ projects:farmrobot:step-by-step-example [2021/04/01 02:25] (current)
@@ Line 17: / Line 17: @@
   * Display (over HDMI or DisplayPort)
   * USB keyboard
-  * CSI or USB camera
+  * CSI or USB camera (In this example: [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]])
   * Configured Arduino
@@ Line 44: / Line 44: @@
 </file>
-You should be able to see a ''video0'' device in the command's output. This represents the connected camera.
+You should be able to see a ''video0'' device in the terminal output. This represents the connected camera.
+The [[https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/|NVIDIA Jetson Xavier NX]] provides a tool called ''nvgstcapture-1.0'' that can test a camera's output by using pre-constructed [[https://gstreamer.freedesktop.org/|GStreamer]] pipelines.
+In this example we assume that we have connected a [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]] camera, which is a USB camera.
+To test a USB camera with the ''nvgstcapture-1.0'' tool, run the following:
+<file shell>
+nvgstcapture-1.0 --camsrc=0 --cap-dev-node=0
+</file>
+You should now be able to see the camera's output in an additional window.
+<WRAP info>
+Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about the ''nvgstcapture-1.0'' tool.
+</WRAP>
+Furthermore, you should check the camera's supported formats, as those will be relevant later when configuring [[https://github.com/ultralytics/yolov5|YOLOv5]]'s source.
+The ''v4l-utils'' package makes this process very simple. First, install it via ''apt'', like so:
+<file shell>
+sudo apt install -y v4l-utils
+</file>
+If the installation was successful, you can go ahead and use it:
+<file shell>
+v4l2-ctl -d /dev/video0 --list-formats-ext
+</file>
+For the camera used in this example ([[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]]) it will produce the following output:
+<file>
+ioctl: VIDIOC_ENUM_FMT
+        Index       : 0
+        Type        : Video Capture
+        Pixel Format: 'MJPG' (compressed)
+        Name        : Motion-JPEG
+                Size: Discrete 1600x1200
+                        Interval: Discrete 0.067s (15.000 fps)
+                Size: Discrete 3264x2448
+                        Interval: Discrete 0.067s (15.000 fps)
+                Size: Discrete 2592x1944
+                        Interval: Discrete 0.067s (15.000 fps)
+                Size: Discrete 2048x1536
+                        Interval: Discrete 0.067s (15.000 fps)
+                Size: Discrete 1280x960
+                        Interval: Discrete 0.067s (15.000 fps)
+                Size: Discrete 1024x768
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 800x600
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 640x480
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 329x240
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 1600x1200
+                        Interval: Discrete 0.067s (15.000 fps)
+        Index       : 1
+        Type        : Video Capture
+        Pixel Format: 'YUYV'
+        Name        : YUYV 4:2:2
+                Size: Discrete 1600x1200
+                        Interval: Discrete 0.100s (10.000 fps)
+                Size: Discrete 3264x2448
+                        Interval: Discrete 0.067s (2.000 fps)
+                Size: Discrete 2592x1944
+                        Interval: Discrete 0.333s (3.000 fps)
+                Size: Discrete 2048x1536
+                        Interval: Discrete 0.333s (3.000 fps)
+                Size: Discrete 1280x960
+                        Interval: Discrete 0.100s (10.000 fps)
+                Size: Discrete 1024x768
+                        Interval: Discrete 0.100s (10.000 fps)
+                Size: Discrete 800x600
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 640x480
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 329x240
+                        Interval: Discrete 0.067s (30.000 fps)
+                Size: Discrete 1600x1200
+                        Interval: Discrete 0.100s (10.000 fps)
+</file>
+==== - Checking configured Arduino ====
+List all current devices on the system with:
+<file shell>
+la /dev
+</file>
+You should be able to see a ''ttyUSB0'' device in the terminal output. This represents the connected configured Arduino.
 ===== - Setting up our custom implementation of YOLOv5 =====
+==== - Cloning the repositories ====
+First, clone the [[https://gitlab.com/fablabkamplintfort1/farmrobot|repository]] into the home (''~'') directory:
+<file shell>
+git clone https://gitlab.com/fablabkamplintfort1/farmrobot.git
+</file>
+Then, change into the repository's directory and clone the official [[https://github.com/ultralytics/yolov5|YOLOv5]] into it:
+<file shell>
+cd farm-robot-yolov5
+git clone https://github.com/ultralytics/yolov5.git
+</file>
+Once this is set up, you are theoretically ready to build the [[https://www.docker.com/|Docker]] image. But you usually want to configure the implementation via the ''.toml'' configuration files first.
+==== - Configuration ====
+The repository comes with a ''default_config.toml'' which contains the default configuration. **However, you should not edit this file directly, since it is version controlled by the repository!** Instead, copy the file with a new name (''config.toml''), like so:
+<file shell>
+cp default_config.toml config.toml
+</file>
+The ''config.toml'' file is ignored by default, so you can edit it however you like.
+In the newly created ''config.toml'' you will have to edit atleast one value, the ''source''. The ''source'' value should consist of a [[https://gstreamer.freedesktop.org/|GStreamer]] pipeline. With the [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]] camera, you can use the following [[https://gstreamer.freedesktop.org/|GStreamer]] pipeline:
+<file shell>
+v4l2src device=/dev/video0 ! image/jpeg, width=2592, height=1944, framerate=15/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv flip-method=0 ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink
+</file>
+<WRAP important>
+It is important to replace the ''width='', ''height='' and ''framerate='' values, with the values of a supported format of the used camera!
+</WRAP>
+The configured ''config.toml'' should look something like this now:
+<file toml>
+[yolov5]
+# Input source.
+source = "v4l2src device=/dev/video0 ! image/jpeg, width=2592, height=1944, framerate=15/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv flip-method=0 ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink"
+...
+</file>
+<WRAP info>
+Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about [[https://gstreamer.freedesktop.org/|GStreamer]] pipelines.
+</WRAP>
+<WRAP important>
+If you intend to run the [[https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/|NVIDIA Jetson Xavier NX]] without an internet connection, you have to manually download the configured ''weights'', before you build and instantiate the [[https://www.docker.com/|Docker]] container. The default [[https://github.com/ultralytics/yolov5|YOLOv5]] weights can be found on its [[https://github.com/ultralytics/yolov5/releases|releases page]].
+</WRAP>
+==== - Building the Docker image ====
+Building the [[https://www.docker.com/|Docker]] image is easy and straightforward. Simply run:
+<file shell>
+sudo docker build -t farm-robot-yolov5 .
+</file>
+<WRAP important>
+The build process of the [[https://www.docker.com/|Docker]] image could take up to an hour!
+</WRAP>
+==== - Instantiating the Docker container ====
+When instantiating the [[https://www.docker.com/|Docker]] container you need to pay attention to mount all required devices (camera and configured Arduino)! With the devices used in this example, you can instantiate the container, like so:
+<file shell>
+sudo docker run -it --rm --runtime nvidia --device /dev/video0 --device /dev/ttyUSB0 farm-robot-yolov5
+</file>
+<WRAP info>
+Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about mounting devices to a [[https://www.docker.com/|Docker]] container.
+</WRAP>
+==== - Starting the inference ====
+Once the interactive [[https://www.docker.com/|Docker]] container booted, you can run the ''__main__.py'':
+<file shell>
+python3 .
+</file>
+If everything is configured correctly, the delta arm should start to move to its home position and then to its configured initial position. A few seconds after that, [[https://github.com/ultralytics/yolov5|YOLOv5]]'s inference should start. When [[https://github.com/ultralytics/yolov5|YOLOv5]] detects an object that has been declared as a "target" (via the ''targets'' list in the ''config.toml''), the delta arm should move to the object's position.

FabLab Kamp-Lintfort

User Tools

Site Tools

Differences

Page Tools