FabLab Kamp-Lintfort

User Tools

Site Tools

Trace: • step-by-step-example
projects:farmrobot:step-by-step-example

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision 		Previous revision
projects:farmrobot:step-by-step-example [2021/02/27 15:48]
jason 		projects:farmrobot:step-by-step-example [2021/04/01 02:25] (current)
Line 17: Line 17:
   * Display (over HDMI or DisplayPort)   * Display (over HDMI or DisplayPort)
   * USB keyboard   * USB keyboard
-  * CSI or USB camera+  * CSI or USB camera (In this example: [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]])
   * Configured Arduino   * Configured Arduino
  
Line 46: Line 46:
 You should be able to see a ''video0'' device in the terminal output. This represents the connected camera. You should be able to see a ''video0'' device in the terminal output. This represents the connected camera.
  
-The [[https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/|NVIDIA Jetson Xavier NX]] provides a tool called ''nvgstcapture-1.0'' that can test a cameras output by using pre-constructed [[https://gstreamer.freedesktop.org/|GStreamer]] pipelines.+The [[https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/|NVIDIA Jetson Xavier NX]] provides a tool called ''nvgstcapture-1.0'' that can test a camera's output by using pre-constructed [[https://gstreamer.freedesktop.org/|GStreamer]] pipelines.
  
 In this example we assume that we have connected a [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]] camera, which is a USB camera. In this example we assume that we have connected a [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]] camera, which is a USB camera.
Line 59: Line 59:
  
 <WRAP info> <WRAP info>
-Consult the repository's [[https://git.hsrw.eu/jason.theiler/farm-robot-yolov5/-/blob/master/README.md|README]] to get more information about the ''nvgstcapture-1.0'' tool.+Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about the ''nvgstcapture-1.0'' tool.
 </WRAP> </WRAP>
 +
 +Furthermore, you should check the camera's supported formats, as those will be relevant later when configuring [[https://github.com/ultralytics/yolov5|YOLOv5]]'s source.
 +
 +The ''v4l-utils'' package makes this process very simple. First, install it via ''apt'', like so:
 +
 +<file shell>
 +sudo apt install -y v4l-utils
 +</file>
 +
 +If the installation was successful, you can go ahead and use it:
 +
 +<file shell>
 +v4l2-ctl -d /dev/video0 --list-formats-ext
 +</file>
 +
 +For the camera used in this example ([[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]]) it will produce the following output:
 +
 +<file>
 +ioctl: VIDIOC_ENUM_FMT
 +        Index       : 0
 +        Type        : Video Capture
 +        Pixel Format: 'MJPG' (compressed)
 +        Name        : Motion-JPEG
 +                Size: Discrete 1600x1200
 +                        Interval: Discrete 0.067s (15.000 fps)
 +                Size: Discrete 3264x2448
 +                        Interval: Discrete 0.067s (15.000 fps)
 +                Size: Discrete 2592x1944
 +                        Interval: Discrete 0.067s (15.000 fps)
 +                Size: Discrete 2048x1536
 +                        Interval: Discrete 0.067s (15.000 fps)
 +                Size: Discrete 1280x960
 +                        Interval: Discrete 0.067s (15.000 fps)
 +                Size: Discrete 1024x768
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 800x600
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 640x480
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 329x240
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 1600x1200
 +                        Interval: Discrete 0.067s (15.000 fps)
 + 
 +        Index       : 1
 +        Type        : Video Capture
 +        Pixel Format: 'YUYV'
 +        Name        : YUYV 4:2:2
 +                Size: Discrete 1600x1200
 +                        Interval: Discrete 0.100s (10.000 fps)
 +                Size: Discrete 3264x2448
 +                        Interval: Discrete 0.067s (2.000 fps)
 +                Size: Discrete 2592x1944
 +                        Interval: Discrete 0.333s (3.000 fps)
 +                Size: Discrete 2048x1536
 +                        Interval: Discrete 0.333s (3.000 fps)
 +                Size: Discrete 1280x960
 +                        Interval: Discrete 0.100s (10.000 fps)
 +                Size: Discrete 1024x768
 +                        Interval: Discrete 0.100s (10.000 fps)
 +                Size: Discrete 800x600
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 640x480
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 329x240
 +                        Interval: Discrete 0.067s (30.000 fps)
 +                Size: Discrete 1600x1200
 +                        Interval: Discrete 0.100s (10.000 fps)
 +</file>
  
 ==== - Checking configured Arduino ==== ==== - Checking configured Arduino ====
Line 73: Line 142:
  
 ===== - Setting up our custom implementation of YOLOv5 ===== ===== - Setting up our custom implementation of YOLOv5 =====
 +
 +==== - Cloning the repositories ====
 +
 +First, clone the [[https://gitlab.com/fablabkamplintfort1/farmrobot|repository]] into the home (''~'') directory:
 +
 +<file shell>
 +git clone https://gitlab.com/fablabkamplintfort1/farmrobot.git
 +</file>
 +
 +Then, change into the repository's directory and clone the official [[https://github.com/ultralytics/yolov5|YOLOv5]] into it:
 +
 +<file shell>
 +cd farm-robot-yolov5
 +
 +git clone https://github.com/ultralytics/yolov5.git
 +</file>
 +
 +Once this is set up, you are theoretically ready to build the [[https://www.docker.com/|Docker]] image. But you usually want to configure the implementation via the ''.toml'' configuration files first.
 +
 +==== - Configuration ====
 +
 +The repository comes with a ''default_config.toml'' which contains the default configuration. **However, you should not edit this file directly, since it is version controlled by the repository!** Instead, copy the file with a new name (''config.toml''), like so:
 +
 +<file shell>
 +cp default_config.toml config.toml
 +</file>
 +
 +The ''config.toml'' file is ignored by default, so you can edit it however you like.
 +
 +In the newly created ''config.toml'' you will have to edit atleast one value, the ''source''. The ''source'' value should consist of a [[https://gstreamer.freedesktop.org/|GStreamer]] pipeline. With the [[https://www.elpcctv.com/-p-248.html|ELP-USB8MP02G-SFV]] camera, you can use the following [[https://gstreamer.freedesktop.org/|GStreamer]] pipeline:
 +
 +<file shell>
 +v4l2src device=/dev/video0 ! image/jpeg, width=2592, height=1944, framerate=15/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv flip-method=0 ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink
 +</file>
 +
 +<WRAP important>
 +It is important to replace the ''width='', ''height='' and ''framerate='' values, with the values of a supported format of the used camera!
 +</WRAP>
 +
 +The configured ''config.toml'' should look something like this now:
 +
 +<file toml>
 +[yolov5]
 +# Input source.
 +source = "v4l2src device=/dev/video0 ! image/jpeg, width=2592, height=1944, framerate=15/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv flip-method=0 ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink"
 +
 +...
 +</file>
 +
 +<WRAP info>
 +Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about [[https://gstreamer.freedesktop.org/|GStreamer]] pipelines.
 +</WRAP>
 +
 +<WRAP important>
 +If you intend to run the [[https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-xavier-nx/|NVIDIA Jetson Xavier NX]] without an internet connection, you have to manually download the configured ''weights'', before you build and instantiate the [[https://www.docker.com/|Docker]] container. The default [[https://github.com/ultralytics/yolov5|YOLOv5]] weights can be found on its [[https://github.com/ultralytics/yolov5/releases|releases page]].
 +</WRAP>
 +
 +==== - Building the Docker image ====
 +
 +Building the [[https://www.docker.com/|Docker]] image is easy and straightforward. Simply run:
 +
 +<file shell>
 +sudo docker build -t farm-robot-yolov5 .
 +</file>
 +
 +<WRAP important>
 +The build process of the [[https://www.docker.com/|Docker]] image could take up to an hour!
 +</WRAP>
 +
 +==== - Instantiating the Docker container ====
 +
 +When instantiating the [[https://www.docker.com/|Docker]] container you need to pay attention to mount all required devices (camera and configured Arduino)! With the devices used in this example, you can instantiate the container, like so:
 +
 +<file shell>
 +sudo docker run -it --rm --runtime nvidia --device /dev/video0 --device /dev/ttyUSB0 farm-robot-yolov5
 +</file>
 +
 +<WRAP info>
 +Consult the repository's [[https://gitlab.com/fablabkamplintfort1/farmrobot/-/blob/master/README.md|README]] to get more information about mounting devices to a [[https://www.docker.com/|Docker]] container.
 +</WRAP>
 +
 +==== - Starting the inference ====
 +
 +Once the interactive [[https://www.docker.com/|Docker]] container booted, you can run the ''__main__.py'':
 +
 +<file shell>
 +python3 .
 +</file>
 +
 +If everything is configured correctly, the delta arm should start to move to its home position and then to its configured initial position. A few seconds after that, [[https://github.com/ultralytics/yolov5|YOLOv5]]'s inference should start. When [[https://github.com/ultralytics/yolov5|YOLOv5]] detects an object that has been declared as a "target" (via the ''targets'' list in the ''config.toml''), the delta arm should move to the object's position.
 +
projects/farmrobot/step-by-step-example.1614440928.txt.gz · Last modified: 2021/02/27 15:48 by jason

Page Tools

Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 4.0 International
CC Attribution-Share Alike 4.0 International Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki