Streaming video from Raspberry Pi to a powerful computer

Thanks to the hardware-accelerated video encoders in all modern ARM devices, including Raspberry Pi, a valid alternative to performing computer vision on board an embedded device is to use the device to just capture video and stream it across a network in real time to a PC or server rack. All Raspberry Pi models contain the same video encoder hardware, so an Raspberry Pi 1A+ or Raspberry Pi Zero with a Pi Cam is quite a good option for a low-cost, low-power portable video streaming server. Raspberry Pi 3 adds Wi-Fi for additional portable functionality.

There are numerous ways live camera video can be streamed from a Raspberry Pi, such as using the official Raspberry Pi V4L2 camera driver to allow the Raspberry Pi Cam to appear like a webcam, then using GStreamer, liveMedia, netcat, or VLC to stream the video across a network. However, these methods often introduce one or two seconds of latency and often require customizing the OpenCV client code or learning how to use GStreamer efficiently. So instead, the following section will show how to perform both the camera capture and network streaming using an alternative camera driver named UV4L:

1. Install UV4L on the Raspberry Pi by following the instructions at http://www.linux-projects.org/uv4l/installation/:

curl http://www.linux-projects.org/listing/uv4l_repo/lrkey.asc
 sudo apt-key add -
 sudo su
 echo "# UV4L camera streaming repo:">> /etc/apt/sources.list
 echo "deb http://www.linux-
 projects.org/listing/uv4l_repo/raspbian/jessie main">>
 /etc/apt/sources.list
 exit
 sudo apt-get update
 sudo apt-get install uv4l uv4l-raspicam uv4l-server

1. Run the UV4L streaming server manually (on the Raspberry Pi) to check that it works:

sudo killall uv4l
 sudo LD_PRELOAD=/usr/lib/uv4l/uv4lext/armv6l/libuv4lext.so
 uv4l -v7 -f --sched-rr --mem-lock --auto-video_nr
 --driverraspicam --encoding mjpeg
 --width 640 --height 480 --framerate15

1. Test the camera's network stream from your desktop, following these steps to check all is working fine:
   • Install VLC Media Player.
   • Navigate to Media | Open Network Stream and enter http://192.168.2.111:8080/stream/video.mjpeg.
   • Adjust the URL to the IP address of your Raspberry Pi. Run hostname -I on Raspberry Pi to find its IP address.
2. Run the UV4L server automatically on bootup:

sudo apt-get install uv4l-raspicam-extras

1. Edit any UV4L server settings you want in uv4l-raspicam.conf, such as resolution and frame rate to customize the streaming:

sudo nano /etc/uv4l/uv4l-raspicam.conf
 drop-bad-frames = yes
 nopreview = yes
 width = 640
 height = 480
 framerate = 24

You will need to reboot to make all changes take effect.

1. Tell OpenCV to use our network stream as if it was a webcam. As long as your installation of OpenCV can use FFMPEG internally, OpenCV will be able to grab frames from an MJPEG network stream just like a webcam:

./Cartoonifier http://192.168.2.101:8080/stream/video.mjpeg

Your Raspberry Pi is now using UV4L to stream the live 640 x 480 24 FPS video to a PC that is running Cartoonifier in sketch mode, achieving roughly 19 FPS (with 0.4 seconds of latency). Notice this is almost the same speed as using the PS3 Eye webcam directly on the PC (20 FPS)!

Note that when you are streaming the video to OpenCV, it won't be able to set the camera resolution; you need to adjust the UV4L server settings to change the camera resolution. Also note that instead of streaming MJPEG, we could have streamed H.264 video, which uses a lower bandwidth, but some computer vision algorithms don't handle video compression such as H.264 very well, so MJPEG will cause fewer algorithm problems than H.264.