Friday, September 16, 2016

Implementing Software-defined radio and Infrared Time-lapse Imaging with Tensorflow on a custom Linux distribution for the Raspberry Pi 3

GNURadio Companion Qt Gui Frequency Sync - multiple FIR filter taps
sample running on Raspberry Pi 3 custom Linux distribution
The Raspberry Pi 3 is powered by the ARM® Cortex®-A53 processor.  This 1.2GHz 64-bit quad-core processor fully supports the ARMv8-A architecture.
For this project, a custom Linux distribution was created for the Raspberry Pi 3.

The custom Linux distribution includes support for GNURadio, several FPGA and ARM Powered® SDR devices, D-STAR (hotspot, repeater, and dongle support), hsuart, libusb, hardware real-time clock support, Sony 14 megapixel NoIR image sensor, HDMI and 3.5mm audio, USB Microphone input, X-windows with xfce, lighttpd and php, bluetooth, WiFi, SSH, TCPDump, Docker, Docker registry, MySQL, Perl, Python, QT, GTK, IPTables, x11vnc, SELinux, and full native-toolchain development support.  

The Sony 14 megapixel image sensor with the infrared filter removed can be connected to the Raspberry Pi 3's MIPI camera serial interface.  Image capture and recognition can then be performed over contiguous periods of time, and time-lapsed video can be created from the images.  With support for Tensorflow and OpenCV, object recognition within images can be performed. 

D-STAR hotspot with time-lapsed infrared imaging.

For the initial run, an infrared Time-lapse Video was created from an initial image capture run of  one 3280x2460 infrared jpeg image captured every 15 seconds for three hours.  40, 5mm, 940nm LEDs, powered by 500ma over 12v DC provided infrared illumination in the 940nm wavelength.

Tensorflow was running in the background (on v4l2 kmod) and providing continuous object recognition and scoring within each image via a sample model.  Finally, OpenCV was also installed in the root file system.

The time-lapse infrared video was captured of my living room using the above setup and custom Linux distribution.  Below this image are images of Tensorflow running in a terminal in the background on the Raspberry Pi 3 and recognizing/scoring objects in my living room.


The Yocto distribution configuration file, image configuration file, custom recipes, and build configuration files are available on github at github.com/bryanhinton/hampi.


Tensorflow running on the Raspberry Pi 3 and continuously capturing frames from the image sensor and scoring objects
GNURadio Companion running on xfce on the Raspberry Pi 3

No comments:

Post a Comment