Tag Archives: computer vision

Pixy 2 – Computer Vision at a Whole New Level

Computer vision started as a way for computers to understand their surroundings, this requires making a computer with a high-level understanding of digital images or videos. A device that performs computer vision needs to acquire, process, and analyze images to extract data from the real world and turn it into numerical information that can be used for something. The main application for this technology has always been artificial intelligence since giving a computer the ability to understand its surroundings (and learn from them) it’s a huge step towards decision making which is a fundamental part of AI.

Makers have also started using this type of technology which lead Charmed Labs to create Pixy in 2013. Pixy is a small, easily programmable device used to recognize certain things in its sight. Pixy can be taught objects, and it can also recognize color codes. This year, Pixy 2 was announced, and it can do everything Pixy could plus some additional features.

Pixy 2 has a custom pan tilt mechanism, making it easy to look around. Also, the image processing is now at 60 frames per second. It includes new algorithms for line detection, so it can track lines, and it’s now capable of identifying intersections, and reading signals to make decisions. Signals are simple barcodes which can be printed out and can be easily programmed to a certain instruction to be performed at the sight of that specific barcode.

The device includes a cable to plug it directly into the Arduino, or it can be connected to Raspberry PI via USB cable. It can also communicate via SPI, I2C and UART giving the makers a wide range of options to work with. Finally, the new version has a LED light meant to be used in dark spaces.

A lot of projects for Pixy can be found on the internet, and with the new additions that Pixy 2 offers, there would soon be a lot of applications for this device too. Pixy 2 is smaller, faster, and smarter. As a result, makers will find creative ways to exploit these characteristics in their projects. Finally, Pixy can also be used with Lego Mindstorms (NXT and EV3).

The first Pixy was launched on Kickstarter, but Pixy 2 is not crowdfunding, and its already available to be bought on Amazon or on its official website.

Role Of Vision Processing With Artificial Neural Networks In Autonomous Driving

In next 10 years, the automotive industry will bring more change than we have seen in the last 50, due to technological advancement. One of the largest changes will be the move to autonomous vehicles, usually known as the self-driving car. Scientists from many universities are striving to implement vision processing with the artificial neural network to provide driver assistance in self-driving cars.

vision processing
Vision processing using convolutional artificial neural networks

Vision processing, as well as artificial neural networks, have been around for many years. Convolutional artificial neural networks (CNN) are sets of algorithms that extract meaningful information from sensor input. CNN’s are very computationally efficient at analyzing a scene. They are also able to identify objects as cars, people, animals, road signs, road junctions, road marking etc. enabling them to determine the relevant reality of the scene. As this system runs in real-time, the decision can be made as soon as the sensing part is complete.

One of the major steps in visual environment understanding for automotive applications is key points tracking and estimating ego-motion and environment structure subsequently from the trajectories of these key points. A propagation based tracking (PBT) method is popularly used to obtain the 2D trajectories from a sequence of images in a monocular camera setup.

The inputs from one or all of the sensors like LIDAR, RADAR, camera, IR, etc. are evaluated and decisions are taken accordingly. For example, if a car in the front suddenly brakes, the onboard computer would instantly verify the distance and calculate the speed with help of the existing sensors. Then it would apply the brakes faster than any human would be able to do. This method helps to prevent an accident with 90% efficiency.

The use of vision processing with CNN is rapidly increasing in automotive applications to enable camera-based autonomous driving. This technology sets a new driving standard. With this technology in our hand, fewer accidents, fewer fatalities, and less pollution are experienced. Vision processing in autonomous driving also enables efficient journeys, reduced crowding, car sharing, and packing cars in more tightly via vehicle to vehicle communication.

JeVois, The Open-Source Smart Vision Camera

JeVois, which can be translated from French as: I see, is an open-source quad-core camera that can be connected easily with your project whether you are using Arduino, Raspberry Pi or just running it on your PC. JeVois contains a video sensor, quad-core CPU, USB video and a serial port in only 1.7 cubic inches. To start working with your JeVois you only need to insert a microSD card loaded with the provided open-source machine vision algorithms and then connecting it to your computer. It will work immediately just by opening a camera software.

The process is as follows: video captured by the camera sensor, processed on JeVois processor, and results are sent over USB to the host computer or to the micro controller.

On your computer, you can use any camera software to see the results, also you can check different vision algorithms by selecting different resolutions and frame rates.

It has the following software and hardware frameworks:

“For ease of programming and configuration, all of the operating system, core JeVois software, and any necessary data files are stored on a single high-speed Micro-SD card that can easily be removed and plugged into a desktop or laptop computer.  The JeVois software framework combines custom Linux kernel drivers for camera sensor and for USB output, written in C, and a custom high-level vision processing framework, written in C++-17. “

Easy to integrate  with other open-source libraries, including tiny-dnn, OpenCV, boost, zBar, Eigen, turbojpeg, etc.  This framework is scalable since the operating system infrastructure is built using the buildroot framework where adding and using different libraries is easy. New vision modules can be added to the core of JeVois thanks to the fact the core software is managed by cmake. Thus, you can customize the vision algorithm you would like to run your JeVois.

In addition, it is easy to use, for example only 4 Wires are needed to connect it with Arduino: 5 or 3.3 V, GND, Tx and Rx!

JeVois is now live in a Kickstarter Campaign, check this video for better understanding:

For more information about the specifications and technical details, check the campaign page. You can pre-order your JeVois now for $45, there are still 20 days to go.

JeVois started as an educational project, to encourage the study of machine vision, computational neuroscience, and machine learning as part of introductory programming and robotics courses at all levels (from K-12 to Ph.D.). It is funded by Science Foundation (NSF) and the Defense Advanced Research Projects Agency (DARPA).

If you are interested in developing the core of JeVois check the documentation provided here.