Tag Archives: computer vision

Role Of Vision Processing With Artificial Neural Networks In Autonomous Driving

In next 10 years, the automotive industry will bring more change than we have seen in the last 50, due to technological advancement. One of the largest changes will be the move to autonomous vehicles, usually known as the self-driving car. Scientists from many universities are striving to implement vision processing with the artificial neural network to provide driver assistance in self-driving cars.

vision processing
Vision processing using convolutional artificial neural networks

Vision processing, as well as artificial neural networks, have been around for many years. Convolutional artificial neural networks (CNN) are sets of algorithms that extract meaningful information from sensor input. CNN’s are very computationally efficient at analyzing a scene. They are also able to identify objects as cars, people, animals, road signs, road junctions, road marking etc. enabling them to determine the relevant reality of the scene. As this system runs in real-time, the decision can be made as soon as the sensing part is complete.

One of the major steps in visual environment understanding for automotive applications is key points tracking and estimating ego-motion and environment structure subsequently from the trajectories of these key points. A propagation based tracking (PBT) method is popularly used to obtain the 2D trajectories from a sequence of images in a monocular camera setup.

The inputs from one or all of the sensors like LIDAR, RADAR, camera, IR, etc. are evaluated and decisions are taken accordingly. For example, if a car in the front suddenly brakes, the onboard computer would instantly verify the distance and calculate the speed with help of the existing sensors. Then it would apply the brakes faster than any human would be able to do. This method helps to prevent an accident with 90% efficiency.

The use of vision processing with CNN is rapidly increasing in automotive applications to enable camera-based autonomous driving. This technology sets a new driving standard. With this technology in our hand, fewer accidents, fewer fatalities, and less pollution are experienced. Vision processing in autonomous driving also enables efficient journeys, reduced crowding, car sharing, and packing cars in more tightly via vehicle to vehicle communication.

JeVois, The Open-Source Smart Vision Camera

JeVois, which can be translated from French as: I see, is an open-source quad-core camera that can be connected easily with your project whether you are using Arduino, Raspberry Pi or just running it on your PC. JeVois contains a video sensor, quad-core CPU, USB video and a serial port in only 1.7 cubic inches. To start working with your JeVois you only need to insert a microSD card loaded with the provided open-source machine vision algorithms and then connecting it to your computer. It will work immediately just by opening a camera software.

The process is as follows: video captured by the camera sensor, processed on JeVois processor, and results are sent over USB to the host computer or to the micro controller.

On your computer, you can use any camera software to see the results, also you can check different vision algorithms by selecting different resolutions and frame rates.

It has the following software and hardware frameworks:

“For ease of programming and configuration, all of the operating system, core JeVois software, and any necessary data files are stored on a single high-speed Micro-SD card that can easily be removed and plugged into a desktop or laptop computer.  The JeVois software framework combines custom Linux kernel drivers for camera sensor and for USB output, written in C, and a custom high-level vision processing framework, written in C++-17. “

Easy to integrate  with other open-source libraries, including tiny-dnn, OpenCV, boost, zBar, Eigen, turbojpeg, etc.  This framework is scalable since the operating system infrastructure is built using the buildroot framework where adding and using different libraries is easy. New vision modules can be added to the core of JeVois thanks to the fact the core software is managed by cmake. Thus, you can customize the vision algorithm you would like to run your JeVois.

In addition, it is easy to use, for example only 4 Wires are needed to connect it with Arduino: 5 or 3.3 V, GND, Tx and Rx!

JeVois is now live in a Kickstarter Campaign, check this video for better understanding:

For more information about the specifications and technical details, check the campaign page. You can pre-order your JeVois now for $45, there are still 20 days to go.

JeVois started as an educational project, to encourage the study of machine vision, computational neuroscience, and machine learning as part of introductory programming and robotics courses at all levels (from K-12 to Ph.D.). It is funded by Science Foundation (NSF) and the Defense Advanced Research Projects Agency (DARPA).

If you are interested in developing the core of JeVois check the documentation provided here.