Software-enabled hyperspectral cameras

The visible range of the electromagnetic spectrum contains information beyond the three RGB values generally expected from traditional color images. This additional information is contained in hyperspectral images.

Hyperspectral image capture can be done using purpose-built hyperspectral cameras. However, these devices are usually expensive, bulky, slow (not real-time), and with a low spatial resolution. There are several attempts to improve the speed and spatial resolution of hyperspectral capture devices, but they lead to a significant reduction in color accuracy. Alternatively, spectral reconstruction from RGB images can be used as a means of hyperspectral image capture. This hyperspectral image capture paradigm has recently seen increased interest with the organization of international challenges. However, the best methods presented have prohibitive processing times for realtime applications. In addition, the algorithms are so heavy in terms of memory occupancy that it is impossible to implement them within devices (embedded systems).

Traditional methods can take several minutes to reconstruct a spectral image of a few megapixels. Our method takes a few milliseconds to reconstruct a 4K image.

The methods developed so far are based on the concept of learning a mapping between RGB values and the corresponding spectral response from an appropriate training set consisting of RGB-spectrum value pairs. Several methods for learning this mapping have been reported in the literature, including Radial Basis Functions (RBF), sparse coding, and Convolutional Neural Networks (CNN), with the best methods being CNN/deep learning based.

These methods are extremely onerous in terms of computational resources and computational time.

The software method developed and patented by us allows a spectral reconstruction from RGB data accurate and extremely efficient.

The performance in terms of color accuracy is comparable to the best methods presented in the literature, while from a computational point of view the method developed by us is thousands of times more efficient than the state of the art methods. The memory occupancy of our method is extremely low and would easily allow its integration into an embedded acquisition device.

Our method could then allow the development of hyperspectral cameras from traditional RGB (or RGB+IR) cameras simply by implementing our method in the firmware of the camera itself (software embedding) or as a post-processing of image and/or video acquisition.

Once the partnership is established, the time for the integration of our software in the camera would be extremely short (a few weeks), it does not present any business risks and could be brought to market in a short time. We also see several opportunities for further development starting from our method and our knowledge in the field of imaging (www.ivl.disco.unimib.it).