PxE's new Holographic RGB-IR-Depth Cameratechnology can transform standard 2D cameras in ... [+] smartphones, laptops, and cars into immersive, high-performance 3D cameras. The technology would enable the camera to capture color, infrared, and depth data simultaneously from one sensor.P X E Holographic ImagingIn 2022, 81% of smartphones had biometrics enabled, including face unlock.The face unlock feature on smartphones today traditionally requires three technologies: a laser dot projector, an infrared illuminator, and a standard 2D selfie camera.According to Yoav Berlatzky, CEO and co-founder of PxE Holographic Imaging, this results in a complex and expensive system that limits the feature to high-end phones like the iPhone.Earlier this month, PxE introduced a new Holographic RGB-IR-Depth CameraBerlatzky says that combining these technologies into one sensorcolor and 3D informationallows the face unlock feature to be released to all smartphones and laptop models, not just high-end devices."This would enable 2D color, infrared, and depth images per frame, and on a single sensor with the potential to upgrade all standard 2D cameras on the market to advanced 3D camera systems," said Berlatzky. "It can completely transform how consumers experience and capture the world through their everyday devices."Berlatzky says other applications for the technology will enhance user experiences, such as improved photography and portrait mode. "For example, background blur known as bokeh can be improved through depth mapping and augmented HDR, enabling smartphones to optimize lighting and exposures better."2D images, depth maps, and infrared images at the same time"Standard color cameras consist of a lens that focuses rays of light on a light-sensitive film, namely, the Bayer filter, which captures the intensity of light and its color, and this information is recorded on a CMOS imaging sensor," said BerlatzkyPxE's holographic imaging sensor utilizes the same lens and CMOS imaging sensor, but instead of treating light as rays onto a light-sensitive film, It harnesses the wavefront information of light."To implement PxE's holographic imaging, the Bayer filter is replaced with a layer of holographic optics, PxE's HoloCoder," said Berlatzky. "Instead of red, blue, and green being filtered by the Bayer filter, the HoloCoder produces a white light hologram consisting of the light's wavefront information."This means that the captured holograms are decoded in real time by the company's proprietary, fully deterministic algorithm and then output as natively fused RGB-IR-Depth color, infrared, and 3D channels. Standard cameras can only generate colored 2D images, but our camera can generate colored 2D images, depth maps, and infrared images at the same time.Emerging applicationsBerlatzky says that new and upcoming applications, such as augmented reality (AR) and gaming, 3D scanning for online shopping and environment mapping, gesture recognition and touchless control, enhanced medical and wellness applications, and all things metaverse, spatial computing, and personalized AI interactions, will require 3D input."The enhancement of mobile phone cameras over the years has essentially brought professional-level cameras into the hands of consumers, unlocking a world of user-generated content, influencers, etc.," said Berlatzky. "Bringing 3D into the mix without any additional cost to the consumer will unlock the next generation of photography and videography at our fingertips.""PxE's technology enables these applications to be implemented across all phone and laptop models without adding additional cost or complexity to the product," he said.Digital twin applicationsHolographic technology from PxE enhances image quality and opens the door to immersive 3D imaging, providing a real-time digital twin on the device.A digital twin is a virtual representation of a physical object. Across the industry, digital twins are being used in markets such as energy, aerospace, automotive, construction, healthcare, and retail, utilities to allow companies to improve and optimize design and critical infrastructure, predictive maintenance, urban planning, predict diseases, and test treatments.However, according to Berlatzky, digital twins are also being used in the metaverse, smart homes, and other AI-driven applications."Users can interact with other people in the metaverse as if it were in real life as opposed to an avatar," he said. "A good example is enabling a virtual shopping experience where you can browse a store and its products online as if you were in a physical store and then use your digital twin to have a very accurate understanding of how an item will look on you."Drone applicationsAccording to Berlatzky, each product and industry has a different use case for depth and color images.In smartphones, it is user-centric and enables biometric features like face unlock or background blur."For drones, the usage can be obstacle avoidance and landing instructions," said Berlatzky. "Drones would benefit from a single-sensor solution like PxE because they are very sensitive to weight and size, and today's drones require many sensors to map the environment with color and depth information in order to have a safe and semi-autonomous drone."By reducing the form factor to a single sensor while still providing relevant information, drones can reduce their size and power consumption. This makes them safer and more operational, said Berlatzky.