Qualcomm has already captured the market of Android smartphones and now the company has long set its sights beyond this core market and looking forward to the field of IoT as one of its next major growth areas. Earlier this year, it launched its new embedded platform for IoT developers. Now, it is introducing two new system-on-a-chip for IoT, the QCS605 and QCS603, that combine a multicore ARM with the company’s AI engine and an image signal processor.
The Company is introducing its first family of system-on-chips (SoCs) purpose-built for the Internet of Things (IoT) using advanced 10nm FinFET process technology. Now the QCS605 and QCS603 SoCs have huge power and thermal efficiency in delivering powerful computing for on-device camera processing and machine learning across a wide range of IoT applications. The SoCs integrate Qualcomm Technologies’ most advanced image signal processor (ISP) to date and the Qualcomm Artificial Intelligence (AI) Engine, along with a heterogeneous computer architecture including a cutting-edge ARM-based multicore CPU, vector processor, and GPU.
The Vision Intelligence Platform has a lot of things like Qualcomm Technologies’ advanced camera processing software, machine learning and computer vision software development kits (SDKs), as well as proven Qualcomm Technologies’ connectivity and security technologies. These chips are to be used on several platforms like the smart security cameras for both consumers and industry, sports cameras, wearable cameras, VR, robotics, and smart displays — it means it can be used anywhere when there is a need for good amount of computing power, as well as the ability to interpret images and run pre-trained machine learning models.
The Vision Intelligence Platform is a lot more useful and advanced than one could think. Artificial Intelligence or AI power is only useful when there is good data to work with, so the team also focused strongly on the actual image processing part of the chips. The Platform integrates the Qualcomm AI Engine with several integrated hardware and software components to accelerate on-device AI.
The Qualcomm AI Engine, with its Qualcomm Snapdragon Neural Processing Engine (NPE) software framework, manages a lot of tasks like analysis, optimization, and debugging tools in order to develop with the help of Tensorflow, Caffe, and Caffe2 frameworks, Open Neural Network Exchange interchange format, as well as Android Neural Networks API and Qualcomm Hexagon Neural Network library. With the Qualcomm AI Engine and Snapdragon NPE software framework, the Vision Intelligence Platform delivers up to 2.1 TOPS* of computing performance for deep neural network inferences, which is more than double compared to the specifications of some of the leading alternative solutions.
Coming to its image quality, there are a lot more surprises as the Vision Intelligence Platform supports up to 4K video resolution at 60 frames per second (fps) or 5.7K at 30 fps which enables multiple videos to stream at lower resolutions. And to give some amazing image quality, the platform integrates the most powerful camera processor ever created by Qualcomm Technologies, a dual 14-bit Qualcomm Spectra 270 ISP supporting dual 16 megapixels sensors the evolution of premium ISP capabilities that has topped DxOMark benchmarks over many generations in the recent past. Some more features of the Vision Intelligence Platform include advanced vision processing capabilities necessary for IoT segments, such as staggered HDR to prevent the “ghost” effect in high-dynamic range video, advanced electronic image stabilization, de-warp, de-noise, chromatic aberration correction, and motion compensated temporal filters in hardware.
Our goal is to make IoT devices significantly smarter as we help customers bring powerful on-device intelligence, camera processing and security. AI is already enabling cameras with object detection, tracking, classification and facial recognition, robots that avoid obstacles autonomously, and action cameras that learn and generate a video summary of your latest adventure, but this is really just the beginning. This venture is going to be a head turner for everybody .The Qualcomm Vision Intelligence Platform is the culmination of years of advanced research and development that brings together breakthrough advancements in camera, on-device AI and heterogeneous computing. The platform is a premier launchpad for manufacturers and developers to create a new world of intelligent IoT devices. – Joseph Bousaba, vice president, product management, Qualcomm Technologies, Inc.
Another feature of the Vision Intelligence Platform is the QCS605 heterogeneous computing architecture with eight Qualcomm Kryo 360 CPU cores, the Qualcomm Adreno 615 GPU, and Hexagon 685 Vector Processor. Along with this, the Vision Intelligence Platform’s integrated display processor provides for a range of display options up to WQHD resolution touch display with hardware accelerated composition, 3D overlays, and support for the major graphics APIs, including OpenGL, OpenCL, and Vulkan. This architecture acts as a support to various high-level operating systems in order to allow developers and manufacturers to orient extraordinary features into their solutions such as on-device stitching for VR 360 cameras, autonomous robot navigation and obstacle avoidance, and video summarization for action cameras.
The Vision Intelligence Platform is said to support up to 2×2 802.11ac Wi-Fi with MU-MIMO and dual-band simultaneous transmission, Bluetooth® 5.1, Qualcomm 3D Audio Suite, Qualcomm Aqstic Audio Technologies and Qualcomm apt Audio. The platform also features Qualcomm Noise and Echo Cancellation along with advanced on-device audio analytics and processing features so that it can support natural language processing, audio speech recognition, and “barge-in” capability for a reliable voice interface even in loud or noisy environments or when users are away from the device.
Even the platform’s hardware-based security is designed to help support trustworthy IoT devices that constitute features like secure boot from hardware root of trust, trusted execution environment, hardware crypto engines, storage security, debug security with lifecycle control, key provisioning and wireless protocol security.
Also to the accelerate development further, manufacturers can rely on an ecosystem of technology providers offering solutions that are complementary to the Vision Intelligence Platform. These include AI providers such as SenseTime for face, image and object recognition, Pilot.ai for a variety of vision tasks such as detection, classification, and tracking of actions and objects, and MM Solutions for cutting-edge image quality tuning services.
The QCS605-based VR 360 reference camera designs from Qualcomm Technologies and Altek Corporation, a leading original design manufacturer for cameras, are available now. Whereas the QCS603-based industrial security camera reference designs are supposed to be available from the second half of 2018.
