Tony Kim
Oct 08, 2024 06:23
Microsoft Bing Visible Search achieves a 5.13x speedup utilizing NVIDIA’s TensorRT, CV-CUDA, and nvImageCodec, enhancing effectivity and lowering prices.
Microsoft Bing Visible Search, a software enabling customers worldwide to go looking utilizing images, has been considerably optimized via a collaboration with NVIDIA, leading to a outstanding efficiency increase. In keeping with NVIDIA Technical Weblog, the combination of NVIDIA’s TensorRT, CV-CUDA, and nvImageCodec into Bing’s TuringMM visible embedding mannequin has led to a 5.13x improve in throughput for offline indexing pipelines, lowering each vitality consumption and prices.
Multimodal AI and Visible Search
Multimodal AI applied sciences, like Microsoft’s TuringMM, are important for functions that require seamless interplay between totally different information varieties comparable to textual content and pictures. A preferred mannequin for joint image-text understanding is CLIP, which makes use of a twin encoder structure to course of a whole bunch of thousands and thousands of image-caption pairs. These superior fashions are important for duties comparable to text-based visible search, zero-shot picture classification, and picture captioning.
Optimization Efforts
The optimization of Bing’s visible embedding pipeline was achieved by leveraging NVIDIA’s GPU acceleration applied sciences. The hassle centered on enhancing the efficiency of the TuringMM pipeline by utilizing NVIDIA’s TensorRT for mannequin execution, which improved the effectivity of computationally costly layers in transformer architectures. Moreover, the usage of nvImageCodec and CV-CUDA accelerated the picture decoding and preprocessing levels, resulting in a big discount in latency for picture processing duties.
Implementation and Outcomes
Previous to optimization, Bing’s visible embedding mannequin operated on a GPU server cluster that dealt with inference duties for numerous deep studying providers throughout Microsoft. The unique implementation, utilizing ONNXRuntime with CUDA Execution Supplier, confronted bottlenecks as a consequence of picture decoding processes dealt with by OpenCV. By integrating NVIDIA’s libraries, the pipeline’s throughput elevated from 88 queries per second (QPS) to 452 QPS, showcasing a 5.14x speedup.
These enhancements not solely improved processing pace but additionally lowered the computational load on CPUs by offloading duties to GPUs, thus maximizing energy effectivity. The NVIDIA TensorRT contributed most to the efficiency beneficial properties, whereas the nvImageCodec and CV-CUDA libraries added a further 27% enchancment.
Conclusion
The profitable optimization of Microsoft Bing Visible Search highlights the potential of NVIDIA’s accelerated libraries in enhancing AI-driven functions. The collaboration demonstrates how GPU sources might be successfully utilized to speed up deep studying and picture processing workloads, even when baseline methods already make use of GPU acceleration. These developments pave the best way for extra environment friendly and responsive visible search capabilities, benefiting each customers and repair suppliers.
For extra detailed insights into the optimization course of, go to the unique NVIDIA Technical Weblog.
Picture supply: Shutterstock
