Alvin Lang
Mar 18, 2025 20:51
NVIDIA introduces the NeMo Curator, a GPU-accelerated streaming pipeline for environment friendly video processing on DGX Cloud, optimizing AI mannequin improvement and decreasing prices.
The appearance of bodily AI has considerably elevated video content material technology, with a single autonomous car producing over 1 TB of video every day, in keeping with NVIDIA. To handle and make the most of this huge knowledge effectively, NVIDIA has launched the NeMo Curator, a GPU-accelerated streaming pipeline out there on the NVIDIA DGX Cloud.
Challenges with Conventional Processing
Conventional batch processing techniques have struggled with the exponential knowledge development, usually resulting in underutilization of GPUs and elevated prices. These techniques accumulate giant knowledge volumes for processing, which may trigger inefficiencies and latency points in AI mannequin improvement.
GPU-Accelerated Streaming Resolution
To handle these challenges, the NeMo Curator introduces a versatile streaming pipeline that leverages GPU acceleration for large-scale video curation. This superior pipeline incorporates auto-scaling and load-balancing strategies to optimize throughput throughout varied levels, maximizing {hardware} utilization and decreasing complete value of possession (TCO).
Optimized Throughput and Useful resource Utilization
The streaming processing method permits for direct piping of intermediate knowledge between levels, decreasing latency and bettering effectivity. By separating CPU-intensive duties from GPU-intensive ones, the system can higher align with the precise capability of obtainable infrastructure, avoiding idle assets and guaranteeing balanced throughput.
Structure and Implementation
The NeMo Curator pipeline, constructed on the Ray framework, is split into a number of levels, from video decoding to embedding computation. Every stage makes use of a pool of Ray actors to course of knowledge in parallel, with the orchestration thread managing enter and output queues to take care of optimum throughput. The system dynamically adjusts the actor pool measurement to accommodate various stage speeds, guaranteeing constant circulate and effectivity.
Efficiency and Future Prospects
In comparison with conventional batch processing, the streaming pipeline achieves a 1.8x speedup, processing one hour of video per GPU in roughly 195 seconds. The NeMo Curator pipeline has proven an 89x efficiency enchancment over baseline, able to processing round 1 million hours of 720p video on 2,000 H100 GPUs in a single day. NVIDIA continues to work with early entry companions to refine the system additional and develop its capabilities.
For extra detailed insights, go to the NVIDIA weblog.
Picture supply: Shutterstock
