Nvidia, the tech giant is bringing Artificial intelligence software, TensorRT 8 that is claimed to be twice as powerful and accurate as its predecessors and can cut interference time in half for language queries. TensorRT 8 can be used for search engines, chatbots, and ad recommendations.
Siddharth Sharma, head of the product marketing team for Nvidia’s AI software further informed that TensorRT is being used by companies like American Express, LG Ford, KLA, Verizon, USPS, GE Healthcare and has been downloaded more than 2.5 million times. TensorRT 8 can now deploy the entire Bert-Large, one of the world’s most widely used transformer-based models, in 1.2 milliseconds, he further informed.
Detailing about its features, Sharma said TensorRT 8 can reduce the amount of computing and memory required for running applications to a much significant amount. TensorRT 8 is aiming to lead a new generation of conversational AI applications, Sharma said.
Among its other specifications, Sharma said TensorRT 8 will allow companies to double or triple their model size for improvement in accuracy. He further added that TensorRT 8’s AI interference was made possible through Sparsity and Quantisation.
According to Greg Estates, VP of Developer Programs at Nvidia, citing the worldwide surge in demand for real-time applications that uses AI, TensorRT 8 will enable companies to deliver conversational AI applications to their customers “with a level of quality and responsiveness that was never before possible.” He further added that it can be deployed in embedded or automotive product platforms and hyperscale data centres.
Over 350,000 developers across 27,500 companies have used TensorRT, the company informed.
One of the clients of TensorRT 8, GE Healthcare, said that the software is used in computer vision applications for ultrasounds and the tool was vital in helping doctors and clinical staff to move faster.
According to Erik Steen, chief engineer of Cardiovascular Ultrasound at GE Healthcare, TensorRT 8’s cardiac view recognition algorithm helps in selecting appropriate images for analysis of cardiac wall motion and with its real-time inference capabilities, improves the performance of the view detection algorithm thus shortening their time.