NVIDIA’s “Chat With RTX” Is Now ChatRTX, Update Brings New AI Models

NVIDIA’s “Chat With RTX” Is Now ChatRTX, Update Brings New AI Models

Earlier this year, NVIDIA released Chat With RTX, an AI chatbot assistant that runs entirely on the GPU by tapping into the Tensor Cores found in modern RTX GPUs. The premise of this chatbot is simple: you don’t need to send data to the cloud for it to work, which is a great plus for privacy.

Now, Team Green is bringing an update alongside the renaming of the app, which is now shortened to just “ChatRTX”. The latest tech demo brings more large language models (LLMs), including Google’s new model, Gemma, and ChatGLM3 – which supports both English and Chinese languages. It also gains image recognition capabilities via the new Contrastive Language-Image Pre-training (CLIP) model, while OpenAI’s Whisper model is responsible for speech-to-text for those who’d rather speak than type.

The secret sauce that makes ChatRTX work is TensorRT-LLM paired with retrieval-augmented generation (RAG), which is the technical speak for the capability of LLMs to refer back to external sources (such as local files) for any information it generates to the user. This negates the use of a big data center to host information, and it’s why ChatRTX can run directly on any decently performant PC.

You can try it out for yourself – but keep in mind the system requirements: you’ll need Windows 11, at least 16GB of RAM, 35GB of free storage space, and any Ampere/Ada Lovelace-based GPU (such as RTX 30/40 Series) with at least 8GB of VRAM. To download the app, click this blue text.

Pokdepinion: That looks very neat. It’ll be nice if Windows 10 is supported, though.