AI News Feed
Market watch
LG UltraFine Evo 32U990A-S Hits Record-Low $1,099 on Amazon, TechRadar ReportsNewegg Labor Day Deal: 32GB Kingston Fury Beast DDR4-3200 Kit Nearly Half Price at $179.99 with CodeGalaxy Z Fold 8 inner screen corners reported with excessive giveHuawei unveils Mate XT 2 tri-fold with privacy-screen edition as Apple prepares folding iPhoneWall Street Analysts Recommend Nvidia, Uber and Marvell for Long-Term GrowthLenovo, AMD and Huawei Put Local Large-Model AI at Center of New Devices
LG UltraFine Evo 32U990A-S Hits Record-Low $1,099 on Amazon, TechRadar ReportsNewegg Labor Day Deal: 32GB Kingston Fury Beast DDR4-3200 Kit Nearly Half Price at $179.99 with CodeGalaxy Z Fold 8 inner screen corners reported with excessive giveHuawei unveils Mate XT 2 tri-fold with privacy-screen edition as Apple prepares folding iPhoneWall Street Analysts Recommend Nvidia, Uber and Marvell for Long-Term GrowthLenovo, AMD and Huawei Put Local Large-Model AI at Center of New Devices
Products & Applications

NVIDIA Releases TensorRT Model Connect in Public Preview, Converting Hugging Face Checkpoints to C++ Inference in Two Commands

NVIDIA has released TensorRT Model Connect in public preview, an open-source tool that converts Hugging Face checkpoints to native C++ TensorRT inference in two commands, without an ONNX step.

AI News Feed · 2026-08-19 08:03

The project is Apache-2.0 licensed and is delivered as a collection of family-owned reference implementations rather than a single generic converter. NVIDIA stated that the entire project, including model implementations, performance tuning, tests, integrations, and documentation, was built using OpenAI Codex agents under human direction and review. The bundle design splits build and runtime at a versioned artifact: Python handles checkpoint resolution and TensorRT engine construction, while native profiles execute inference in C++ without PyTorch. A small number of hybrid profiles invoke a helper Python executable, with their manifests explicitly declaring that dependency.

Applications call task APIs such as generate(), transcribe(), generate_image(), embed(), and solve(), instead of maintaining conversion stages and per-model application glue. The trtmc inspect command exposes bundle kind, model family, precision, runtime identity, and engines, making the artifact auditable. The conventional route of PyTorch to ONNX or TorchScript to TensorRT to model-specific C++ integration is replaced, removing export gaps, repeated per-model integration, and validation spread across conversion artifacts.

The quick start example builds and runs Qwen3-0.6B with two commands: trtmc build Qwen/Qwen3-0.6B --precision bf16 --max-cache-length 16384 --output qwen3-0.6b.bundle, then trtmc run ./qwen3-0.6b.bundle --prompt "What is the capital of France? Answer in one word." --chat-template --no-thinking. The same bundle loads from C++ with trtmc::load("./qwen3-0.6b.bundle").

Release wheels currently target Linux aarch64 only, with Python 3.10 or 3.12, glibc 2.39 or newer, and TensorRT 11.1.0.106. x86_64 wheels are not published, so x86_64 users must use the Docker source-build path. MarkTechPost notes that the July 29, 2026 GB300 snapshot covers 105 profiles across 76 families, and 102 beat their declared reference by more than 5 percent. The project is best suited for teams that already own their inference stack, including NVIDIA-focused startups, robotics and device companies, and platform teams in mid-size and large enterprises. Small teams shipping a Python service would get less from it, and regulated enterprises should wait for a tagged release before standardizing on it. Use cases include on-device text generation, speech recognition and synthesis, OCR, document parsing, embeddings, reranking for retrieval services written in C++, diffusion image and video generation, segmentation, and time-series forecasting.