NVIDIA/TensorRT

概览

解决什么问题

UAV edge AI deployments require low-latency, high-throughput inference on power-constrained NVIDIA GPU/Jetson hardware; TensorRT provides the optimization runtime to convert trained models into optimized engines for these targets.

主要用例

• Deploying detection/tracking models (YOLO, DETR variants) onboard
• Accelerating segmentation/classification for navigation
• TensorRT engine generation from ONNX/TorchScript exports

次要用例

• Benchmarking model latency on target hardware
• Layer fusion and kernel auto-tuning for custom operators
• Integration with DeepStream for multi-camera pipelines

何时使用

考虑使用

• Hardware is exclusively NVIDIA (GPU or Jetson)
• Model export to ONNX is feasible
• Latency budget demands kernel-level optimization
• Team can maintain version-locked TensorRT/CUDA/cuDNN stack

采用前需验证

• ? TensorRT version compatibility with JetPack / CUDA driver on target
• ? ONNX opset coverage for model operators (custom plugins may be needed)
• ? INT8 calibration dataset representativeness for quantization accuracy
• ? Memory footprint of optimized engine vs. device RAM/VRAM limits

从这里开始

采用检查清单

• 需验证 TensorRT version compatibility with JetPack / CUDA driver on target
• 需验证 ONNX opset coverage for model operators (custom plugins may be needed)
• 需验证 INT8 calibration dataset representativeness for quantization accuracy
• 需验证 Memory footprint of optimized engine vs. device RAM/VRAM limits

以上检查项只有在官方来源确认后才能标记为“已验证”；无法确认的保持未验证。

已知限制与未知项

已知限制

• NVIDIA hardware only (no AMD/Intel/ARM NPU support)
• Version coupling: TensorRT, CUDA, cuDNN, and JetPack must align
• Custom operator support requires C++ plugin development
• Large model engine build times can be significant on Jetson

替代与相关工具

替代方案

• onnx-runtime — 替代

如何使用？

从下方记录的入口开始，然后对照技术清单进行验证。

• 官方网站 https://developer.nvidia.com/tensorrt
• 仓库 https://github.com/NVIDIA/TensorRT
• 文档 https://developer.nvidia.com/tensorrt

技术清单

• 通过 许可证已识别 已记录: Apache-2.0
• 通过 维护信号 活跃维护
• 通过 验证状态 官方确认
• 通过 已附加来源证据 1 个来源记录
• 需复核 已记录最新版本 未记录

官方链接

元数据与治理

数据集事实

以上事实仅来自官方数据集卡，未确认字段保持未知。

相关资源与依赖

• onnx-runtime — alternative to (confirmed)

近期动态

相关知识

下方相关指南/合集/对比的标题与摘要暂保留英文，待编辑翻译复核；事实字段与链接已验证且保持不变。

指南

• Deploying AI Inference on UAV Edge Compute — A repeatable workflow for choosing and validating an inference runtime for an onboard UAV computer.

合集

• 边缘AI感知入门套件 — 一套用于在边缘部署工作之前对空中感知模型进行原型验证的起步技术栈。