Inferensi, Deployment & Runtime

OpenVINO

openvinotoolkit/openvino

Open-source toolkit for optimizing and deploying AI inference across edge-to-cloud environments.

Buka repositori resmi ↗ Beranda proyek ↗
★ 10,6KBintang
⑂ 3,3KFork
725Isu terbuka
C++Bahasa
Apache-2.0Lisensi
Q@project.QualityScoreSkor editorial

Ringkasan

OpenVINO converts models from supported training frameworks into a deployable representation, compiles them for a selected device, and runs optimized inference. It covers computer vision, speech recognition, natural-language processing, recommendation, diffusion, and generative AI workloads. The primary repository language is C++, while APIs are available for C++, Python, C, and Node.js.

Fitur utama

  • Inference optimization for computer vision, speech, NLP, generative AI, and other deep-learning workloads
  • Model conversion from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, and JAX/Flax
  • Hugging Face Transformers and Diffusers integration through Optimum Intel
  • Inference on x86 and ARM CPUs, Intel integrated and discrete GPUs, and Intel NPUs
  • C++, Python, C, and Node.js APIs
  • GenAI API for optimized model pipelines
  • Integrations with ONNX Runtime, Keras 3, torch.compile, ExecuTorch, vLLM, LangChain, LlamaIndex, and LLMWare
  • Python and C++ samples plus Python notebook tutorials

Persyaratan, instalasi, dan mulai cepat

Quick Python installation: `pip install -U openvino`. Other distributions are documented at https://docs.openvino.ai/2026/get-started/install-openvino.html. Verify the Python installation with `import openvino as ov` followed by `print(ov.__version__)`. GenAI installation is documented separately at https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-genai.html.

Penggunaan

Quick start: import OpenVINO, load a trained model with its source framework, call `ov.convert_model(...)`, create `ov.Core()`, compile with `core.compile_model(ov_model, 'CPU')`, and invoke the compiled model with prepared input data. A guided first-model example is available at https://docs.openvino.ai/2026/get-started.html. Additional Python and C++ samples are listed at https://docs.openvino.ai/2026/get-started/learn-openvino/openvino-samples.html.

Kompatibilitas model dan kasus penggunaan

The README states support for models originating from PyTorch, TensorFlow, ONNX, TensorFlow Lite, Keras, PaddlePaddle, and JAX/Flax. Transformers and Diffusers models from Hugging Face can be integrated through Optimum Intel. Compatibility depends on supported models, operations, devices, and conversion paths; an exhaustive compatibility matrix is not included in the supplied record.

Catatan lisensi dan risiko

Apache License 2.0 (SPDX: Apache-2.0). The README states that contributions are released under the project's license and copyright terms. License reference: https://api.github.com/licenses/apache-2.0

Rilis dan pemeliharaan

No specific release version or changelog entries are included in the supplied record. Release notes are available at https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html.

GPUStack

gpustack/gpustack

★ 5,4KPython