Inferenza, distribuzione e runtime

OpenVINO

openvinotoolkit/openvino

Open-source toolkit for optimizing and deploying AI inference across edge-to-cloud environments.

Apri il repository ufficiale ↗ Pagina iniziale del progetto ↗
★ 10,6KStelle
⑂ 3,3KFork
725Problemi aperti
C++Lingua
Apache-2.0Licenza
Q@project.QualityScorePunteggio editoriale

Panoramica

OpenVINO converts models from supported training frameworks into a deployable representation, compiles them for a selected device, and runs optimized inference. It covers computer vision, speech recognition, natural-language processing, recommendation, diffusion, and generative AI workloads. The primary repository language is C++, while APIs are available for C++, Python, C, and Node.js.

Funzionalità principali

  • Inference optimization for computer vision, speech, NLP, generative AI, and other deep-learning workloads
  • Model conversion from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, and JAX/Flax
  • Hugging Face Transformers and Diffusers integration through Optimum Intel
  • Inference on x86 and ARM CPUs, Intel integrated and discrete GPUs, and Intel NPUs
  • C++, Python, C, and Node.js APIs
  • GenAI API for optimized model pipelines
  • Integrations with ONNX Runtime, Keras 3, torch.compile, ExecuTorch, vLLM, LangChain, LlamaIndex, and LLMWare
  • Python and C++ samples plus Python notebook tutorials

Requisiti, installazione e avvio rapido

Quick Python installation: `pip install -U openvino`. Other distributions are documented at https://docs.openvino.ai/2026/get-started/install-openvino.html. Verify the Python installation with `import openvino as ov` followed by `print(ov.__version__)`. GenAI installation is documented separately at https://docs.openvino.ai/2026/get-started/install-openvino/install-openvino-genai.html.

Utilizzo

Quick start: import OpenVINO, load a trained model with its source framework, call `ov.convert_model(...)`, create `ov.Core()`, compile with `core.compile_model(ov_model, 'CPU')`, and invoke the compiled model with prepared input data. A guided first-model example is available at https://docs.openvino.ai/2026/get-started.html. Additional Python and C++ samples are listed at https://docs.openvino.ai/2026/get-started/learn-openvino/openvino-samples.html.

Compatibilità del modello e casi d'uso

The README states support for models originating from PyTorch, TensorFlow, ONNX, TensorFlow Lite, Keras, PaddlePaddle, and JAX/Flax. Transformers and Diffusers models from Hugging Face can be integrated through Optimum Intel. Compatibility depends on supported models, operations, devices, and conversion paths; an exhaustive compatibility matrix is not included in the supplied record.

Note su licenza e rischi

Apache License 2.0 (SPDX: Apache-2.0). The README states that contributions are released under the project's license and copyright terms. License reference: https://api.github.com/licenses/apache-2.0

Rilascio e manutenzione

No specific release version or changelog entries are included in the supplied record. Release notes are available at https://docs.openvino.ai/2026/about-openvino/release-notes-openvino.html.

GPUStack

gpustack/gpustack

★ 5,4KPython