openvino

• Inference Optimization: Boost deep learning performance in computer vision, automatic speech recognition, generative AI, natural language processing with large and small language models, and many other common tasks.
• Flexible Model Support: Use models trained with popular frameworks such as PyTorch, TensorFlow, ONNX, Keras, PaddlePaddle, and JAX/Flax. Directly integrate models built with transformers and diffusers from the Hugging Face Hub using Optimum Intel. Convert and deploy models without original frameworks.
• Broad Platform Compatibility: Reduce resource demands and efficiently deploy on a range of platforms from edge to cloud. OpenVINO™ supports inference on CPU (x86, ARM), GPU (Intel integrated & discrete GPU) and AI accelerators (Intel NPU).
• Community and Ecosystem: Join an active community contributing to the enhancement of deep learning performance across various domains.

Check out the OpenVINO Cheat Sheet and Key Features for a quick reference.

Installation

Get your preferred distribution of OpenVINO or use this command for quick installation:

pip install -U openvino

Verify Installation

After installation, you can verify OpenVINO is installed correctly by running:

import openvino as ov
print(ov.__version__)

Check system requirements and supported devices for detailed information.

Tutorials and Examples

OpenVINO Quickstart example will walk you through the basics of deploying your first model.

Learn how to optimize and deploy popular models with the OpenVINO Notebooks 📚:

• Create an LLM-powered Chatbot using OpenVINO
• YOLOv11 Optimization
• Text-to-Image Generation
• Multimodal assistant with LLaVa and OpenVINO
• Automatic speech recognition using Whisper and OpenVINO

Discover more examples in the OpenVINO Samples (Python & C++) and Notebooks (Python).

Here are easy-to-follow code examples demonstrating how to run PyTorch and TensorFlow model inference using OpenVINO:

PyTorch Model

import openvino as ov
import torch
import torchvision

# load PyTorch model into memory
model = torch.hub.load("pytorch/vision", "shufflenet_v2_x1_0", weights="DEFAULT")

# convert the model into OpenVINO model
example = torch.randn(1, 3, 224, 224)
ov_model = ov.convert_model(model, example_input=(example,))

# compile the model for CPU device
core = ov.Core()
compiled_model = core.compile_model(ov_model, 'CPU')

# infer the model on random data
output = compiled_model({0: example.numpy()})

TensorFlow Model

import numpy as np
import openvino as ov
import tensorflow as tf

# load TensorFlow model into memory
model = tf.keras.applications.MobileNetV2(weights='imagenet')

# convert the model into OpenVINO model
ov_model = ov.convert_model(model)

# compile the model for CPU device
core = ov.Core()
compiled_model = core.compile_model(ov_model, 'CPU')

# infer the model on random data
data = np.random.rand(1, 224, 224, 3)
output = compiled_model({0: data})

OpenVINO supports the CPU, GPU, and NPU devices and works with models from PyTorch, TensorFlow, ONNX, TensorFlow Lite, PaddlePaddle, and JAX/Flax frameworks. It includes APIs in C++, Python, C, NodeJS, and offers the GenAI API for optimized model pipelines and performance.

Generative AI with OpenVINO

Get started with the OpenVINO GenAI installation and refer to the detailed guide to explore the capabilities of Generative AI using OpenVINO.

This is the beginning of the README. Read the rest at the original. Read the full README ↗