Nuclio - "Serverless" framework for Real-Time Events and Data Processing
In this document
Translations:
Overview
Nuclio is a high-performance "serverless" framework focused on data, I/O, and compute intensive workloads. It is well integrated with popular data science tools, such as Jupyter and Kubeflow; supports a variety of data and streaming sources; and supports execution over CPUs and GPUs. The Nuclio project began in 2017 and is constantly and rapidly evolving; many start-ups and enterprises are now using Nuclio in production.
You can use Nuclio as a standalone Docker container or on top of an existing Kubernetes cluster; see the deployment instructions in the Nuclio documentation. You can also use Nuclio through a fully managed application service (in the cloud or on-prem) in the Iguazio Data Science Platform.
If you wish to create and manage Nuclio functions through code - for example, from Jupyter Notebook - see the Nuclio Jupyter project, which features a Python package and SDK for creating and deploying Nuclio functions from Jupyter Notebook.
Nuclio is also an integral part of the new open-source MLRun library for data science automation and tracking and of the open-source Kubeflow Pipelines framework for building and deploying portable, scalable ML workflows.
Nuclio is extremely fast: a single function instance can process hundreds of thousands of HTTP requests or data records per second.
This is 10-100 times faster than some other frameworks. To learn more about how Nuclio works, see the Nuclio architecture documentation, read this review of Nuclio vs. AWS Lambda, or watch the Nuclio serverless and AI webinar.
You can find links to additional articles and tutorials on the Nuclio web site.
Nuclio is secure: Nuclio is integrated with Kaniko to allow a secure and production-ready way of building Docker images at run time.
For further questions and support, click to join the Nuclio Slack workspace.
Why another "serverless" project?
None of the existing cloud and open-source serverless solutions addressed all the desired capabilities of a serverless framework:
- Real-time processing with minimal CPU/GPU and I/O overhead and maximum parallelism
- Native integration with a large variety of data sources, triggers, processing models, and ML frameworks
- Stateful functions with data-path acceleration
- Portability across low-power devices, laptops, edge and on-prem clusters, and public clouds
- Open-source but designed for the enterprise (including logging, monitoring, security, and usability)
Nuclio was created to fulfill these requirements. It was intentionally designed as an extendable open-source framework, using a modular and layered approach that supports constant addition of triggers and runtimes, with the hope that many will join the effort of developing new modules, developer tools, and platforms for Nuclio.
Quick-start steps
The simplest way to explore Nuclio is to run its graphical user interface (GUI) of the Nuclio dashboard. All you need to run the dashboard is Docker:
docker run -p 8070:8070 -v /var/run/docker.sock:/var/run/docker.sock --name nuclio-dashboard quay.io/nuclio/dashboard:stable-amd64
Browse to http://localhost:8070, create a project, and add a function. When run outside of an orchestration platform (for example, Kubernetes), the dashboard will simply deploy to the local Docker daemon.
Assuming you are running Nuclio with Docker, as an example, create a project and deploy the pre-existing template "dates (nodejs)".
With docker ps, you should see that the function was deployed in its own container.
You can then invoke your function with curl; (check that the port number is correct by using docker ps or the Nuclio dashboard):
curl -X POST -H "Content-Type: application/text" -d '{"value":2,"unit":"hours"}' http://localhost:37975
This is the beginning of the README. Read the rest at the original. Read the full README ↗
