Skip to main content

Decentralized AI overview

Intermediate
Concept
DeAI

Decentralized AI, or DeAI for short, refers to the intersection between AI and blockchain technology. The term denotes a wide range of applications, starting from peripheral elements such as tokenization or decentralized marketplaces, and going all the way to running AI models fully on-chain as smart contracts.

What is DeAI?​

The term "DeAI" may be used to refer to or describe a variety of different things. In some cases, the term is diluted and used by platforms and applications that are neither decentralized nor trustworthy. In other contexts, such as the context of ICP, DeAI refers to artificial intelligence running on a fully decentralized and secure platform.

Here is a list of various DeAI applications ordered from the strongest to weakest form of decentralization:

  1. Fully on-chain training and inference: This is the truest form of DeAI that brings the security and trustworthiness guarantees of smart contracts to DeAI applications. Supporting this use case is the long-term vision of ICP that will become possible after integrating GPU-enabled nodes.

  2. Fully on-chain inference using models trained off-chain: This assumes that the model was trained off-chain and then uploaded on-chain. Inference happens fully on-chain and has the same security and trustworthiness guarantees as regular smart contracts. ICP currently supports this use case for models with millions of parameters. See an example of on-chain image classification.

  3. Storing the model on-chain and running inference on a user's device: This assumes that the model was trained off-chain and then uploaded on-chain. Inference happens on the user's device after downloading the model. If the user trusts their own device, then they can trust that the inference ran correctly. A disadvantage here is that the model needs to be downloaded to the user's device with corresponding drawbacks of less confidentiality of the model and decreased user experience due to increased latency. ICP supports this use case for practically all existing models because a smart contract on ICP can store models up to 400GiB. See [an example]((https://github.com/patnorris/DecentralizedAIonIC) of an in-browser AI chatbot that uses an open-source LLM model served from ICP.

  4. Tokenization, marketplaces, orchestration: This refers to using smart contracts as the tokenization, marketplace, and orchestration layer for AI models and AI hardware. Since ICP is a general-purpose blockchain, it supports such smart contracts of arbitrary complexity.

  5. Smart contracts calling Web2 AI services: Smart contracts running on ICP can make HTTP requests to Web2 services including OpenAI and Claude. See an example of a smart contract calling the OpenAI API.

Why use DeAI?​

DeAI in its truest form can potentially solve AI's trust problem. Today, users have to blindly trust AI running on centralized servers. They have no visibility into how their data is used, how AI models produce responses, or whether they work correctly, reliably, and consistently.

Since AI models behave like black boxes to users, building trustworthy AI models is a difficult challenge. This can be addressed if users are able to verify how a model has been trained and that the inference process uses that very model to generate an output.

Trustworthy DeAI is possible on ICP using canister smart contracts.

Fully on-chain DeAI​

Running AI models on-chain is too compute and memory-intensive for traditional blockchains. ICP was designed to make smart contracts powerful by leveraging the following features:

  1. The WebAssembly virtual machine that provides near-native performance.
  2. Deterministic time slicing that automatically splits long-running computation over multiple blocks.
  3. Powerful node hardware with a standardized specification. Nodes have 32-core CPUs, 512GiB RAM, and 30TB NVMe.

Currently, ICP supports on-chain inference of small models using AI libraries such as Sonos Tract that compile to WebAssembly. Check out the image classification example to learn how it works. The long-term vision of DeAI on ICP is to support on-chain GPU compute to enable both training and inference of larger models.

Technical working group: DeAI​

A technical working group dedicated to discussing decentralized AI and related projects meets bi-weekly on Thursdays at 5pm UTC. You can join via the community Discord server.

You can learn more about the group, review the notes from previous meetings, and ask questions on the DFINITY forum.

ICP AI projects​

Several community projects that showcase how to use AI on ICP are available.

On-chain inference frameworks​

  • Sonos Tract: An open-source AI inference engine written in Rust that supports ONNX, TensorFlow, PyTorch models and compiles to WebAssembly. The image classification example explains how to integrate it into a canister to run on ICP.
  • Rust-Connect-Py-AI-to-IC: Open-source tool for deploying and running Python AI models on-chain using Sonos Tract.
  • Burn: An open-source deep learning framework written in Rust that supports ONNX, PyTorch models and compiles to WebAssembly. The MNIST example explains how to integrate it into a canister to run on ICP. Try it here.
  • Candle: a minimalist ML framework for Rust that compiles to WebAssembly. An AI chatbot example shows how to run a Qwen 0.5B model in a canister on ICP.

Vector databases​

  • Vectune: Vectune is a lightweight VectorDB with incremental indexing, based on FreshVamana written in Rust. See a forum post that explains how it works.
  • ArcMind Vector DB: A vector database that supports text, image, and audio embedding.

Language models, agents, and chatbots​

Calling OpenAI from a canister​

  • Juno + OpenAI: An example using Juno and OpenAI to generate images from prompts.

Programming language specific resources​