Format

1. 1. Intro

   1. Team – experience
   2. Ecosystem
      • Statistics - interest in rust & AI
      • Motivating for edge AI
      • Workshop overview

2. 2. Lecture: Deep Learning for Computer Vision

   1. Introduce WasmEdge / Rust for edge devices
      1. High level overview of WebAssembly and the WebAssembly Component Model
      2. What is WasmEdge?
      3. An example of a WasmEdge Rust application
   2. Lecture on Computer Vision
      1. High-level introduction to computer vision tasks / problems: classification, object detection, segmentation
      2. What exactly is a neural network? High-level introduction to CNNs.
      3. What exactly is an (image) embedding?
      4. Common libraries in the ecosystem (from python: PyTorch, 🤗HuggingFace, etc)
   3. Introduce Rust libraries for AI
      1. Compare the use of Rust in deep learning compared to Python, showcase equivalent libraries
      2. Candle - Lightweight minimalist ML framework for Rust that allows the use of 🤗HuggingFace models
      3. Mediapipe-rs - for on edge Computer Vision
      4. WasmEdge - running LlamaEdge for deploying LLMs
   4. HuggingFace (presented by HuggingFace)
      1. Platform
         1. resources, search, selection
         2. Deployment
         3. Rust & HuggingFace - Candle

3. 3. Hands-on: Air-gapped face recognition on Pi

   Use pretrained models from mediapipe-rs and HuggingFace🤗 to build a simple Face Authentication pipeline

   1. Stream video input from webcam to Raspberry Pi server
   2. Deploy a Face Detection model on the Raspberry Pi
   3. Deploy an Face Embedding model on the Raspberry Pi
   4. Save identities to a vector database like QDrant Rust SDK
   5. Perform privacy-preserving on-device authentication

4. 4. Lecture: Deep Learning for NLP

   1. Introduction to Deep Learning for NLP
      1. Overview of the history of NLP methods
      2. Development of Transformers
   2. How does an LLM work? Tokenizers, pre-training, post-training.
   3. High-level introduction to concepts about prompt engineering: Chain-of-Thought, RAG, In-Context Learning
   4. What exactly is a (text) embedding?
   5. Introduce Rust libraries for NLP – Rust is actively used for developing the ecosystem around training LLMs (e.g., tokenizers)
      1. tokenizers - Rust - A Fast version of the tokenizers library used ubiquitously in the LLM ecosystem
      2. llama.cpp

5. 5. Hands-on: Chat with a LLM on Pi

   1. Deploy a small, pretrained LLM on a Raspberry Pi
   2. Chat with the LLM
   3. Stream responses from the server (async)
   4. Build a simple RAG pipeline
      1. Embed some (given) texts using a text embedding model (i.e., BERT)
      2. Store texts & embeddings into QDrant vector store
      3. LLM can retrieve relevant passages from the vector store to answer questions
      4. Agentic RAG – Before retrieving passages, use the LLM to summarize passages/select the most important bits

6. 6. Hands-on: Integration

   Integrate both computer vision models and LLMs into a single application.

   1. Develop an application that can register a user using Face Authentication
   2. Adapt the deployed LLM to handle tool calling using system prompts
   3. Users can input a free-text description of their preferences, which the LLM parses into a predefined set
   4. Finally, users can be identified from their faces and can issue commands to the LLM assistant, and the assistant can respond based on their declared preferences.