Beginner Python Programming

Goal

In this module, you will learn how to interact with the Llama Stack server using Python. You’ll set up your environment, install the necessary SDK, and run a Python script to communicate with the Llama Stack server.

Prerequisites

Llama Stack server running (see: Llama-stack Helloworld)
Python 3.10+ installed on your local machine
Python virtual environment setup and activated (see: Llama-stack Helloworld)

Step 1: Install the Llama Stack Client SDK

Install the Llama Stack client SDK using pip:

pip install llama-stack-client==0.2.2

Step 2: Create a Python Script to ChatCompletionResponse with Llama Stack

Create a Python script named llama_chat.py:

cat << 'EOF' > llama_chat.py
from llama_stack_client import LlamaStackClient

# Initialize the client
client = LlamaStackClient(base_url="http://localhost:8321")

# Define the model ID
model_id = "meta-llama/Llama-3.2-3B-Instruct"

while True:
    prompt = input("You: ")
    if prompt.lower() in {"exit", "quit"}:
        break
    response = client.inference.chat_completion(
        messages=[{"role": "user", "content": prompt}],
        model_id=model_id
    )
    print("Llama:", response.completion_message.content)
EOF

This script initializes the Llama Stack client, and allows you to chat with the model. Enter "exit" or "quit" to stop the application.

Step 3: Run the Python Script

Execute the script:

python llama_chat.py

You should see a prompt:

You:

You can now have an interactive chat with the model!

Summary

In this module, you:

Set up a Python environment and installed the Llama Stack client SDK
Created and executed a Python script to chat with the Llama Stack server

This foundational knowledge enables you to build more complex applications leveraging the Llama Stack’s capabilities.