How to Get Started With Ollama: A Practical Guide for Running Local LLMs

If you’ve been exploring AI development lately, you’ve probably noticed a growing shift toward local LLMs — models you can run privately on your own machine without relying on cloud APIs. One of the tools leading this movement is Ollama, a lightweight framework that makes running large language models locally surprisingly simple.

Whether you want to build a chatbot, generate Python code, or experiment with offline AI workflows, Ollama gives you a clean, developer‑friendly way to do it. In this article, we’ll walk through everything you need to get started — from installation to building your first Python integration — using insights from Real Python’s “Get Started With Ollama” training slides.

Why Developers Are Turning to Ollama

Before diving into setup, it’s worth understanding why Ollama is gaining traction:

• Improved privacy — Your data stays on your machine.

• Offline capabilities — Perfect for secure environments or travel.

• Reduced costs — No API bills, no rate limits.

• Lower environmental impact — Local inference avoids massive cloud compute cycles.

If you’re building tools that handle sensitive data or you simply want full control over your AI stack, Ollama is a strong choice.

Prerequisites: What You Need Before Installing

To run Ollama smoothly, make sure your system meets these basic requirements:

• Python 3.8+

• Enough RAM (e.g., 8GB RAM can run a 7B model)

• Adequate disk space for model downloads

• Decent CPU; GPU optional but helpful

• Supported OS installer from Ollama’s website

You can check detailed hardware requirements at the official docs:
https://docs.ollama.com/

Step 1: Install and Run Ollama

Getting Ollama installed is straightforward:

1. Visit https://ollama.com/download

2. Choose the installer for your operating system

3. Run the application

4. Test the CLI with:

bash

ollama -v
ollama

On Linux, you may need to start the service manually:

bash

ollama serve

Once installed, Ollama is ready to pull and run models locally.

Step 2: Download Your First Models

Ollama maintains a model library similar to PyPI — but for LLMs.
Browse it here: https://ollama.com/library

For the examples in the slides, you’ll need:

• llama3.2 (≈2GB)

• codellama (≈3.8GB)

Download them using:

bash

ollama pull llama3.2
ollama pull codellama

Model sizes vary, and larger models require more RAM. As a rule of thumb:
Your RAM (in GB) should exceed the model size (in billions of parameters).

Step 3: Install the Python Ollama Library

To integrate Ollama with Python, install the official library:

bash

python -m pip install ollama

Useful links:

• PyPI: https://pypi.org/project/ollama/

• Source code: https://github.com/ollama/ollama-python

• API docs: https://docs.ollama.com/api/introduction

Step 4: Understanding chat() vs generate()

The Python library exposes two main functions:

1. ollama.chat() — Multi‑turn conversations

Use this when you need:

• Chatbots

• Assistants

• Memory of previous messages

• Tool‑calling workflows

It behaves like a conversational interface with context retention.

2. ollama.generate() — One‑off text generation

Use this for:

• Summaries

• Code generation

• Editing

• Analysis

• Single‑prompt tasks

This is ideal when you don’t need conversation history.

Step 5: Calling Ollama’s Chat Interface

A simple chat example looks like this:

python

import ollama

response = ollama.chat(
    model="llama3.2",
    messages=[
        {"role": "user", "content": "Explain what a neural network is."}
    ]
)

print(response["message"]["content"])

This gives you a multi‑turn conversational AI running entirely on your machine.

Step 6: Building a Chat Application

Once you understand the basics, you can wrap ollama.chat() into a simple loop:

python

import ollama

print("Local AI Chat — powered by Ollama")
while True:
    user_input = input("You: ")
    if user_input.lower() in ["exit", "quit"]:
        break

    response = ollama.chat(
        model="llama3.2",
        messages=[{"role": "user", "content": user_input}]
    )

    print("AI:", response["message"]["content"])

This is the foundation for building:

• CLI chatbots

• Desktop assistants

• Local AI tools

• Offline helpdesk apps

Step 7: Using the Text Generation Interface

For one‑off tasks:

python

import ollama

result = ollama.generate(
    model="codellama",
    prompt="Write a Python function that reverses a string."
)

print(result["response"])

This is perfect for code generation, documentation, or automation scripts.

Step 8: Generating Python Code with Ollama

Codellama excels at producing structured code:

python

code = ollama.generate(
    model="codellama",
    prompt="Generate a FastAPI endpoint that returns a list of users."
)

print(code["response"])

You can pipe this output directly into files, editors, or automation pipelines.

Summary: What You’ve Learned

By now, you should be able to:

• Install and run Ollama locally

• Pull and manage LLM models

• Use Python’s ollama library

• Build chat applications

• Generate text and code offline

Ollama opens the door to private, cost‑free, and highly customizable AI development — perfect for developers who want full control over their workflow.