You have files — documents, code, spreadsheets, notes — that you want AI to help with. But you do not want to upload them to ChatGPT, Claude, or any other cloud service.
With a local AI model, you can process those files on your own machine. This guide covers the practical setup and common file-processing tasks you can do locally.
Quick answer
Run a local model with Ollama, then write a simple script that reads your files, sends them to the local model with a specific prompt, and saves the results. Common tasks include summarisation, extraction, classification, and search.
- You have sensitive files that cannot be uploaded to cloud AI services.
- You want to batch-process files without per-token API costs.
- You need AI assistance with files while working offline.
Setting up for file processing
Install Ollama and pull a model (Gemma 4 12B is a good balance of quality and speed). The model runs as a local service you can call from any script.
For file processing, you mainly interact through the API rather than the chat interface — you need to programmatically read files, send them to the model, and collect results.
Common file tasks
The most useful local AI file tasks fall into a few categories: understanding what files contain, extracting specific information, transforming content, and finding patterns across multiple files.
- Summarise documents — get key points from long reports or articles
- Extract structured data — pull names, dates, amounts from unstructured text
- Classify files — tag documents by topic, priority, or category
- Search semantically — find files related to a concept, not just a keyword
- Generate descriptions — create metadata or alt text for files
Handling different file types
Text files work directly. For PDFs, use a library like PyMuPDF or pdfplumber to extract text first. For Word documents, use python-docx. For spreadsheets, use openpyxl or pandas.
The pattern is always the same: extract the text content, send it to the model with your prompt, and process the response.
import fitz # PyMuPDF
import requests
def process_pdf(pdf_path: str, prompt: str) -> str:
# Extract text from PDF
doc = fitz.open(pdf_path)
text = "\n".join(page.get_text() for page in doc)
# Send to local model
response = requests.post("http://localhost:11434/api/generate",
json={"model": "gemma3", "prompt": f"{prompt}\n\n{text}",
"stream": False})
return response.json()["response"]Batch processing patterns
For processing many files, use a simple loop with progress tracking. Save results to a CSV or JSON file so you do not lose work if the script is interrupted.
Consider processing files in parallel if your hardware can handle it — but watch memory usage with larger models.
Working with large files
Local models have context window limits. If a file is too large to fit in a single prompt, split it into chunks and process each chunk separately. Then use a final prompt to combine the chunk results.
For very large documents, consider extracting only the relevant sections (headings, specific pages, key paragraphs) rather than sending the entire content.
Worked example: processing a folder of meeting notes
You have 30 meeting note files in a folder. A script sends each one to local Gemma 4 with the prompt 'Extract all action items with assigned owners and deadlines.' The script saves the results to a CSV. Total time: 10 minutes. Total data uploaded to the cloud: zero.
Common mistakes
- Sending files larger than the model's context window without splitting.
- Not saving intermediate results — if the script crashes halfway through, you lose everything.
- Using the chat interface instead of the API for batch work.
When to use something else
If you want to build a full RAG system over your documents, see building a RAG app. For setting up the local model itself, see running Gemma 4 on your own machine.
Frequently asked questions
How do I run AI on my own files locally?
Run a local model with Ollama, then script it: read each file, send it to the model with a task prompt, and save the result. Common tasks are summarising, extracting, classifying and searching.
What do I need to set up?
Install Ollama and pull a model (Gemma 4 12B is a good quality-and-speed balance). It runs as a local service you call from any script — no API key, and nothing leaves your machine.
How do I handle PDFs, Word docs and spreadsheets?
Extract text first: PyMuPDF or pdfplumber for PDFs, python-docx for Word, openpyxl or pandas for spreadsheets. Plain text files you can feed in directly.
What tasks is local AI good for on files?
Understanding what files contain, pulling out specific fields, transforming content, and finding patterns across many files — all without sending data to a cloud API.
Why choose local over a cloud API here?
Privacy and cost: sensitive files never leave your machine and there is no per-token bill, at the cost of some quality and your own hardware setup. Ideal for bulk, private document work.
How do I process a large folder reliably?
Loop over the files, write each result to disk as you go, and wrap each call in error handling so one bad file does not kill the batch. Keep a record of processed files so you can resume.
Related guides on this site
These guides cover local model setup, RAG systems, and private AI patterns.