AI tasks that take more than a few seconds — document processing, multi-step agent workflows, batch analysis — need background jobs. Blocking a request for 30 seconds while the model thinks is a terrible user experience.
This guide covers practical patterns for moving AI work to background queues, tracking progress, and delivering results.
Quick answer
Submit long AI tasks to a job queue, return a job ID immediately, process the task in a background worker, and let the client poll or subscribe for results. This pattern handles timeouts, retries, and progress tracking cleanly.
- AI tasks take more than 5-10 seconds to complete.
- You need to process multiple documents or run multi-step agent workflows.
- Users need progress updates while the AI is working.
When to use background jobs
If the AI task can finish in under 3 seconds, handle it inline. If it takes 3-10 seconds, consider streaming the response. If it takes more than 10 seconds, use a background job.
Multi-step agent tasks, document batch processing, and large-context analysis are the most common candidates for background jobs.
Job queue design
Use a standard job queue (Redis Queue, Celery, BullMQ, or cloud-native options like AWS SQS). Submit the AI task with all needed context — the prompt, model parameters, and any file references.
Keep the job payload self-contained. The worker should not need to call back to the main application to get the information it needs to run the task.
- Submit jobs with a unique ID and all required context
- Set reasonable timeouts (AI tasks can hang on rate limits or network issues)
- Include retry logic with exponential backoff
- Store job status and results in a persistent store
Progress tracking
For multi-step tasks, update progress as each step completes. This could be as simple as 'Step 3 of 7: Analysing document' or as detailed as per-step results.
Store progress in a shared state (Redis, database) that the client can query. WebSocket or SSE connections work for real-time updates.
Result delivery
The simplest pattern is polling — the client checks the job status every few seconds. For better UX, use webhooks or real-time connections to push results when ready.
Store results with the job so the client can retrieve them at any time, not just when the job finishes.
Error handling and retries
AI API calls can fail for many reasons: rate limits, network errors, context length exceeded, content policy violations. Your background worker needs to handle each case differently.
Rate limits should trigger a retry with backoff. Content policy violations should not be retried. Network errors get a limited number of retries. Always store the error with the job so the user knows what happened.
Worked example: batch document analysis
A user uploads 50 documents for analysis. The app creates a background job for each document, tracks progress ('Analysed 23 of 50 documents'), and delivers results as they complete. The user can close the browser and come back later — results are stored with the job.
Common mistakes
- Blocking HTTP requests for long AI tasks.
- Not setting timeouts on AI API calls in workers.
- Retrying content policy violations (they will fail again).
When to use something else
If the task is short enough for streaming, see reasoning summaries in production AI. For reducing the cost of background AI jobs, see cutting AI API costs.
Frequently asked questions
When do I need a background job for an AI task?
A rough rule: under three seconds, handle it inline; three to ten seconds, stream the response; over ten seconds, use a background job. Long tasks blow past HTTP timeouts and need async handling.
What is the basic pattern?
Submit the task to a queue, return a job ID immediately, process it in a worker, and let the client poll or subscribe for the result. This cleanly handles timeouts, retries and progress.
Which job queue should I use?
A standard one: Redis Queue, Celery or BullMQ if self-hosting, or SQS and similar if you are on a cloud. Submit the full context — prompt, parameters, file references — with the job.
How do I show progress on a long task?
Update a progress field per step, such as 'Step 3 of 7: analysing document', that the client polls, or push updates over websockets or SSE. Even coarse progress greatly improves perceived reliability.
How do I handle failures and retries?
Make jobs idempotent, set max retries with backoff, and record a terminal failed state with an error the client can show. Do not silently drop a job the user is waiting on.
How does the client get the result?
It either polls the job-status endpoint until done, or subscribes via websocket or SSE for a push. Polling is simplest; push is nicer for very long jobs.
Related guides on this site
These guides cover related patterns for building production AI applications.