<<<<<<< HEAD

Task Queue System

A distributed task queue system built with Python, FastAPI, and Redis.

Features

• Priority-based task scheduling
• Exponential backoff retries
• Dead-letter queue (DLQ)
• Live metrics and health endpoints
• Redis-backed queue, storage, and DLQ
• Optional in-memory fallback for development
• Worker daemon for background processing

Requirements

• Python 3.12+
• Redis 7+
• Docker / Docker Compose (optional)

Install

pip install -r requirements.txt

Local development

Start Redis

Option 1: Docker

docker compose up -d redis

Option 2: Local Redis

Start Redis locally on localhost:6379.

Start the API server

uvicorn api.main:app --reload --port 8000

Start the worker daemon

python worker_daemon.py

Verify

Open:

• http://localhost:8000/docs
• http://localhost:8000/health/live

If Redis is available, startup logs should show:

• ✓ Connected to Redis at redis://localhost:6379/0

Docker

Run the full stack with Redis, API, and worker:

docker compose up --build

Project structure

• api/ — FastAPI application and routes
• core/ — queue, dispatcher, scheduler, task, and worker logic
• plugins/ — storage, Redis backend, middleware, and extensions
• monitoring/ — metrics collection
• worker_daemon.py — standalone worker process
• ui/ — dashboard assets
• tests/ — unit and integration tests

Redis configuration

The application uses REDIS_URL from the environment, defaulting to:

redis://localhost:6379/0

You can override it before starting the app:

export REDIS_URL=redis://localhost:6379/0

On Windows PowerShell:

$env:REDIS_URL = "redis://localhost:6379/0"

Running tests

pytest

GitHub deployment

This workspace is not currently a Git repository, so I cannot push to GitHub from here.

To deploy to GitHub:

git init
git add README.md .
git commit -m "Add README"
git remote add origin <your-github-repo-url>
git branch -M main
git push -u origin main

Replace <your-github-repo-url> with the GitHub repository URL.