Async Tasks

IDK uses Celery workers and Celery Beat (scheduler) for asynchronous and scheduled jobs. Tasks run out-of-band to keep web requests fast and to process background work reliably.

How it works

  • Broker/Backend: configured in Django settings (Redis by default)
  • Workers: execute @shared_task and @app.task functions
  • Beat: schedules periodic tasks (e.g., syncs, cleanups)
  • Notifications: some tasks integrate with the Notifications app to emit real-time progress and completion events

Running locally

  • Start services with Docker compose (Redis, workers, beat)
  • Trigger tasks via Django views, management commands, or the shell

Task catalog by app

Below is a non-exhaustive list of tasks grouped by app.

Core (idk/core/tasks.py)

  • app.task: core maintenance tasks
  • precompute_dashboard_data (shared_task): precomputes dashboard data

Services (idk/services/tasks.py and idk/services/tasks_quickbooks.py)

  • Syncing and integration tasks for ClickUp, Toggl, QuickBooks, Front, Jira, Tempo
  • Mixed @shared_task and @app.task definitions, including bulk syncs, data normalization, and cache rebuilds
  • QuickBooks tasks: invoice number sync, accounting entity syncs

Reports (idk/reports/tasks.py)

  • Report generation: build and cache reports (contract/monthly and related jobs)
  • Scheduled report preparation helpers

Scheduled Reports (idk/scheduled_reports/tasks.py)

  • Periodic scheduling and delivery of reports (email/chat)

Alerts (idk/alerts/tasks.py)

  • Alert processing and notifications

Invoicing (idk/invoicing/tasks.py)

  • process_payment_webhook (shared_task): process incoming payment webhooks

Team Members (idk/team_members/tasks.py)

  • Member sync/update utilities (shared_task, app.task)

Notifications (idk/notifications/celery_integration.py)

  • Decorators and base task (NotifyingTask) to emit progress/events for any Celery task

Adding a new task

  1. Define a task using @shared_task (prefer) or @app.task when you need custom base/queues
  2. Keep the task idempotent; handle retries and rate limits for external calls
  3. If long‑running, emit progress via the Notifications integration
  4. Add periodic entries to Celery Beat if it should be scheduled
  5. Cover with tests; use fakes/mocks for external APIs