Running Python Services in Production: Docker, Services and Scheduled Jobs

From script to service

Many useful tools start life as a script on a developer's laptop. Turning them into dependable services means thinking about packaging, configuration, restarts, scheduling and backups. We run several Python services, including extraction APIs, queue workers and web dashboards, and the following practices have served us well.

Package with Docker where it helps

A container image bundles the code, dependencies and runtime, so the service behaves the same on every machine. Good practice includes a multi-stage build to keep the image small, running as a non-root user, and a health check endpoint so the platform can tell if the service is alive. A compose file describes how services and their dependencies run together.

Use a service manager for long-running processes

On servers where containers are not needed, a service manager such as systemd keeps processes running. It starts them at boot, restarts them if they crash, captures their logs and lets you start or stop workers with a single command. Each worker or API can be a separate service, which makes scaling and troubleshooting straightforward.

Configuration belongs in the environment

Keep secrets and settings, such as database addresses, API keys and the choice of database backend, in environment variables or protected files, never in the source code. Separate development and production values, and rotate any credential that might have been exposed. A config module that loads and validates settings at startup catches mistakes early.

Schedule recurring work

Jobs such as a monthly enrichment run or a nightly refresh fit naturally into cron. A small wrapper script that activates the environment, runs the command and writes to a log file is easy to maintain and test by hand. Log every run and alert someone when a scheduled job fails.

Databases: choose one interface, support two backends

For one of our services we hid the database behind a common interface so that the same application can run against Postgres directly or against Supabase, selected by a single setting. This made development, testing and migration easier. A clean boundary around storage pays off when requirements change.

Backups and recovery

Take regular, automatic backups of databases and code, store copies off the server and test restoring them. We keep dated archives of each service so we can roll back or rebuild a server quickly. A backup that has never been restored is only a hope.

Monitoring

• Expose a health endpoint on every API.
• Log in a consistent format and keep the logs somewhere searchable.
• Watch queue sizes, error rates and worker heartbeats.
• Alert on failure, not only on success.
• Review resource usage so you can plan capacity.

Let us help

If your scripts need to become reliable services, we can handle the packaging, deployment and monitoring. See our infrastructure and software services, or contact us.