Debugging Container Timezone and Locale Inconsistencies Across Multi-region Deployments: Synchronizing System Settings for Distributed Logging and Scheduled Tasks

Why I Started Tracking This Problem

I run containers across multiple Proxmox hosts in different physical locations. Some are at home, some on a VPS in Singapore, and I've tested deployments in Europe for redundancy. Each location has its own system timezone, and I learned the hard way that containers don't automatically inherit sensible time settings.

The problem surfaced when I set up Cronicle for scheduled tasks. Jobs that should have run at 2 AM local time were firing at random hours. Logs from my n8n workflows showed timestamps that didn't match what I saw in my monitoring dashboard. When I tried to correlate events across regions, nothing lined up. I was essentially flying blind.

This wasn't an academic exercise. I needed distributed logging that made sense and cron jobs that actually ran when expected.

What I Was Actually Running

My setup at the time:

• Proxmox hosts in Asia/Singapore (GMT+8) and Europe/London (GMT+0)
• Docker containers running n8n, Cronicle, and custom Node.js services
• A Synology NAS collecting logs via syslog
• Some containers built from Alpine base images, others from Debian

I assumed that setting the host timezone would be enough. It wasn't. Each container was essentially making its own decision about what time it was, and those decisions were inconsistent.

The First Signs of Trouble

I noticed the issue when checking logs for a failed n8n workflow. I ran:

docker logs -t n8n-container

The output showed Docker's timestamp in UTC, but the application log inside had a different timestamp—also in UTC, even though I'd set TZ=Asia/Singapore in my docker-compose file. The container didn't care.

When I exec'd into the container and ran date, it returned UTC. The TZ variable was set, but something wasn't applying it. Turns out Alpine-based images need the tzdata package installed, or the TZ variable does nothing. Debian images usually have it, but not always in minimal variants.

Debugging the Actual Configuration

I started by checking what each container actually thought the time was:

docker exec n8n-container date
docker exec cronicle-container date

Different answers. One showed GMT+8, another showed UTC, a third showed GMT+0 but formatted differently. None of this was intentional—it was just whatever the base image decided.

Then I checked the host:

timedatectl

The Proxmox host was correctly set to Asia/Singapore. But containers don't inherit that unless you explicitly pass it through.

The Locale Problem I Didn't Expect

While digging into timezone issues, I found that some containers were also using different locales. One container defaulted to POSIX, another to en_US.UTF-8. This caused date formatting inconsistencies in logs—some used 24-hour time, others 12-hour with AM/PM. When aggregating logs, this made parsing a nightmare.

I checked with:

docker exec cronicle-container locale

The output showed LANG=C, which explained why date strings looked different than expected.

What Actually Worked

Standardizing on UTC Everywhere

My first instinct was to force every container to use the local timezone. I tried mounting /etc/localtime and /etc/timezone from the host:

volumes:
  - /etc/localtime:/etc/localtime:ro
  - /etc/timezone:/etc/timezone:ro

This worked on Debian-based containers but failed silently on Alpine. Alpine doesn't use /etc/timezone the same way.

After fighting with this across multiple base images and regions, I made a different decision: keep everything in UTC. Containers log in UTC, Docker timestamps are UTC, and I convert to local time only when I'm actually reading logs.

This removed the entire class of problems. No more wondering if a timestamp was local or UTC. No more cron jobs firing at the wrong hour because the container thought it was in a different timezone.

Setting TZ Explicitly When Needed

For containers that absolutely needed local time (like Cronicle, which displays schedules in a web UI), I set the TZ variable and ensured tzdata was installed:

environment:
  - TZ=Asia/Singapore

In the Dockerfile for Alpine-based images:

RUN apk add --no-cache tzdata
ENV TZ=Asia/Singapore

For Debian:

RUN apt-get update && apt-get install -y tzdata
ENV TZ=Asia/Singapore

I verified each one by exec'ing in and running date. If it didn't show the right timezone, the configuration wasn't working.

Handling Locale Consistently

I added locale settings to containers that generated user-facing output:

environment:
  - LANG=en_US.UTF-8
  - LC_ALL=en_US.UTF-8

For Alpine, I had to install the musl-locales package (or accept that full locale support isn't really there). For Debian, I ran:

RUN apt-get update && apt-get install -y locales && \
    locale-gen en_US.UTF-8

This made date formatting consistent across containers, which mattered when I was parsing logs programmatically.

The Logging Aggregation Reality

I send logs from all containers to my Synology NAS using Docker's syslog driver:

logging:
  driver: syslog
  options:
    syslog-address: "tcp://synology-ip:514"
    tag: "{{.Name}}"

The Synology receives logs with UTC timestamps. I don't try to convert them on the NAS—I leave them as-is. When I search logs, I do the timezone math in my head or use a script to convert.

For quick checks, I wrote a small shell function:

utc_to_local() {
  date -d "$1 UTC" "+%Y-%m-%d %H:%M:%S %Z"
}

I can pipe a UTC timestamp through it and get local time. It's not elegant, but it works.

Scheduled Tasks Across Regions

Cronicle was the biggest pain point. It runs scheduled jobs, and those schedules are defined in the web UI using whatever timezone the container thinks it's in.

I set Cronicle's container to Asia/Singapore explicitly, so when I schedule a job for "2:00 AM," it means 2 AM Singapore time. But I also run a Cronicle instance in Europe for redundancy. That one is set to Europe/London.

To keep things sane, I document every scheduled job with its timezone in the job description. If a job needs to run at the same absolute time across regions, I calculate the offset manually and set different schedules.

It's not automated. It's manual and error-prone. But it's also explicit, which I prefer over some abstraction that might break.

What Didn't Work

Assuming TZ Would Just Work

Setting TZ=Asia/Singapore in docker-compose and assuming the container would respect it was my first mistake. Alpine images ignore it without tzdata. Even with tzdata, some applications (especially those written in Go) use their own timezone database and ignore the system setting entirely.

Mounting Host Timezone Files Everywhere

I tried mounting /etc/localtime into every container as a blanket solution. This broke on Alpine, caused confusion when I moved containers between hosts with different timezones, and made my docker-compose files harder to read.

It also didn't solve the problem of applications that log in UTC regardless of system timezone.

Trying to Parse Mixed-Timezone Logs

Before I standardized on UTC, I had logs coming in with a mix of UTC, GMT+8, and local timestamps without clear indicators. I wrote a script to try to detect and normalize them. It was fragile and failed constantly.

The script made assumptions about timestamp formats that broke when an application changed its logging library. I deleted it and went back to manual conversion.

Key Takeaways

UTC everywhere is simpler than trying to match local timezones. I spent days trying to make every container use the local timezone before realizing that UTC is the path of least resistance. Convert to local time when you read logs, not when you write them.

TZ alone doesn't work on Alpine without tzdata. If you're using Alpine base images, you must install tzdata explicitly. This isn't documented in most Docker tutorials.

Docker's log timestamps are always UTC. The -t flag on docker logs adds UTC timestamps regardless of container or host timezone. Don't fight this—accept it and convert when needed.

Locale matters for log parsing. If you're aggregating logs from multiple containers, inconsistent locale settings will cause date parsing to fail. Set LANG and LC_ALL explicitly.

Scheduled tasks need explicit timezone documentation. When you have cron jobs or scheduled workflows across regions, write down what timezone each schedule uses. Don't rely on "the system timezone" because that's ambiguous in a distributed setup.

Verify every container individually. Don't assume your configuration worked. Exec into each container, run date and locale, and confirm the output matches your expectations.