unattended upgrades are not scheduled correctly

[mjourdan]

Description

Unattended upgrades are scheduled only once, due to an empty "OnCalendar" line in timer definition.

To reproduce

Deploy the role so that apt-daily is triggered in a couple of minutes:

- name: Testing              
  hosts: all
  become: yes
  become_method: ansible.builtin.sudo

  tasks:

	- name: Debian - unattended upgrades
	  ansible.builtin.import_role:
	  	name: hifis.toolkit.unattended_upgrades 
      vars:
        unattended_systemd_timer_override: true
        unattended_apt_daily_oncalendar: "*-*-* 13:52"
        unattended_apt_daily_randomizeddelaysec: "1m"

Connect to the host and list timers to check the schedule date.

Current behavior

Timers is scheduled once:

# systemctl list-timers
NEXT                         LEFT           LAST                         PASSED       UNIT                         ACTIVATES
Fri 2025-05-23 13:52:34 CEST 1min 17s left  Fri 2025-05-23 11:48:01 CEST 2h 3min ago  apt-daily.timer              apt-daily.service

After timers passed, it is not scheduled again:

:~# systemctl list-timers
NEXT                         LEFT           LAST                         PASSED       UNIT                         ACTIVATES
-                            -              Fri 2025-05-23 13:52:34 CEST 37s ago      apt-daily.timer              apt-daily.service

Expected behavior

Timers is scheduled again after its first run.

OS / Environment

Debian 12

Ansible version

ansible [core 2.18.6]
  config file = /home/mjourdan/git/ansible/ansible.cfg
  configured module search path = ['/home/mjourdan/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/mjourdan/.ansible_virtualenvs/ansible-11.6.0/lib/python3.11/site-packages/ansible
  ansible collection location = /home/mjourdan/.ansible/collections:/usr/share/ansible/collections
  executable location = /home/mjourdan/.ansible_virtualenvs/ansible-11.6.0/bin/ansible
  python version = 3.11.2 (main, Apr 28 2025, 14:11:48) [GCC 12.2.0] (/home/mjourdan/.ansible_virtualenvs/ansible-11.6.0/bin/python3)
  jinja version = 3.1.6
  libyaml = True

Collection version

v5.3.0

Additional context

Please feel free to add any other context about the problem here.

[Normo]

Thanks for reporting this, @mjourdan! Feel free to open a pull request to address the issue.

[Normo]

Dear @mjourdan, I took a closer look at this issue.

The first line OnCalendar= clears all previous definitions, which is intentional and systemd-compliant. The second line then defines the new daily time. See also the systemd.timer documentation:

If the empty string is assigned to any of these options, the list of timers is reset (both OnCalendar= timers and monotonic timers, see above), and all prior assignments will have no effect.

The behavior you’re seeing - a NEXT time missing directly after execution - is standard behavior of systemd timers that have just fired. This is a transient state and the next activation time is usually recomputed shortly after, or after a daemon reload.

Before first execution:

root@instance2:/# systemctl list-timers apt-daily*
NEXT                        LEFT         LAST                        PASSED    UNIT                    ACTIVATES                
Mon 2025-05-26 08:00:00 UTC 4s left      Mon 2025-05-26 07:30:36 UTC 29min ago apt-daily.timer         apt-daily.service
Mon 2025-05-26 08:02:00 UTC 2min 4s left Mon 2025-05-26 07:30:36 UTC 29min ago apt-daily-upgrade.timer apt-daily-upgrade.service

Immediately after the first run:

root@instance2:/# systemctl list-timers apt-daily*
NEXT                        LEFT          LAST                        PASSED    UNIT                    ACTIVATES                
Mon 2025-05-26 08:02:00 UTC 1min 53s left Mon 2025-05-26 07:30:36 UTC 29min ago apt-daily-upgrade.timer apt-daily-upgrade.service
-                           -             Mon 2025-05-26 08:00:02 UTC 3s ago    apt-daily.timer         apt-daily.service

This is normal. The timer has just expired, systemd has not yet calculated the next date, or it is only temporarily not in the list.

A few seconds later:

root@instance2:/# systemctl list-timers apt-daily*
NEXT                        LEFT          LAST                        PASSED    UNIT                    ACTIVATES                
Mon 2025-05-26 08:02:00 UTC 1min 48s left Mon 2025-05-26 07:30:36 UTC 29min ago apt-daily-upgrade.timer apt-daily-upgrade.service
Tue 2025-05-27 08:00:00 UTC 23h left      Mon 2025-05-26 08:00:02 UTC 9s ago    apt-daily.timer         apt-daily.service

The next execution is usually scheduled again within a few seconds or minutes. If not, you can trigger a reload with systemctl daemon-reload. If the problem persists, please run

systemctl cat apt-daily.timer

and share the output with us.

[mjourdan]

The first line OnCalendar= clears all previous definitions, which is intentional and systemd-compliant. The second line then defines the new daily time. See also the systemd.timer documentation:

That's what I thought but when I tried to check it out in the documentation I somehow managed to miss the relevant part. Thanks for the pointer!

After a systemctl daemon-reload and even a systemctl restart apt-daily.timer, here is what I have:

# systemctl list-timers apt-daily*
NEXT LEFT LAST                         PASSED       UNIT                    ACTIVATES
-    -    Sat 2025-05-24 04:03:23 CEST 2 days ago   apt-daily-upgrade.timer apt-daily-upgrade.service
-    -    Mon 2025-05-26 16:22:46 CEST 2min 59s ago apt-daily.timer         apt-daily.service
                                                                                                                                                                                                                 2 timers listed.
Pass --all to see loaded but inactive timers, too.

# systemctl cat apt-daily.timer
# /lib/systemd/system/apt-daily.timer
[Unit]
Description=Daily apt download activities

[Timer]
OnCalendar=*-*-* 6,18:00
RandomizedDelaySec=12h
Persistent=true

[Install]
WantedBy=timers.target

# /etc/systemd/system/apt-daily.timer.d/schedule_override.conf
[Unit]
Description=Daily apt download activities

[Timer]
OnCalendar=*-*-* 03:00
RandomizedDelaySec=30m
Persistent=true

[Install]
WantedBy=timers.target

[Normo]

Thanks @mjourdan.

It looks like the line OnCalendar= is still missing in your current override. Without the explicit OnCalendar= reset, systemd does not clear the original schedule defined in the base unit (/lib/systemd/system/apt-daily.timer), but instead adds your override entry on top.

I think this can lead to unexpected behavior, especially when the original timer includes a large RandomizedDelaySec like 12h. In such cases, the timer might appear stuck or not be rescheduled at all.

Could you please re-add the line OnCalendar=? Also, it might help to try a smaller value for unattended_apt_daily_randomizeddelaysec. Please let me know if it helps.

[mjourdan]

Thanks for your help @Normo. I did further tests including the OnCalendar= line. Long story short: the lack of a trigger date was because of old pending apt.systemd.daily processes. Killing those processes seem to be enough to get the expected bahavior back.