In the last two months, we have got quite a few issues with the server time and date occasionally changing to some absolute random values. Sometimes it is a few days in the future or past, sometimes - even several months.
We have got the problem on several VMs, so it is not the problem with one VM unfortunately.
We run Windows Server 2019 (Version 1809 OS Build 17763.3406) on Amazon EC2.
We synced time using AWS default endpoint 169.254.169.123, and then we switched to managed AWS Active Directory. The provisioning is done with Ansible, not sure if it matters though.
All time change events are logged in the system, and look identical, like this one:
>
The system time has changed to 2022-12-23T01:10:04.150070300Z from >2022-09-14T12:22:57.146781800Z.
Change Reason: An application or system component changed the time.
Process: '\Device\HarddiskVolume1\Windows\System32\svchost.exe' (PID 372).
As you can see, the change is very significant and thus, affects the web applications run on IIS server on these VMs.
Do you know what can be the reason, and where we should look in order to fix the issue?
Edit: adding details according to @Greg's comment:
>
C:\Windows\system32>w32tm /query /status /verbose Leap Indicator: 0(no
warning) Stratum: 6 (secondary reference - syncd by (S)NTP) Precision:
-23 (119.209ns per tick) Root Delay: 0.0643878s Root Dispersion: 7.8759466s ReferenceId: 0x0A050C5D (source IP: 10.3.15.74) (edited) Last Successful Sync Time: 11/2/2022 11:05:39 PM Source:
WIN-BOLJH2K6OOL.companydomain.cloud (edited) Poll Interval: 6 (64s)
Phase Offset: -0.0023147s ClockRate: 0.0156250s State Machine: 1
(Hold) Time Source Flags: 8 (SignatureAuthenticated ) Server Role: 0
(None) Last Sync Error: 0 (The command completed successfully.) Time
since Last Good Sync Time: 8.5343114s
Windows Server 2019 randomly changers time and date to incorrect values
Re: Windows Server 2019 randomly changers time and date to incorrect values
It seems the issue is caused by a setting Microsoft enabled in 2016 called UtilizeSSLTimeData. Here is a good explanation of the situation... [https://arstechnica.com/security/2023/08/windows-feature-that-resets-system-clocks-based-on-random-data-is-wreaking-havoc/](https://arstechnica.com/security/2023/08/windows-feature-that-resets-system-clocks-based-on-random-data-is-wreaking-havoc/)
Here's Microsoft's solution... [https://learn.microsoft.com/en-us/windows-server/networking/windows-time-service/windows-server-2016-improvements#secure-time-seeding](https://learn.microsoft.com/en-us/windows-server/networking/windows-time-service/windows-server-2016-improvements#secure-time-seeding)
>
Secure Time Seeding
W32time in Server 2016 includes the Secure Time Seeding feature. This
feature determines the approximate current time from outgoing SSL
connections. This time value is used to monitor the local system clock
and correct any gross errors. You can read more about the feature in
this blog post. In deployments with a reliable time source(s) and well
monitored machines that include monitoring for time offsets, you may
choose to not use the Secure Time Seeding feature and rely on your
existing infrastructure instead.
You can disable the feature with these steps:
Set the UtilizeSSLTimeData registry configuration value to 0 on a specific machine...
reg add
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\w32time\Config /v
UtilizeSslTimeData /t REG_DWORD /d 0 /f
If you are unable to reboot the machine immediately due to some
reason, you can notify W32time service about the configuration update.
This stops time monitoring and enforcement based on time data
collected from SSL connections.
W32tm.exe /config /update
Rebooting the machine makes the setting effective immediately and also
causes it to stop collecting any time data from SSL connections. The
latter part has a very small overhead and should not be a perf
concern.
To apply this setting in an entire domain, please set the
UtilizeSSLTimeData value in W32time group policy setting to 0 and
publish the setting. When the setting is picked up by a Group Policy
Client, W32time service is notified and it will stop time monitoring
and enforcement using SSL time data. The SSL time data collection will
stop when each machine reboots. If your domain has portable slim
laptops/tablets and other devices, you may want to exclude such
machines from this policy change. These devices will eventually face
battery drain and need the Secure Time Seeding feature to bootstrap
their time.
Here's Microsoft's solution... [https://learn.microsoft.com/en-us/windows-server/networking/windows-time-service/windows-server-2016-improvements#secure-time-seeding](https://learn.microsoft.com/en-us/windows-server/networking/windows-time-service/windows-server-2016-improvements#secure-time-seeding)
>
Secure Time Seeding
W32time in Server 2016 includes the Secure Time Seeding feature. This
feature determines the approximate current time from outgoing SSL
connections. This time value is used to monitor the local system clock
and correct any gross errors. You can read more about the feature in
this blog post. In deployments with a reliable time source(s) and well
monitored machines that include monitoring for time offsets, you may
choose to not use the Secure Time Seeding feature and rely on your
existing infrastructure instead.
You can disable the feature with these steps:
Set the UtilizeSSLTimeData registry configuration value to 0 on a specific machine...
reg add
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\w32time\Config /v
UtilizeSslTimeData /t REG_DWORD /d 0 /f
If you are unable to reboot the machine immediately due to some
reason, you can notify W32time service about the configuration update.
This stops time monitoring and enforcement based on time data
collected from SSL connections.
W32tm.exe /config /update
Rebooting the machine makes the setting effective immediately and also
causes it to stop collecting any time data from SSL connections. The
latter part has a very small overhead and should not be a perf
concern.
To apply this setting in an entire domain, please set the
UtilizeSSLTimeData value in W32time group policy setting to 0 and
publish the setting. When the setting is picked up by a Group Policy
Client, W32time service is notified and it will stop time monitoring
and enforcement using SSL time data. The SSL time data collection will
stop when each machine reboots. If your domain has portable slim
laptops/tablets and other devices, you may want to exclude such
machines from this policy change. These devices will eventually face
battery drain and need the Secure Time Seeding feature to bootstrap
their time.