(Updated Dec 14) Critical Alert: Do Not Restart Domino Servers After 2024-12-13 Without the Hot Fix
14 Dec 2024 UPDATE: Less than 24 hours from first posting and the technote below has already been updated to include links to the hot fixes for the latest versions of 11, 12, and 14 with additional fixes coming. Note: You must be current on your software support license to obtain the patches. Also updated that patches will be provided for V9 and V10 for customers paying for extended support. HCL is doing a great job handling this.
HCL released this alert regarding all versions of Domino on all platforms.
In short, there is a date/time bug in the code that will cause mail routing to fail if the server is restarted as of 13 December 2024. Read the technote for details:
“CRITICAL ALERT: Mail not routing after Domino restarts beginning 13 December 24”
https://support.hcl-software.com/csm?id=kb_article&sysparm_article=KB0118192
This is the first time I have ever seen such a notice regarding Lotus Notes / Domino ever. To this I will say,
First, impressive that this is the first time in 30 years (including leading up to Y2K) that they have had such a notice and
Second, that by mid day, they had identified the issue and are working on the hot fix.
Third, this is only an issue if you restart Domino, (though I would avoid restarting the router task as well) and Domino is renowned for its reliability, so hopefully you we can all just leave well enough alone and wait for the HF.
NOTICE: if you are one of those companies who quit paying your software and support licensing, you’re about to get a reminder that software isn’t free. You will need to re-instate your licenses to get access to the hot fix. Even if you are running *OLD versions* (9.0 and earlier even), your server will not route mail once it is restarted. An impactful time bomb for sure. (UPDATE: The technote has been updated that a hot fix will also be created for V9.0.1 and V10, provided you are on extended support)
With that in mind, If you need to re-instate your licenses so you can access the hot fix, send an email to me at Info@divergentnw.com . Don’t wait until your server restarts.
My heart goes out to the developers working to generate a hot fix for this. Not the kind of pressure anyone wants going into a weekend. My turn will come in this event. I will be the one installing the hot fix on servers everywhere as soon as it is available.
Posted on December 13, 2024, in HCL, Lotus and tagged Domino, HCL, HCL Domino, Lotus Notes. Bookmark the permalink. 14 Comments.
The Notes Guy in Seattle 
whoops, my bad
could be you Ray, as it started back in R9 days 🙂
There will be a hot fix for version 9 and 10 as stated in the Critical Alert: “NOTE: Fixes will be provided for versions 9 and 10 for customers with support extensions.”
//Christian Gravgaard
Problem is related to field $Hops. If You delete this field mail would be transfered. Outgoing STMP mail works also from the source server (if home server and SMTP server are the same)
it is possible to route between Domino servers using the SMTP protocol as a workaround, provided that SMTP continues to function normally
I finally hit the right search keywords to find this. Thank you for posting! And, thank you, Artur Koro, for including the keyword “$Hops” in your reply!
I know it’s not supported, but as a workaround I created a scheduled agent using the code from the ‘Release | Resend Select Held Messages’ action button in the mail.box and this is currently working until we get our support in order and fix downloaded and applied. I got tired of manually releasing the held messages. 🙂
In case anyone is trying that method, well, mail.box agent quit working at midnight unless you restart the server or router, or resave the agent. So, I added a program document to run the agent.
Program name: nserver
Command line: -c “Tell amgr run mail.box ‘Release'”
Thanks for posting the additional details as we discover more about the specific conditions that cause the failure. It helps to have just one mail.box to avoid this issue as well.
Hi
Does this address
Maximum hop count exceeded. Message probably in a routing loop. We are continuously receiving these mails in the mail server log and mail not received when we have this issue
Yes, that is it.
We are using Lotus note 7.0.1 version May I know exactly what to do here?
ATTENTION: A temporary workaround has been released as seen in tech note below and fully supported by NotesMail – HCL Business Partner. It is a Notes app which can be installed on your gateway mail server to resolve the issue.
https://www.notesmail.com/home.nsf/tip20241213?openpage
The critical bug which existed dormant for 35 years happened during the worst possible time in the holiday season. If you are currently an unlicensed customer for Domino 11 and higher or Domino 10 and lower I am sure you are feeling frustrated. Anyway, we wanted to let you know we have a fully supported workaround if you don’t have access to the HCL fix packs and interim fixes yet due to many reasons (i.e. end of year financial budget issue, resources and/or management on vacation, frozen Domino IT budget, etc.). FYI: We even have a customer case where it fixed the issue on a 32-bit Domino 9.0.x Server which wasn’t included in their current Domino license.
Wishing you happy holidays and a happy new year! #dominoforever
I am clarifying the post above for many of you who are submitting questions on our service case link. Yes, Auto Router Buf Fix (ARF) works perfectly as a workaround fix. It is running in production at several companies! However, it does not resolve the Mail Rules issue, but includes a tech note for a semi-automated Mail Rules workaround fix. Basically, ARF is running in production at least 15-20 companies including environments running Domino 7.0.1, 8.5, 9.0.1, 12 and 14. Each company confirmed the mail started routing properly that was stuck in a mail loop. It works! Note: After following instructions they released their dead mail as well which started routing if dead mail was stuck in a mail loop.
You can download the ARF Notes app tool by clicking link below. Please specify your Domino gateway mail server names on the form. Happy New Year!
FYI
CRITICAL BUG: Download Workaround Fix – Domino mail not routing starting Dec 13, 2024 ⚠️❗❗
https://www.notesmail.com/home.nsf/tip20241213?openpage
The Notes app tool in link above is a workaround which automatically fixes the mail routing issue in HCL knowledge article KB0118192, but not the Mail Rules issue. However, refer to step 5 to semi-automatically fix the Mail Rules. It supports Domino versions 14 and lower including 10.0, 9.0, 8.5 and 7.0.1 which are End of Life (EOL), as well as, single and multiple mail router mailboxes.
The critical bug was dormant for 35 years and activated at the worst possible time during the holiday season. If you are an unlicensed Domino customer you most likely are feeling frustrated. The tool is a fully supported workaround fix by NotesMail – HCL Business Partner. If you don’t have access to the HCL fix packs and interim fixes (hotfixes), then you should use this tool.
Hi there i do some fix for Lotus Server 9, i added a external smtp (open relay or with id) Configuration -Messaging Settings – General , Host relay ENable and enable use of smtp inside of domain of internet with option in all messages. And the magic is force Domino server to use an “Internet Address” from the Person document in Domino Directory as a sender’s adress with this Domino server configuration to remove “INetFrom” item from email header, url with this steps:
https://ypastov.blogspot.com/2013/01/how-to-force-outbound-email-to-use.html
How can help to others without patching. With this workaround the rules for send to another email works.
Regards and sorry for my bad english 😉