.avif)
.avif)
Time-to-Live (TTL) is one of the most critical automation mechanisms in MongoDB, enabling organizations to retain only relevant data and automatically purge aged records without manual intervention. But what happens when this automation silently breaks?
During a recent client engagement, we ran into a production issue where TTL completely stopped working on MongoDB 8.0.4. No documents were getting deleted. Disk usage kept climbing. And here's the kicker—there were no errors in the mongod logs, and TTL monitor metrics looked completely normal.
After digging deep, we traced the problem to a known MongoDB issue: SERVER-97368, which affects TTL processing under a very specific condition.
The Silent TTL Failure: What Happened
Even though the TTL index was configured correctly, no documents were getting deleted, and disk usage continued to grow. There were no errors in mongod logs, and TTL monitor metrics also appeared normal, making this a particularly challenging issue to diagnose.
Further deep-dive analysis revealed that TTL was silently skipped for the entire collection because one or more documents contained a timestamp beyond the supported Unix epoch window—January 19, 2038.
This behaviour impacts time-series collections on MongoDB 8.0.4.
Root Cause: The 2038 Timestamp Problem
Impact Condition
TTL evaluation halts completely when:
- The collection is time-series
- At least one document contains a future timestamp beyond 2038-01-19T03:14:07Z
This date isn't random. It's the Unix epoch's 32-bit timestamp limit—the moment when systems using signed 32-bit integers to represent seconds since January 1, 1970, hit their maximum value.
How We Found the Culprit
Example Discovery Query
db.collection.find(
{ "timestamp": { "$gt": ISODate("2038-01-19T03:14:07Z") } },
{ timestamp: 1 }
)Returned Record
{
"_id": ObjectId("67b18810ea4ff641b0f87d6c"),
"timestamp": ISODate("2038-08-16T16:00:52.801Z"),
"metadata": {
"h3_9r": "893da18c933ffff",
"rider_id": "4939"
}
}That single document with an August 2038 timestamp was enough to break TTL for the entire collection.
Impact on Time-Series Collections
The presence of such a document resulted in:
❗ TTL monitor skipping the entire collection rather than purging expired documents
This can lead to:
- Unbounded storage growth
- Increasing compute/I/O pressure over time
- Risk of breaching compliance and retention policies
- Potential production outages due to disk space exhaustion
Silent failures like this can impact production performance similar to write conflicts in high-concurrency environments.
Unlike typical TTL failures where the monitor logs errors or stops running, this bug causes silent skipping—no warnings, no alerts, nothing. That's what made it so dangerous in our case.
Resolution: Upgrading to MongoDB 8.0.12
MongoDB has addressed this bug in 8.0.12. The fix enables TTL deletes on time-series collections containing data with timestamps outside the standard Unix epoch range (including dates prior to 1970 and beyond 2038).
After upgrading from 8.0.4 → 8.0.12, TTL immediately resumed purging expired records, and disk usage started stabilizing.
Verification After Upgrade
After upgrading, we confirmed TTL was back in action by checking the TTL monitor status:
db.serverStatus().metrics.ttlThe passes counter was incrementing every ~60 seconds, and deletedDocuments was climbing again as expired documents are removed.
Prevention and Monitoring Best Practices
Immediate Actions:
Detection Query
Run this diagnostic check on your time-series collections to identify problematic documents:
db.collection.find(
{ "timestamp": { "$gt": ISODate("2038-01-19T03:14:07Z") } },
{ timestamp: 1 }
)Key Takeaways
Final Note
This incident is a strong reminder that data validation rules are as important as database configurations. Even a single malformed timestamp can disable lifecycle automation across an entire time-series dataset.
If your workloads rely on TTL for retention-based purging, it is highly recommended to review your version compatibility and implement timestamp validation safeguards as part of your ingestion pipeline.
We've seen firsthand how subtle data issues can snowball into production incidents. The lesson here? Don't trust that everything's working just because there are no errors in the logs. Proactive monitoring and version awareness matter.
For a comprehensive understanding of TTL indexes, check out our Ultimate Guide to MongoDB TTL Indexes.
Running MongoDB 8.0.4 with time-series collections? Consider upgrading to 8.0.12 a priority item. Need help with the migration or want us to audit your TTL configuration? Get in touch with our team.






