Slow query log persistence
Overview
Slow query log persistence records operations that exceed the configured slowQueryMs threshold and lets you query aggregated slow-query statistics later. In the current MongoDB runtime, records are stored in MongoDB by default and may use memory storage for local or custom scenarios.
Core Features
- Simple enablement: use
slowQueryLog: truewithslowQueryMs. - Aggregated records: records with the same
queryHash, database, collection, and operation are upserted into one statistic row. - Batch writing: enabled by default with configurable size, interval, and buffer limit.
- Automatic expiration: MongoDB storage creates a TTL index on
lastSeen. - Query API:
getSlowQueryLogs(filter, options)returns stored slow-query statistics. - Current storage backends:
mongodbandmemory.
Working principle
Quick start
Simplest configuration (recommended)
Automatic effects
- Slow queries are automatically saved to the
admin.slow_query_logscollection - Similar query records are aggregated by
queryHash, database, collection, and operation - TTL index automatically cleans data from 7 days ago
- Reuse business connections without additional connection overhead
Configuration instructions
Global slow query configuration
Basic configuration (set during initialization)
Detailed explanation of configuration options
Operation level configuration
Slow query thresholds can be configured individually for certain operations:
Method 1: Through options parameter
Method 2: Use global configuration
Log format
JSON format (default)
Text format
Log output configuration
Use custom Logger
Listen for slow query events
Configuration level
monSQLize provides a three-tier configuration architecture to meet different usage scenarios:
Level 1: Zero configuration (recommended)
Default configuration:
enabled: true- Enablestorage.type: 'mongodb'- MongoDB storage for the MongoDB runtimestorage.useBusinessConnection: true- Multiplexed business connectionstorage.database: 'admin'- stored in admin databasestorage.collection: 'slow_query_logs'- collection namestorage.ttl: 604800- Expires in 7 daysbatch.enabled: true- Enable batch writingbatch.size: 10- batch sizebatch.interval: 5000- refresh in 5 seconds
Level 2: Basic configuration (commonly used)
Level 3: Complete Configuration (Advanced)
Detailed explanation of configuration parameters
storage storage configuration
batch batch configuration
filter filter configuration
API Reference
getSlowQueryLogs(filter, options)
Query slow query log (supports scheme B aggregation data)
Parameters:
Return Value:
Usage Example:
Usage example
Example 1: Zero configuration enablement
Example 2: Independent connection (isolated resources)
Example 3: Custom TTL
Example 4: Filter a specific collection
Example 5: Memory storage for local analysis
Example 6: Real-time writing mode
Best Practices
Threshold setting suggestions
Choose an appropriate slow query threshold according to different scenarios:
Example: Different scenario configurations
Monitoring and Analysis
1. Real-time monitoring
2. Regular analysis
3. Export slow query statistics
Optimization suggestions
1. Create index
2. Optimize query conditions
3. Use projection to reduce data transmission
4. Enable caching
Best Practices for Production Environments
1. Set TTL appropriately
2. Use reuse connection (default)
When to use independent connections:
- Production database performance is extremely sensitive
- The amount of slow query logs is large (>10,000 entries/day)
- Requires independent permission control and resource isolation
3. Configure alarms
4. Monitor storage space
5. Integrated log system
Troubleshooting
Problem 1: Slow query log is not saved
Possible reasons:
-
Function not enabled
-
The query does not exceed the threshold
-
The batch queue is not refreshed
Solution:
- Check whether the configuration is correct
- Check the log output for errors
- If batch mode is enabled, call
await msq.getSlowQueryLogManager()?.queue?.flush()before reading recent logs.
Problem 2: Storage connection failed
Error message:
Possible reasons:
- MongoDB connection URI error
- The network is unreachable
- Insufficient permissions
Solution:
Question 3: Querying the slow query log returns empty
Possible reasons:
- Slow query log is not generated
- Expired (TTL deleted)
- Query conditions do not match
Solution:
Performance optimization
Performance impact analysis
Conclusion: Logging overhead depends on sampling, storage, serialization, and workload. Measure it in the target environment and set an explicit budget; do not assume it is negligible.
Optimization suggestions (performance optimization)
-
Use batch writes (enabled by default)
-
Keep the built-in aggregation key
-
Set TTL reasonably
-
Filter unimportant collections
Appendix
A. Data model
slow_query_logs collection structure:
Index: