It is generally known and established that different database versions provide different sets of features and performance rates. Along with the binary versions, MongoDB has a property called Feature Compatibility Value, which allows administrators to run a binary version without enabling any features that are incompatible with the previous one. This property is generally helpful for upgrades. But what is the impact of keeping these “legacy” features on performance? Can the current version run at its peak with them? The objective of this post is to make a brief comparison between a couple of binary versions with and without the FCV of the previous version.
Environment
For this comparison, I chose the latest three (and only non-EOL) versions of MongoDB to date. I used the Percona Server for MongoDB distribution and the latest minor versions available with stock configurations:
- 6.0.20
- 7.0.16
- 8.0.4
Regarding hardware, I used four AWS EC2 VMs (one client and a replica set with three instances) of size m5a.xlarge (4 CPUs, 16GB of RAM).
Process
To benchmark these versions, I used the mongo-perf tool and chose 5 tests to cover the basic CRUD plus aggregations (given they are a popular feature in MongoDB): simple_insert, simple_update, simple_query, simple_sort_queries, and simple_remove.
Each test ran for 60 seconds, then I rebooted the VMs to clean the cache and dropped the database before moving on to the next. I ran 3 rounds of this and calculated the average of operations per second of each test as the final evaluation.
The comparison
I made eight comparisons to come to reach the following conclusions. Green cells mark the fastest, and red cells mark the slowest results:
- All versions and FCVs
- v7.0 binaries and FCV 6.0 vs v7.0 binaries and FCV 7.0 (same binaries, different FCVs)
- v8.0 binaries and FCV 7.0 vs v8.0 binaries and FCV 8.0 (same binaries, different FCVs)
- v6.0 binaries and FCV 6.0 vs v7.0 binaries and FCV 6.0 (same FCV, different binaries)
- v7.0 binaries and FCV 7.0 vs v8.0 binaries and FCV 7.0 (same FCV, different binaries)
- v6.0 binaries and FCV 6.0 vs v7.0 binaries and FCV 7.0 (binaries and respective FCVs)
- v6.0 binaries and FCV 6.0 vs v8.0 binaries and FCV 8.0 (binaries and respective FCVs)
- v7.0 binaries and FCV 7.0 vs v8.0 binaries and FCV 8.0 (binaries and respective FCVs)
Fast (and hot?) takes
- MongoDB v8.0 is the fastest in general, with considerable margins
- MongoDB v6.0 is the slowest in general
- MongoDB v7.0 runs faster with FCV 6, but the difference is minimal
- Reads are a little faster in MongoDB v8.0 with FCV 7 when compared to FCV 8
- The FCV 6 is faster for reads in MongoDB v7.0 than in v6.0, while writes are a little faster in the latter
- The FCV 7 is way faster in MongoDB v8.0 than in v7.0
- MongoDB v7.0 is faster than v6.0 in reads and a little slower in writes when using their respective FCVs
Conclusion
The results agree with MongoDB Inc.’s announcement that v8.0 is the fastest version (at least compared with the two previous ones), and with the article MongoDB 8.0 Performance; Does It Live Up to the Hype? by a Perconian when this version was released.
It is visible that changing the binaries has a more considerable difference in performance than changing the FCV. Keeping the FCV of the previous version has a good impact in some specific cases, likely due to some simpler code paths, but the benefit generally is so small that it does not justify the loss of the new features and the risk of having issues during the next upgrade.
These tests were designed to cover a broad range of real-life scenarios, comprising operations of low and medium complexity. Tests using different tools, methods, and workloads can result in different numbers. If you did similar tests and reached very different results, tell us in the comments.
If you have any doubts on the topic or issues upgrading your MongoDB environments, Percona Experts are available 24/7 to assist you.
Whether you’re a seasoned MongoDB DBA or just getting started, this eBook offers clear strategies for upgrading smoothly and keeping your databases optimized:
MongoDB Upgrade Best Practices
