Featured
Percona Products
Percona Services
Use Cases
Akira KuroganeThis article continues on from Part 1: Building “wt” and “Part 2: wt dump” to show how to extract any of your MongoDB documents directly from WiredTiger’s raw data files. It’ll also show how to take a peek into the index files. Lastly, it’ll show how to also look in the WT transaction log to […]
Percona Server for MongoDB (PSMDB) has provided a ‘hot’ backup of its underlying data db directory files using WiredTiger library methods since v3.4. When you use the { createBackup: … } command it will copy them to whichever other filesystem directory or object store bucket location you specify. createBackup partially freezes the WiredTiger *.wt Btree […]
This article contains one normal section – how to print information directly from the raw WiredTiger files using wt dump – followed by really important, otherwise undocumented information about how to get to the MongoDB binary data inside WT tables. See “WiredTiger File Forensics (Part 1: Building “wt”)” for: How to build the “wt” utility […]
Most of the files in a data directory of a MongoDB server are made by the WiredTiger storage engine. If you want to look at the content inside them you can use the tool “wt” from the WiredTiger library: https://github.com/wiredtiger/wiredtiger/ http://source.wiredtiger.com/10.0.0/command_line.html Inspection of the WiredTiger files is not an essential MongoDB DBA skill – it’s […]
Percona Backup for MongoDB (PBM) has reached a new step with the release of version 1.5.0 today, May 13th, 2021. Azure Blob Storage Support Now you can use Azure Blob Storage as well as S3-compatible object stores. Configuration example:
|
1 2 3 4 5 6 7 8 |
storage: type: azure azure: account: <string> container: <string> prefix: <string> credentials: key: <your-access-key> |
Preference Weight for Backup Source Nodes Until now PBM would use a secondary as a […]
For MongoDB data directories only XFS is recommended. The ext4 filesystem isn’t so bad but when there are a very, very high number of random accesses (which WiredTiger can reach) it can hit a bottleneck. To be fair most deployments will never hit this bottleneck, but it does remain an official production recommendation of MongoDB […]
I’ve been involved with MongoDB for quite a while now. I follow it from the code and I’ve been reviewing the changes in the 4.3 development version, plus the recent 4.4 release candidate versions rc0 – rc7. There are a few thousand github commits, but at the human scale, you’d say 100 ~ 200 things […]
In this blog post, we will discuss how to deal with jumbo chunks in MongoDB. Scenario: You are a MongoDB DBA, and your first task of the day is to remove a shard from your cluster. It sounds scary at first, but you know it is pretty easy. You can do it with a simple […]
Percona Backup for MongoDB v1.2 is out! In a nutshell it offers (much) faster compression choices, backup cancellation, and assorted convenience features such as a universal backup deletion command and progress logging. And, for the DBAs out there in the field, we’ve added a whole extra binary (pbm-speed-test) that will give a quick way to […]
Introducing a new MongoDB Storage Engine Q. What is the Heterogeneous-memory Storage Engine? A key value store library developed (and open-sourced) by Micron that will work with any normal storage but works especially well with emerging SSDs or NVDIMMS (or other Storage Class Memory) that have faster even NVM media in them (Bleeding-edge NAND, or […]
“Get big fast” was a mantra in the second eCommerce boom. MongoDB’s success during its startup years was due in large part to being the best database to support that. But maybe, after your growth years, the data’s rent bill has become a little high? Big data was nice, but you absorbed a lot of […]
Abstract: Storage engine algorithmic gains have mostly settled and Moore’s law for CPU speed is bottoming out, but database performance still stands to increase ~10x thanks to continuing NAND Flash improvement, Optane, Flash-idiomatic SSD drives (e.g. OpenChannel, Zoned Namespaces), KeyValue SSD, etc. The downside is each storage engine revolution historically has broken encapsulation/reuse and required […]
Recently I noticed the site Jan 2019 results download link) are a subset of the mongo javascript shell five dbaas_*.yml test suites used are sensible-for-DBAAS subsets of the same aggregation, decimal, and competition-comparison page at mongodb.com. But the scripts are not 1-to-1 tests of features. There are more than twice as many test scripts as […]
TLDR: It is not profile level 1 that is the problem; it’s a gotcha with the optional ‘slowms’ argument that causes users to accidentally set verbose logging and fill their disk with log files. In MongoDB, there are two ways to see, with individual detail, which operations were executed and how long they took. Profiling. […]
Editor’s Note: The first version of this post contained a section criticizing what appeared to be a major regression concerning dropDatabase and movePrimary commands. It was found out that it was merely a documentation error in the MongoDB 4.2 release notes, which is now fixed: https://jira.mongodb.org/browse/DOCS-12474. The “(In)Stability” section is now removed. At Percona we’ve […]
Why do I need Network encryption? In our previous blog post MongoDB Security vs. Five ‘Bad Guys’ there’s an overview of five main areas of security functions. Let’s say you’ve enabled #1 and #2 (Authentication, Authorization) and #4 (Storage encryption a.k.a. encryption-at-rest and Auditing) mentioned in the previous blog post. Only authenticated users will be […]
Most any commercially mature DBMS provides the following five ways to secure the data you keep inside it: Authentication of user connections (== Identity) Authorization (== DB command permissions) (a.k.a. Role-based access control) Network Encryption (a.k.a. Transport encryption) Storage Encryption (a.k.a. Encryption-at-rest) Auditing (MongoDB Enterprise or Percona Server for MongoDB only) MongoDB is no exception. […]
Mistakes can happen. If only we could go back in time to the very second before that mistake was made. Act 1: The Disaster Plain text version for those who cannot run the asciicast above:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 |
akira@perc01:/data$ #OK, let's get this party started! akira@perc01:/data$ # The frontend has been shut down for 20 mins so they can akira@perc01:/data$ # update that part, and I can update the schema in he akira@perc01:/data$ # backend simultaneously. akira@perc01:/data$ #Easy-peasy ... akira@perc01:/data$ date Tue Jul 2 13:34:09 JST 2019 akira@perc01:/data$ #Just set my auth details.(NO PEEKING!) akira@perc01:/data$ conn_args="--host localhost:27017 --username akira --password secret --authenticationDatabase admin" akira@perc01:/data$ mongo ${conn_args} --quiet testrs:PRIMARY> use payments switched to db payments testrs:PRIMARY> show collections TheImportantCollection testrs:PRIMARY> //Ah, there it is. Time to work! testrs:PRIMARY> db.TheImportantCollection.count() 174662 testrs:PRIMARY> db.TheImportantCollection.findOne() { "_id" : 0, "customer" : { "fn" : "Smith", "gn" : "Ken", "city" : "Georgevill", "street1" : "1 Wishful St.", "postcode" : "45031" }, "order_ids" : [ ] } testrs:PRIMARY> //Ah, there it is. The "customer" object that has the testrs:PRIMARY> //address fields in it. We're going to move those out. testrs:PRIMARY> //Copy the whole collection, adding the new "addresses" array testrs:PRIMARY> var counter = 0; testrs:PRIMARY> db.TheImportantCollection.find().forEach(function(d) { ... d["adresses"] = [ ]; ... db.TheImportantCollectionV2.insert(d); ... counter += 1; ... if (counter % 25000 == 0) { print(counter + " updates done"); } ... }); 25000 updates done 50000 updates done 75000 updates done 100000 updates done 125000 updates done 150000 updates done testrs:PRIMARY> //Cool. Let's look at the temp table testrs:PRIMARY> db.TheImportantCollectionV2.findOne() { "_id" : 0, "customer" : { "fn" : "Smith", "gn" : "Ken", "city" : "Georgevill", "street1" : "1 Wishful St.", "postcode" : "45031" }, "order_ids" : [ ], "adresses" : [ ] } testrs:PRIMARY> //?AH!! testrs:PRIMARY> //typo. I misspelled "addresses". testrs:PRIMARY> //I'll just drop this and go again testrs:PRIMARY> db.TheImportantCollectionV2.remove({}) WriteResult({ "nRemoved" : 174662 }) testrs:PRIMARY> //ooops. Why did I bother deleting the docs? testrs:PRIMARY> //I need to *drop* the collection testrs:PRIMARY> db.TheImportantCollection.drop() true testrs:PRIMARY> //!!!! testrs:PRIMARY> //Wait! testrs:PRIMARY> show collections TheImportantCollectionV2 testrs:PRIMARY> //... testrs:PRIMARY> //I've done a bad thing .... testrs:PRIMARY> //Let me see testrs:PRIMARY> //in the oplog testrs:PRIMARY> use local switched to db local testrs:PRIMARY> db.oplog.rs.findOne({"o.drop": "TheImportantCollection"}) { "ts" : Timestamp(1562042272, 1), "t" : NumberLong(6), "h" : NumberLong("6726633412398410781"), "v" : 2, "op" : "c", "ns" : "payments.$cmd", "ui" : UUID("abc9c1f9-71c0-45ea-aeba-ea239b975a95"), "wall" : ISODate("2019-07-02T04:37:52.171Z"), "o" : { "drop" : "TheImportantCollection" } } testrs:PRIMARY> //AH. 1562042272, you are the worst unix epoch second of my testrs:PRIMARY> // life. testrs:PRIMARY> |
Act 2: Time travel with a Snapshot restore + Oplog replay Plain text version for those who cannot run […]
In my previous blog post “Percona’s View on MongoDB’s 4.2 Release – The Good, the Bad, and the Ugly…” I discussed the release of transaction support in sharded clusters, field-level encryption, search-engine integration, and the new update command syntaxes. Those are all very important, but to me making MongoDB easier to use and removing technical […]
This is part 1 of 2: The keynote-marketed features. Percona’s MongoDB Tech Lead Akira Kurogane takes a look at MongoDB’s 4.2 release. Initial thoughts? Some great! Some not so compelling. Distributed Transactions Including distributed transactions is a great accomplishment, making MongoDB the only popular NoSQL distributed database to plug the gap with this fundamental feature, previously […]