-
Notifications
You must be signed in to change notification settings - Fork 81
Description
We have a database where we store information for some IOT devices.
Now our devices routinely send out a value every [5 or 60 or 300] seconds. I'm using a firestore (native mode) database to save these values.
I've got a program which aggregates the messages and updates all the documents every ~10 minutes with all messages received.
The structure of the database is as following:
- sensors (collection)
- sensor1 (document)
- sensor_readings (subcollection)
- readings_10-05-2021 (document)
- readings_11-05-2021 (document)
- readings_12-05-2021 (document)
- sensor2
- etc.
The document "readings_10-05-2021" will have a map with the values
- readings_10-05-2021
- values (map)
- "00:00:01" : 120
- "00:01:01" : 121
- "00:02:01" : 122
- etc.
I'm trying to retreive this document with:
db.collection("sensors").document("sensor1").collection("sensor_readings").document("readings_10-05-2021").get()
When the map has only 1 value it takes about ~.19 seconds
When the map has 1440 values it takes about ~1.5 seconds
When the map has 19200 values it takes about ~400 seconds
I've tested this with documents of the roughly same size in bytes to account for download speeds.
note: fetching this document with the large map on the google cloud terminal takes ~1 second.
As advised by @samtstern posting this here.
Environment details
Python 3.9.2
pip 21.1.1
google-cloud-firestore 2.1.1