Apache Cassandra, part 2 – data model example, machinery

V. Data model example - Twissandra

Twissandra Use CasesGet the friends of a usernameGet the followers of a usernameGet a timeline of a specific user’s tweetsCreate a tweetCreate a userAdd friends to a user

Twissandra – DB UserUseriduser_namepassword

Twissandra - DB FollowersUserUserFollowersiduser_namepasswordiduser_namepassworduser_idfollower_id

Twissandra - DB FollowingUserUserFollowingiduser_namepasswordiduser_namepassworduser_idfollowing_id

Twissandra – DB TweetsUserTweetiduser_namepasswordiduser_idbodytimestamp

Twissandra column familiesUserUsernameFriends, FollowersTweetUserlineTimeline

Twissandra – Users CF<<CF>> User<<CF>> Username<<RowKey>> userid+ username+ password<<RowKey>> username+ userid

Twissandra–Friends and Followers CFs<<CF>> Friends<<CF>> Followers<<RowKey>> userid<<RowKey>> useridfriendidfolloweridtimestamptimestamp

Twissandra – Tweet CF<<CF>> Tweet<<RowKey>> tweetid + userid + body + timestamp

Twissandra–Userline and Timeline CFs<<CF>> Userline<<CF>> Timeline<<RowKey>> userid<<RowKey>> useridtimestamptimestamptweetidtweetid

Cassandra QL – User creationBATCH BEGIN BATCH INSERT INTO User (KEY, username, password) VALUES (‘id', ‘konstantin’, ‘******’)INSERT INTO Username (KEY, userid) VALUES ( ‘konstantin’, ‘id’)APPLY BATCH

Cassandra QL – following a friendBATCH BEGIN BATCHINSERT INTO Friends (KEY, friendid) VALUES (‘userid‘, ‘friendid’)INSERT INTO Followers (KEY, userid) VALUES (‘friendid ‘, ‘userid’)APPLY BATCH

Cassandra QL – Tweet creation BATCH BEGIN BATCHINSERT INTO Tweet (KEY, userid, body, timestamp) VALUES (‘tweetid‘, ‘userid’, ’@ericflo thanks for Twissandra, it helps!’, 123656459847)INSERT INTO Userline (KEY, 123656459847) VALUES ( ‘userid’, ‘tweetid’)INSERT INTO Timeline (KEY, 123656459847) VALUES ( ‘userid’, ‘tweetid’)……..INSERT INTO Timeline (KEY, 123656459847) VALUES ( ‘followerid’, ‘tweetid’)……APPLY BATCH

Cassandra QL – Getting user tweetsSELECT * FROM Userline KEY = ‘userid’SELECT * FROM Tweet WHERE KEY IN (‘tweetid1’, ‘tweetid2’, ‘tweetid3’, …., ‘tweetidn’)

Cassandra QL – Getting user timelineSELECT * FROM Timeline KEY = ‘userid’SELECT * FROM Tweet WHERE KEY IN (‘tweetid1’, ‘tweetid2’, ‘tweetid3’, …., ‘tweetidn’)

Design patternsMaterialized Viewcreate a second column family to represent additional queriesValueless Columnuse column names for valuesAggregate KeyIf you need to find sub item, use composite key

Indexes<<CF>> Item_Properties<<CF>> Container_Items<<RowKey>> item_id<<RowKey>> container_idproperty_nameitem_idproperty_valueinsertion_timestamp

Indexes<<CF>> Container_Items_Property_Index<<RowKey>> container_id + property_namecomposite(property_value, item_id, entry_timestamp)item_idComparator: compositecomparer.CompositeType

Problem with eventual consistencyWhen we update value, we should add new value to index, and remove old value.However, eventual consistency and lack of transactions make it impossible

Solution<<CF>> Container_Item_Property_Index_Entries<<RowKey>> container_id + item_id + property_nameentry_timestampproperty_value

PartitionersPartitioners decide where a key maps onto the ring.Key 1Key 2Key 3Key 4

PartitionersRandomPartitionerOrderPreservingPartitionerByteOrderedPartitionerCollatingOrderPreservingPartitioner

ReplicationReplication controlled by the replication_factor setting in the keyspace definitionThe actual placement of replicas in the cluster is determined by the Replica Placement Strategies.

Placement StrategiesSimpleStrategy - returns the nodes that are next to each other on the ring.

Placement StrategiesOldNetworkTopologyStrategy - places one replica in a different data center while placing the others on different racks in the current data center.

Placement StrategiesNetworkTopologyStrategy - Allows you to configure the number of replicas per data center as specified in the strategy_options.

SnitchesGive Cassandra information about the network topology of the clusterEndpoint snitch – gives information about network topology.Dynamic snitch – monitor read latencies

Endpoint Snitch ImplementationsSimpleSnitch(default)- can be efficient for locating nodes in clusters limited to a single data center.

Endpoint Snitch ImplementationsRackInferringSnitch - extrapolates the topolology of the network by analyzing IP addresses.192.168.191.71In the same rack192.168.191.21192.168.191.71In the same datacenter192.168.171.21192.78.19.71In different datacenters192.18.11.21

Endpoint Snitch ImplementationsPropertyFileSnitch - determines the location of nodes by referring to a user-defined description of the network details located in the property file cassandra-topology.properties.

sequential writes onlyMemtable no disk access, batched writesSSTable become read‐only

indexesMemtables, SSTables, Commit Logs

Write propertiesWrite propertiesNo readsNo seeksFastAtomic within ColumnFamilyAlways writable

Write/Read propertiesRead propertiesRead multiple SSTablesSlower than writes (but still fast)Seeks can be mitigated with more RAMScales to billions of rows

Commit Log durabilityDurability settings reflects PostgreSQL settings.Periodic sync of commit log. With potential probability for data loss.Batch sync of commit log. Write is acknowledged only if commit log is flushed on disk. It is strongly recommended to have separate device for commit log in such case.

Gossip protocolIntra-ring communicationRuns periodicallyFailure detection,hinted handoffs and nodes exchange

Gossip protocolorg.apache.cassandra.gms.GossiperHas the list of nodes that are alive and deadChooses a random node and starts “chat” with it. One gossip round requires three messagesFailure detection uses a suspicion level to decide whether the node is alive or dead

Hinted handoffWriteHintCassandra is always available for write

TombstonesThe data is not immediately deletedDeleted values are markedTombstones will be suppressed during next compactionGCGraceSeconds – amount of seconds that server will wait to garbage-collect a tombstone

CompactionMerging SSTables into onemerging keyscombining columnscreating new indexMain aims:Free up spaceReduce number of required seeks

CompactionMinor:Triggered when at least N SSTables have been flushed on disk (N is tunable, 4 – by default)Merging SSTables of the similar sizeMajor:Merging all SSTablesDone manually through nodetool compactdiscarding tombstones

Replica synchronizationAnti-entropyRead repair

Anti-entropyDuring major compaction the node exchanges Merkle trees (hash of its data) with another nodesIf the trees don’t match, they are repairedNodes maintain timestamp index and exchange only the most recent updates

Read repairDuring read operation replicas with stale values are brought up to dateWeek consistency level (ONE): after the data is returnedStrong consistency level (QUORUM, ALL): before the data is returned

Bloom filtersA bit arrayTest whether value is a member of setReduce disk access (improve performance)

Bloom filtersOn write:`several hashes are generated per keybits for each hash are markedOn read:hashes are generated for the keyif all bits of this hashes are non-empty then the key may probably exist in SSTableif at least one bit is empty then the key has been never written to SSTable

Bloom filtersReadWrite100Hash1Hash1000Key1Hash2Key2Hash2010Hash31Hash30SSTable

Apache Cassandra, part 2 – data model example, machinery

More Related Content

What's hot (20)

Viewers also liked (6)

Similar to Apache Cassandra, part 2 – data model example, machinery (20)

Recently uploaded (20)

Apache Cassandra, part 2 – data model example, machinery

Editor's Notes