Redis : NoSQL key-data store

1
La 1ère école 100 % dédiée à l'open source
Open Source School est fondée à l'initiative de Smile, leader de
l'intégration et de l'infogérance open source, et de l'EPSI,établissement
privé pionnier de l’enseignement supérieur en informatique.
Dans le cadre du Programme d’Investissements d’Avenir (PIA), le
gouvernement français a décidé de soutenir la création de cette école en
lui attribuant une première aide de 1,4M€ et confirme sa volonté de
soutenir la filière du Logiciel Libre actuellement en plein développement.
Avec une croissance annuelle de plus de 10%, et 4 000
postes vacants chaque année dans le secteur du Logiciel
Libre, OSS entend répondre à la pénurie de compétences du
secteur en mobilisant l’ensemble de l’écosystème et en
proposant la plus vaste offre en matière de formation aux
technologies open source tant en formation initiale qu'en
formation continue.

2
Les formations du plein emploi !
 Formation Continue
Open Source School "Executive Education" est un organisme
de formation qui propose un catalogue de plus de 200
formations professionnelles et différents dispositifs de
reconversion permettant le retour à l’emploi (POE) ou une
meilleure employabilité pour de nombreux professionnels de
l’informatique.
 Pour vos demandes : formations@opensourceschool.fr
 Formation Initiale
100% logiciels libres et 100% alternance, le cursus Open
Source School s’appuie sur le référentiel des blocs de
compétences de l’EPSI.
Il est sanctionné par un titre de niveau I RNCP, Bac+5.
Le programme est proposé dans 6 campus à Bordeaux, Lille,
Lyon, Montpellier, Nantes, Paris.

NoSQL introduction Redis overview Installing and conﬁguring Redis Types and commands Redis in your applica
Plan
1 NoSQL introduction
2 Redis overview
3 Installing and conﬁguring Redis
4 Types and commands
5 Redis in your applications
6 Questions ?
www.opensourceschool.fr – Licence Creative Commons (CC BY-SA 3.0 FR) – 2/63

NoSQL introduction

Why not a RDBMS
Short answer : because performance
Computer science is all about the integrity/performance
tradeoﬀ
RDBMS choose integrity
This is important, but sometimes you don’t care

Not Only SQL
Business always needs some integrity, why not split the data by
type and constraints?
Traditional design Using not only SQL

Redis overview

What is Redis?
Redis is:
an advanced key-value store
an in-memory non-relational database
a data structure server

Identity Card
Created by: Salvatore Sanﬁlippo (Italian)
ﬁrst release: April 10, 2009
Written in: C
Licence: BSD
Last Realease: 2.8.9 - April 22, 2014
Sponsored by: VMware until May 2013, Pivotal Software after

Useful links
Oﬃcial Website: http://redis.io/
Sources: https://github.com/antirez/redis
Community Group:
https://groups.google.com/forum/?fromgroups#!forum/redis-db

Interesting features
Data structures
Atomic operations
Conﬁgurable persistence
Asynchronous replication
TTL and LRU (cache-like behavious)
Pub/Sub

What the website won’t tell you
Redis is mono threaded, a single process cannot scale to
multiple CPUs
Sharding is possible if the client supports it
Redis better used for making persistent caches, advanced
caches (tags) or session storage

Redis vs Memcached
Memcached is only in memory, Redis can persist the data
Memcached has only 1 namespace, Redis can have multiple
namespaces
Memcached can only store data in plain string, Redis has 5
types of structure
Memcached has limited data manipulation commands, Redis
has an advanced query language

Installing and conﬁguring Redis

Installation
Installation

Installation
Version
Redis is moving fast
Make sure you’re using at least version 2.4

Installation
Install on Debian
For Wheezy and above :
aptitude install redis-server
For Squeeze, you need to activate the ”backports” repository
aptitude install -t squeeze-backports
redis-server

Installation
Install on RHEL/CentOS 6
First you need to activate the EPEL repository
Install the package :
yum install redis
Activate and start the service :
chkconfig redis on && service redis start

Installation
Configuration
The configuration file is in /etc/redis/redis.conf
(/etc/redis.conf on RHEL/CentOS)
You might want to change the listening address :
bind 0.0.0.0
Configure redis as a cache
maxmemory 2147483648
maxmemory-policy allkeys-lru

Installation
Databases
One redis instance can have multiple namespaces
They are called ”databases”
They are referenced by an integer
By default, redis has 16 databases
Database numbering begins at 0
use SELECT N to change databases in redis-cli

Persistency
Persistency

Persistency
Persistency methods
2 methods:
snapshoting data in memory to disk
append-only ﬁle (AOF) write/delete commands saved in order
in a ﬁle

Persistency
Snapshoting conﬁg
#path of the snapshot file
dbfilename dump.rdb
#compress the data
rdbcompression yes
#if snapshot can’t be performed writes are
#suspended until a manual operation fix the problem
stop-writes-on-bgsave-error no

Persistency
initiate a snaphot
5 methods to initiate a snapshot:
command BGSAVE sent by a client, redis will fork and child
make the snapshot in parallel the father continue to respond
for client requests
command SAVE sent by a client, redis stop processing all
client request and make a snapshot
command SHUTDOWN or a TERM signal, redis launch an internal
SAVE command before stopping
save <sec> <nb> parameter in conﬁg ﬁle, if the condition
match redis launch an internal BGSAVE command
a SYNC command received from another redis server, redis
launch an internal BGSAVE command if one isn’t already
executing or recently completed

Persistency
AOF conﬁg
general conﬁguration information:
#activate AOF
appendonly <yes|no>
#manage writes on disk
appendfsync <always|everysec|no>
#disable sync on rewrite
no-appendfsync-on-rewrite no
#pourcentage of the current file size against
#file size after last rewrite
auto-aof-rewrite-percentage 100
#minimum size of the file before lauching a rewrite
auto-aof-rewrite-min-size 64mb

Scaling
Scaling

Scaling
Replication
Master/Slave architecture
steps:
slave send a SYNC command to the master
master launch a BGSAVE and keep in a backlog all the write
commands received after
master send the snapshot to the slave
slave discard all old data and load the new snapshot
when the new snapshot is loaded master send all the backlog
to the slave
master live stream all the write commands to the slave

Scaling
Replication Config
Replication is asynchronous
It’s easy to setup :
add in config file slaveof <masterHostname>
<masterPort>
or send command SLAVEOF <masterHostname>
<masterPort>
Stop it with : SLAVEOF NO ONE
Check it with : INFO
Look for role and master link status keys

Types and commands

Key Naming
Best practice is to name your key with an object name followed by
an id separated by : or , or / and to keep the same rule for all the
pair in a database.
post:254
article.10
session/45656665

Structure Types
Structure Types

Structure Types
STRING
strings, integers, ﬂoating-point values (max 512 MB)
basic commands:
GET: fetch data at the given key
SET: set data at the given key
DEL: delete the data at the given key

Structure Types
STRING: example
redis 127.0.0.1:6379> set hello world
OK
redis 127.0.0.1:6379> get hello
"world"
redis 127.0.0.1:6379> del hello
(integer) 1
redis 127.0.0.1:6379> get hello
(nil)

Structure Types
STRING: Increment and decrement
if the value is an integer or a ﬂoating-point
INCR: add 1
DECR: minus 1
INCRBY/DECRBY: add/minux X
INCRBYFLOAT: add ﬂoat

Structure Types
STRING: substring manipulation
if the value is a string
APPEND: concatenate the value with the string given
GETRANGE: get a substring of the value
SETRANGE: replace the substring value starting at the oﬀset X
by the string given
GETBIT, SETBIT, BITCOUNT, BITOP: binary manipulation of
the value

Structure Types
LIST
linked list of strings (max. 2ˆ32 -1)
basic commands:
LPUSH, RPUSH: push the value(s) onto the left/right end of
the list
LRANGE: fetch a range of value from the list
LINDEX: fetch an item at a given position in the list
LPOP, RPOP: remove and return the value from the left/right
end of the list
LTRIM: trim the list to only include items at position between
start and end, inclusive

Structure Types
LIST: example
redis 127.0.0.1:6379> rpush ingredients egg
(integer) 1
redis 127.0.0.1:6379> rpush ingredients milk
(integer) 2
redis 127.0.0.1:6379> rpush ingredients egg
(integer) 3
redis 127.0.0.1:6379> lrange ingredients 0 -1
1) "egg"
2) "milk"
3) "egg"
redis 127.0.0.1:6379> lindex ingredients 1
"milk"
redis 127.0.0.1:6379> lpop ingredients
"egg"
redis 127.0.0.1:6379> lrange ingredients 0 -1
1) "milk"
2) "egg"

Structure Types
LIST: blocking and moving
BLPOP, BRPOP: pop the left/right most item of the ﬁrst non
empty list given or wait the timeout to get an item
RPOPLPUSH: pop the righ most item from the ﬁrst list, left
push it to the second list and send it back to the user
BRPOPLPUSH: same as RPOPLPUSH wait the timeout to get an
item in the source list if its empty

Structure Types
SET
Unordered collection of unique strings (max. 2ˆ32 -1)
basic commands:
SADD: add item(s) to the set
SMEMBERS: return all the items contains in the set
SISMEMBER: check if an item is present in the set
SCARD: number of items in the set
SREM: remove item(s) from the set
SRANDMEMBER: return randomly x items of the set, if X is
positive the will be distinct, if X is negative a same value
could be sent more than one time
SPOP: remove and return a random item of the set
SMOVE: move an item from a set to aonther set and return it
to the user

Structure Types
LIST: example
redis 127.0.0.1:6379> sadd ingredients sugar
(integer) 1
redis 127.0.0.1:6379> sadd ingredients floor
(integer) 1
redis 127.0.0.1:6379> sadd ingredients butter
(integer) 1
redis 127.0.0.1:6379> sadd ingredients sugar
(integer) 0
redis 127.0.0.1:6379> smembers ingredients
1) "sugar"
2) "floor"
3) "butter"
redis 127.0.0.1:6379> sismember ingredients chocolate
(integer) 0
redis 127.0.0.1:6379> sismember ingredients sugar
(integer) 1
redis 127.0.0.1:6379> srem ingredients floor
(integer) 1
redis 127.0.0.1:6379> srem ingredients floor
(integer) 0
redis 127.0.0.1:6379> smembers ingredients

Structure Types
SET: combining and manipulating
SDIFF: make the diﬀerence of X sets and return the result
SDIFFSTORE: make the diﬀerence of X sets and store the
result in another set
SINTER: make the intersection of X sets and return the result
SINTERSTORE: make the intersection of X sets and store the
result in another set
SUNION: returns the items that are in at least one of the SETs
SUNIONSTORE: store in a given set the items that are in at
least one of the SETs

Structure Types
HASH
Unordered hash table of keys to values
basic commands:
HSET: set a value at a given key in a hash (HMSET to multiple
key)
HGET: get a value at a given key in a hash (HSET to multiple
key)
HGETALL: fetch all the hash
HDEL: remove a key from a hash
HLEN: return the number of pair in the hash
HEXISTS: check if key exist in the hash
HKEYS: return the list of keys
HVALS: return the list of values
HINCRBY/HINCRBYFLOAT: add X to the value of a key

Structure Types
HASH: example
redis 127.0.0.1:6379> hset ingredients eggs 2
(integer) 1
redis 127.0.0.1:6379> hset ingredients floor 250
(integer) 1
redis 127.0.0.1:6379> hset ingredients eggs 2
(integer) 0
redis 127.0.0.1:6379> hgetall ingredients
1) "eggs"
2) "2"
3) "floor"
4) "250"
redis 127.0.0.1:6379> hdel ingredients floor
(integer) 1
redis 127.0.0.1:6379> hdel ingredients floor
(integer) 0
redis 127.0.0.1:6379> hget ingredients eggs
"2"
redis 127.0.0.1:6379> hgetall ingredients
1) "eggs"

Structure Types
ZSET
Ordered mapping of -string- members (keys) to -ﬂoating-point-
scores (values), ordered by score ﬁrst then by member name can
be accessed by member or order number of score or score
basic commands:
ZADD: add member with a given score to the ZSET
ZRANGE: fetch the items by theire position in sorted order
ZRANGEBYSCORE: fetch the items by score range
ZCARD: number of members
ZINCRBY: increment the score of a member
ZCOUNT: return the number of members with a score between
the provided arguments
ZRANK: position of the given member
ZSCORE: get the score of a member

Structure Types
HASH: example
redis 127.0.0.1:6379> zadd ingredients 110 sugar
(integer) 1
redis 127.0.0.1:6379> zadd ingredients 250 floor
(integer) 1
redis 127.0.0.1:6379> zadd ingredients 250 floor
(integer) 0
redis 127.0.0.1:6379> zrange ingredients 0 -1 withscores
1) "sugar"
2) "110"
3) "floor"
4) "250"
redis 127.0.0.1:6379> zrangebyscore ingredients 0 200 withscores
1) "sugar"
2) "110"
redis 127.0.0.1:6379> zrem ingredients sugar
(integer) 1
redis 127.0.0.1:6379> zrem ingredients sugar
(integer) 0
redis 127.0.0.1:6379> zrange ingredients 0 -1 withscores

Sort
Sort

Sort
Sort
can sort LIST, SET, ZSET
redis 127.0.0.1:6379> rpush sort-input 23 15 110 7
4
redis 127.0.0.1:6379> sort sort-input
[’7’, ’15’, ’23’, ’110’]
redis 127.0.0.1:6379> sort sort-input alpha
[’110’, ’15’, ’23’, ’7’]
redis 127.0.0.1:6379> hset d-7 field 5
1L
1L
1L
1L
redis 127.0.0.1:6379> sort sort-input by d-*->field
[’15’, ’110’, ’7’, ’23’]
redis 127.0.0.1:6379> sort sort-input by d-*->field’ get d-*->field
[’1’, ’3’, ’5’, ’9’]

Transactions
Transactions

Transactions
Transactions
Pipeline of commands:
When redis receive the command MULTI it store the following
commands
When it receive EXEC it execute all the commands receivend
in a single transaction
MULTI
<command1>
<command2>
<command3>
EXEC

Transactions
optimistic locking
if the data is changed before EXEC the client is notiﬁed of the
update and can change is action accordingly
UNWATCH ¡key¿ can be used after a WATCH and before a MULTI
to cancel the optimistic locking
DISCARD ¡key¿ can be used after a WATCH and MULTI to
cancel the optimistic locking and the transaction
WATH <key>
MULTI
<command1>
<command2>
<command3>
EXEC

Expiration
Expiration

Expiration
Expiration
expiration on duration or point in time of a key and all of it’s
content
EXPIRE: set the expiration duration in seconds
EXPIREAT: set the expiration time with a unix timestamp
TTL: return the amount of time remaining before a key will
expire
PERSIST: remove the expiration for a key
PTTL: return the amount of time in milliseconds remaining
before a key will expire
PEXPIRE: set the expiration duration in milliseconds
PEXPIREAT: set the expiration time with a unix timestamp in
milliseconds

Publish / subscribe
Publish / subscribe

Publish / subscribe
Publish / subscribe
A consumer receive all the data sent on a subscribed channel
SUBSCRIBE: subscribe to a channel
UNSUBSCRIBE: unsubscribe from a channel
PUBLISH: publish message to a channel
PSUBSCRIBE: subscribe to channels witha given pattern
PUNSUBSCRIBE: unsubscribe to channels witha given pattern

Publish / subscribe
warning
2 reasons to not use it:
if client is not reading or too slowly it could cause old Redis
version to slow down or crash
if there is network failure the message sent during the failure
will be lost
Use blocking list instead or a real MQ application

Scripting
Scripting

Scripting
Scripting
Redis 2.6 introduce scripting with Lua scriping language
wins:
execute complexe code on server-side
implement missing functionnalities like locking
performance
drawbacks:
block during execution all other client request (single thread)
type mapping is not complete
can’t named a script use sha1 digest instead

Redis in your applications

Redis and PHP
Redis and PHP

Redis and PHP
The phpredis driver
The most widely used solution to connect to redis from PHP
Distributed by PECL under the name redis
Distributed on the Smile Hosting repositories
apt-get install php5-redis
Has support for sharding (”arrays”)
Object-oriented interface

Redis and PHP
PHP session handler
[PHP]
session.save_handler = redis
session.save_path = "tcp://host1:6379?timeout=3,database=1"
Multiple hosts can be given
Timing out on the session manager is fatal

Redis and Magento
Redis and Magento

Redis and Magento
Redis and Magento
Natively supported from Magento 1.13
Smile-provided module for Magento 1.12
Can be used as session manager
Persistent sessions
Replication
Can be used as cache manager
Advanced data types allow to store the entire cache
No more two-level cache
No more tags in databases
Check the wiki for how-to and caveats

Questions ?

Redis : NoSQL key-data store

More Related Content

Similar to Redis : NoSQL key-data store (20)

More from Open Source School (13)

Recently uploaded (20)

Redis : NoSQL key-data store