Table of Contents for
Seven NoSQL Databases in a Week

Version ebook / Retour

Cover image for bash Cookbook, 2nd Edition Seven NoSQL Databases in a Week by Xun Wu Published by Packt Publishing, 2018
  1. Seven NoSQL Databases in a Week
  2. Title Page
  3. Copyright and Credits
  4. Seven NoSQL Databases in a Week
  5. Dedication
  6. Packt Upsell
  7. Why subscribe?
  8. PacktPub.com
  9. Contributors
  10. About the authors
  11. Packt is searching for authors like you
  12. Table of Contents
  13. Preface
  14. Who this book is for
  15. What this book covers
  16. To get the most out of this book
  17. Download the example code files
  18. Download the color images
  19. Conventions used
  20. Get in touch
  21. Reviews
  22. Introduction to NoSQL Databases
  23. Consistency versus availability
  24. ACID guarantees
  25. Hash versus range partition
  26. In-place updates versus appends
  27. Row versus column versus column-family storage models
  28. Strongly versus loosely enforced schemas
  29. Summary
  30. MongoDB
  31. Installing of MongoDB
  32. MongoDB data types
  33. The MongoDB database
  34. MongoDB collections
  35. MongoDB documents
  36. The create operation
  37. The read operation
  38. Applying filters on fields
  39. Applying conditional and logical operators on the filter parameter
  40. The update operation
  41. The delete operation
  42. Data models in MongoDB
  43. The references document data model
  44. The embedded data model
  45. Introduction to MongoDB indexing
  46. The default _id index
  47. Replication
  48. Replication in MongoDB
  49. Automatic failover in replication
  50. Read operations
  51. Sharding
  52. Sharded clusters
  53. Advantages of sharding
  54. Storing large data in MongoDB
  55. Summary
  56. Neo4j
  57. What is Neo4j?
  58. How does Neo4j work?
  59. Features of Neo4j
  60. Clustering
  61. Neo4j Browser
  62. Cache sharding
  63. Help for beginners
  64. Evaluating your use case
  65. Social networks
  66. Matchmaking
  67. Network management
  68. Analytics
  69. Recommendation engines
  70. Neo4j anti-patterns
  71. Applying relational modeling techniques in Neo4j
  72. Using Neo4j for the first time on something mission-critical
  73. Storing entities and relationships within entities
  74. Improper use of relationship types
  75. Storing binary large object data
  76. Indexing everything
  77. Neo4j hardware selection, installation, and configuration
  78. Random access memory
  79. CPU
  80. Disk
  81. Operating system
  82. Network/firewall
  83. Installation
  84. Installing JVM
  85. Configuration
  86. High-availability clustering
  87. Causal clustering
  88. Using Neo4j
  89. Neo4j Browser
  90. Cypher
  91. Python
  92. Java
  93. Taking a backup with Neo4j
  94. Backup/restore with Neo4j Enterprise
  95. Backup/restore with Neo4j Community
  96. Differences between the Neo4j Community and Enterprise Editions
  97. Tips for success
  98. Summary
  99. References 
  100. Redis
  101. Introduction to Redis
  102. What are the key features of Redis?
  103. Performance
  104. Tunable data durability
  105. Publish/Subscribe
  106. Useful data types
  107. Expiring data over time
  108. Counters
  109. Server-side Lua scripting
  110. Appropriate use cases for Redis
  111. Data fits into RAM
  112. Data durability is not a concern
  113. Data at scale
  114. Simple data model
  115. Features of Redis matching part of your use case
  116. Data modeling and application design with Redis
  117. Taking advantage of Redis' data structures
  118. Queues
  119. Sets
  120. Notifications
  121. Counters
  122. Caching
  123. Redis anti-patterns
  124. Dataset cannot fit into RAM
  125. Modeling relational data
  126. Improper connection management
  127. Security
  128. Using the KEYS command
  129. Unnecessary trips over the network
  130. Not disabling THP
  131. Redis setup, installation, and configuration
  132. Virtualization versus on-the-metal
  133. RAM
  134. CPU
  135. Disk
  136. Operating system
  137. Network/firewall
  138. Installation
  139. Configuration files
  140. Using Redis
  141. redis-cli
  142. Lua
  143. Python
  144. Java
  145. Taking a backup with Redis
  146. Restoring from a backup
  147. Tips for success
  148. Summary
  149. References
  150. Cassandra
  151. Introduction to Cassandra
  152. What problems does Cassandra solve?
  153. What are the key features of Cassandra?
  154. No single point of failure
  155. Tunable consistency
  156. Data center awareness
  157. Linear scalability
  158. Built on the JVM
  159. Appropriate use cases for Cassandra
  160. Overview of the internals
  161. Data modeling in Cassandra
  162. Partition keys
  163. Clustering keys
  164. Putting it all together
  165. Optimal use cases
  166. Cassandra anti-patterns
  167. Frequently updated data
  168. Frequently deleted data
  169. Queues or queue-like data
  170. Solutions requiring query flexibility
  171. Solutions requiring full table scans
  172. Incorrect use of BATCH statements
  173. Using Byte Ordered Partitioner
  174. Using a load balancer in front of Cassandra nodes
  175. Using a framework driver
  176. Cassandra hardware selection, installation, and configuration
  177. RAM
  178. CPU
  179. Disk
  180. Operating system
  181. Network/firewall
  182. Installation using apt-get
  183. Tarball installation
  184. JVM installation
  185. Node configuration
  186. Running Cassandra
  187. Adding a new node to the cluster
  188. Using Cassandra
  189. Nodetool
  190. CQLSH
  191. Python
  192. Java
  193. Taking a backup with Cassandra
  194. Restoring from a snapshot
  195. Tips for success
  196. Run Cassandra on Linux
  197. Open ports 7199, 7000, 7001, and 9042
  198. Enable security
  199. Use solid state drives (SSDs) if possible
  200. Configure only one or two seed nodes per data center
  201. Schedule weekly repairs
  202. Do not force a major compaction
  203. Remember that every mutation is a write
  204. The data model is key
  205. Consider a support contract
  206. Cassandra is not a general purpose database
  207. Summary
  208. References
  209. HBase
  210. Architecture
  211. Components in the HBase stack
  212. Zookeeper
  213. HDFS
  214. HBase master
  215. HBase RegionServers
  216. Reads and writes
  217. The HBase write path
  218. HBase writes – design motivation
  219. The HBase read path
  220. HBase compactions
  221. System trade-offs
  222. Logical and physical data models
  223. Interacting with HBase – the HBase shell
  224. Interacting with HBase – the HBase Client API
  225. Interacting with secure HBase clusters
  226. Advanced topics
  227. HBase high availability
  228. Replicated reads
  229. HBase in multiple regions
  230. HBase coprocessors
  231. SQL over HBase
  232. Summary
  233. DynamoDB
  234. The difference between SQL and DynamoDB
  235. Setting up DynamoDB
  236. Setting up locally
  237. Setting up using AWS
  238. The difference between downloadable DynamoDB and DynamoDB web services
  239. DynamoDB data types and terminology
  240. Tables, items, and attributes
  241. Primary key
  242. Secondary indexes
  243. Streams
  244. Queries
  245. Scan
  246. Data types
  247. Data models and CRUD operations in DynamoDB
  248. Limitations of DynamoDB
  249. Best practices
  250. Summary
  251. InfluxDB
  252. Introduction to InfluxDB
  253. Key concepts and terms of InfluxDB
  254. Data model and storage engine
  255. Storage engine
  256. Installation and configuration
  257. Installing InfluxDB
  258. Configuring InfluxDB
  259. Production deployment considerations
  260. Query language and API
  261. Query language
  262. Query pagination
  263. Query performance optimizations
  264. Interaction via Rest API
  265. InfluxDB API client
  266. InfluxDB with Java client
  267. InfluxDB with a Python client
  268. InfluxDB with Go client
  269. InfluxDB ecosystem
  270. Telegraf
  271. Telegraf data management
  272. Kapacitor
  273. InfluxDB operations
  274. Backup and restore
  275. Backups
  276. Restore
  277. Clustering and HA
  278. Retention policy
  279. Monitoring
  280. Summary
  281. Other Books You May Enjoy
  282. Leave a review - let other readers know what you think

Python

Now let's try working with Redis from Python. First, make sure to install the redis-py driver:

pip install redis  

It is also recommended to install Hiredis along with redis-py. Hiredis is a parsing class written by the core Redis team, and redis-py will use Hiredis if it is present. Its use can provide a significant performance gain over the default PythonParser. The hiredis parser can be installed similarly via pip:

pip install hiredis  

Let's write a simple script to set, get, and output a simple welcome message. Create a new Python script and call it redisHelloWorld.py.

First, we will add our imports. We will need StrictRedis, as well as the sys module to pull in command-line arguments:

from redis import StrictRedis 
import sys 

Now we will pull in our hostname and password from the command-line arguments:

hostname = sys.argv[1] 
password = sys.argv[2]

Next, we will pass along the hostname and password to a local object to hold our StrictRedis connection data. Since the port won't be changing much, we'll hardcode that to be the default port of 6379. Additionally, we'll make sure we're sticking to database 0 (zero) by hardcoding that as well. But both port and db could very well be converted to command-line parameters as well:

r = StrictRedis(host=hostname,port=6379,password=password,db=0) 

Here we will create a method to set the key packt:welcome to our message of Hello world from Python!:

def setPacktWelcome(): 
    #SET new value packt:welcome 
    print("Writing \"Hello world from Python!\" to Redis...") 
    r.set('packt:welcome','Hello world from Python!') 

Next, we will create a method to query that message and print it as output:

def getPacktWelcome(): 
    #GET value stored in packt:welcome 
    print("Displaying current welcome message...") 
    value = r.get('packt:welcome') 
    print("message = " + str(value)) 

Finally, we will call the get, set, and then get once more. In this way, we are querying the packt:welcome key for its value, setting it to a new value, and then querying it again and posting the value as output:

getPacktWelcome() 
setPacktWelcome() 
getPacktWelcome() 

Running this from the command line yields the following output:

python redisHelloWorld.py 127.0.0.1 currentHorseBatteryStaple
Displaying current welcome message...
message = Hello world!
Writing "Hello world from Python!" to Redis...
Displaying current welcome message...
message = Hello world from Python!

As you can see, querying the key packt:welcome shows our initial value that we set in the redis-cli example. Next, we're setting the value to Hello world from Python! Finally, we query the same key and see our new message.

Now we'll try something a little different. Let's write a script to manage our entries into the packt:logins list that we have created, and keep the list to a size of three. We'll call this script redisQueryUser.py.

Our imports will be similar to the previous script, except that we'll include the datetime.datetime item:

from redis import StrictRedis 
from datetime import datetime 
import sys 

In addition to passing the hostname and password for our Redis instance, we will also pass the userid and ip address. Additionally, we will connect to Redis the same way that we did in the previous script:

hostname=sys.argv[1] 
password=sys.argv[2] 
userid=sys.argv[3] 
ip=sys.argv[4] 
r = StrictRedis(host=hostname,port=6379,password=password,db=0) 

Next, we will create a new method to log entries for the userid and ip address. This method will also log the current time, concatenate them all into a string, and execute a lpush onto the packt:logins list structure:

def addNewLogin(user,ipaddress): 
    print("Logging entry for " + user + " from " + ipaddress) 
    time = str(datetime.now()) 
    r.lpush('packt:logins',user + " " + ipaddress + " " + time) 

For the final command of that method, I'll execute a ltrim on the list. We only want to keep track of the last three user logins into our system. So, trimming the list (again from the left) from items zero to two ensures that there will only ever be three items in the list and that they will always be the three most recent entries:

r.ltrim('packt:logins',0,2) 

Next, we'll write a simple method to query the complete list, and output it to the screen:

def getList(): 
    list = r.lrange('packt:logins',0,-1) 
    print(list)

Finally, we will call both methods. This will add a new entry to the list, and query the list output:

addNewLogin(userid,ip) 
getList() 

Running the script yields this output:

python redisQueryUser.py 127.0.0.1 currentHorseBatteryStaple aploetz 10.0.0.6
Logging entry for aploetz from 10.0.0.6
['aploetz 10.0.0.6 2017-06-24 16:43:29.228135', 'aploetz 10.0.0.4 2017-06-24 16:31:58.171875', 'aploetz 10.0.0.4 2017-06-24 16:22:04.144998']  

After running the script one more time, I can query it from within redis-cli to see its current contents:

127.0.0.1:6379> lrange packt:logins 0 -1
1) "aploetz 10.0.0.9 2017-06-24 16:43:49.958260"
2) "aploetz 10.0.0.6 2017-06-24 16:43:29.228135"
3) "aploetz 10.0.0.4 2017-06-24 16:31:58.171875"  

As you can see, of the original two entries (done from the command line), only one remains. This affirms that our code to keep the list to a size of three is working correctly.

Next, let's write two scripts to demonstrate the Publish/Subscribe functionality of Redis. Let's assume that we are building a messaging application for baseball games. We'll start by creating our script (named redisPub.py) with our normal imports and argument processing statements:

#!/usr/bin/python 
from redis import StrictRedis 
import sys 
 
if len(sys.argv) < 4: 
    print "Please enter a valid hostname, password, and channel." 
    exit(len(sys.argv)) 
 
hostname=sys.argv[1] 
password=sys.argv[2] 
channel=sys.argv[3] 

Next, we'll set up our connection to Redis, and instantiate a local object named publisher to manage our message publishing:

r = StrictRedis(host=hostname,port=6379,password=password,db=0) 
publisher = r.pubsub()

With that complete, let's build a while loop to show a prompt, process our input, and publish the messages:

while True: 
    message=raw_input("Describe play, or press [Enter] to quit: ") 
 
    if not message: 
        break 
    else: 
        r.publish(channel,message) 

print
"Publish program ended."

Now let's shift gears to our subscriber script. We will name it redisSub.py. It will start with the same imports and argument processing statements as the previous script did:

#!/usr/bin/python 
from redis import StrictRedis 
import sys 
 
if len(sys.argv) < 4: 
    print "Please enter a valid hostname, password, and channel." 
    exit(len(sys.argv)) 
 
hostname=sys.argv[1] 
password=sys.argv[2] 
channel=sys.argv[3] 

Next, we'll set up our connection to Redis, and instantiate a local object for our channels. Additionally, we'll issue a command to subscribe to the provided channel:

r = StrictRedis(host=hostname,port=6379,password=password,db=0) 
channels = r.pubsub() 
channels.subscribe(channel) 

With that complete, we can now build a for loop to listen for messages. We will use an if construct to see if the message consists of the uppercase word END. If it does, we'll break from the loop. Otherwise, we will print the message contents:

for message in channels.listen():
if message['data']=='END': break else: print message['data']

channels.unsubscribe(channel)
print "Unsubscribed"

Now let's see these scripts in action! As with our redis-cli Publish/Subscribe example, we'll need to open two terminal windows: one for the subscriber and one for the publisher. Starting with the subscriber, let's invoke it with our localhost (127.0.0.1), our password, and OMG15UBaseball as our channel:

python redisSub.py 127.0.0.1 currentHorseBatteryStaple OMG15UBaseball         1 

After running that command, Redis returns a 1 as our first message, indicating that we are listening on a single channel. Next, let's run the publisher in another terminal window:

python redisPub.py 127.0.0.1 currentHorseBatteryStaple OMG15UBaseball
        
Describe play; [Enter] to quit: Henry strikes out swinging (Out 1).
     
Describe play; [Enter] to quit: Avery singles to RF.
        
Describe play; [Enter] to quit: Avery steals 2b.
        
Describe play; [Enter] to quit: Gareth doubles to LF, Avery scores.
        
Describe play; [Enter] to quit: Gareth steals 3b.
        
Describe play; [Enter] to quit: Leighton grounds out to SS, Gareth scores (Out 2).
        
Describe play; [Enter] to quit: Aaron grounds out to SS (Out 3).
        
Describe play; [Enter] to quit: END
        
Describe play; [Enter] to quit:
   
Publish program ended.  

When we look at the terminal running our subscriber, the following output is displayed:

1    
Henry strikes out swinging (Out 1).
       
Avery singles to RF.
       
Avery steals 2b.
       
Gareth doubles to LF, Avery scores.
       
Gareth steals 3b.
      
Leighton grounds out to SS, Gareth scores (Out 2).
       
Aaron grounds out to SS (Out 3).
Unsubscribed  
Complete code for all scripts can be found at: https://github.com/aploetz/packt/.