American Institute of Mathematical Sciences

Advanced
July  2016, 1(2&3): 185-216. doi: 10.3934/bdia.2016004

How do I choose the right NoSQL solution? A comprehensive theoretical and experimental survey

Hamzeh Khazaei 1, , Marios Fokaefs 1, , Saeed Zareian 1, , Nasim Beigi-Mohammadi 1, , Brian Ramprasad 1, , Mark Shtern 1, , Purwa Gaikwad 1, and  Marin Litoiu 1,

1. 

Center of Excellence for Research in Adaptive Systems, York University, Toronto, Ontario, Canada, Canada, Canada, Canada, Canada, Canada

Received  October 2015 Revised  December 2015 Published  September 2016

With the advent of the Internet of Things (IoT) and cloud computing,the need for data stores that would be able to store and process big data inan ecient and cost-e ective manner has increased dramatically. Traditionaldata stores seem to have numerous limitations in addressing such requirements.NoSQL data stores have been designed and implemented to address the shortcomingsof relational databases by compromising on ACID and transactionalproperties to achieve high scalability and availability. These systems are designedto scale to thousands or millions of users performing updates, as wellas reads, in contrast to traditional RDBMSs and data warehouses. Althoughthere is a plethora of potential NoSQL implementations, there is no one-size- t-all solution to satisfy even main requirements. In this paper, we explorepopular and commonly used NoSQL technologies and elaborate on their documentation,existing literature and performance evaluation. More speci cally,we will describe the background, characteristics, classi cation, data model andevaluation of NoSQL solutions that aim to provide the capabilities for big dataanalytics. This work is intended to help users, individuals or organizations,to obtain a clear view of the strengths and weaknesses of well-known NoSQLdata stores and select the right technology for their applications and use cases.To do so, we rst present a systematic approach to narrow down the properNoSQL candidates and then adopt an experimental methodology that can berepeated by anyone to nd the best among short listed candidates consideringtheir speci c requirements.
Keywords: key-value datastores, performance evaluation, YCSB, document datastores, NoSQL datastores, future trends, wide-column datastores, systematical selection, graph databases, distributed datastores..
Mathematics Subject Classification: Primary: 68P20, 68P15; Secondary: 68M20, 68M1.
Citation: Hamzeh Khazaei, Marios Fokaefs, Saeed Zareian, Nasim Beigi-Mohammadi, Brian Ramprasad, Mark Shtern, Purwa Gaikwad, Marin Litoiu. How do I choose the right NoSQL solution? A comprehensive theoretical and experimental survey. Big Data & Information Analytics, 2016, 1 (2&3) : 185-216. doi: 10.3934/bdia.2016004
References:
[1]

Y. Abubakar, T. S. Adeyi and I. G. Auta, Performance evaluation of nosql systems using ycsb in a resource austere environment, Performance Evaluation, 7 (2014), 23-27. doi: 10.5120/ijais14-451229.  Google Scholar

[2]

P. Andlinger, 2015,, URL , ().   Google Scholar

[3]

Apache Software Foundation, Apache tinkerpop, 2015,, URL , ().   Google Scholar

[4]

Apache Software Foundation, Technical overview of apache couchdb, 2015,, URL , ().   Google Scholar

[5]

ArangoDB GmbH, Arangodb documentation, 2015,, URL , ().   Google Scholar

[6]

Aurelius LLC, Titan architecture overview, 2015,, URL , ().   Google Scholar

[7]

Basho Technologies, Inc, Riak docs, 2015,, URL , ().   Google Scholar

[8]

M. Burrows, The chubby lock service for loosely-coupled distributed systems, in Proceedings of the 7th symposium on Operating systems design and implementation, USENIX Association, 2006, 335-350. Google Scholar

[9]

R. Casado and M. Younas, Emerging trends and technologies in big data processing, Concurrency and Computation: Practice and Experience, 27 (2015), 2078-2091. doi: 10.1002/cpe.3398.  Google Scholar

[10]

R. Cattell, Scalable sql and nosql data stores, ACM SIGMOD Record, 39 (2010), 12-27. doi: 10.1145/1978915.1978919.  Google Scholar

[11]

F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes and R. E. Gruber, Bigtable: A distributed storage system for structured data, ACM Transactions on Computer Systems (TOCS), 26 (2008), p4. doi: 10.1145/1365815.1365816.  Google Scholar

[12]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan and R. Sears, Benchmarking cloud serving systems with ycsb, in Proceedings of the 1st ACM symposium on Cloud computing, ACM, 2010, 143-154. doi: 10.1145/1807128.1807152.  Google Scholar

[13]

S. Edlich, A. Friedland, J. Hampe, B. Brauer, M. Brückner, S. Edlich, A. Friedland, J. Hampe, B. Brauer and M. Brückner, Brückner,, Nosql., ().   Google Scholar

[14]

A. Feinberg, Project voldemort: Reliable distributed storage, in Proceedings of the 10th IEEE International Conference on Data Engineering, 2011. Google Scholar

[15]

B. Fitzpatrick, Distributed caching with memcached, Linux journal, 2004 (2004), p5. Google Scholar

[16]

S. K. Gajendran, A survey on nosql databases,, University of Illinois., ().   Google Scholar

[17]

J. Gray, Graysort benchmark, 2015,, URL , ().   Google Scholar

[18]

Hibernating Rhinos, Ravendb - the open source nosql database for .NET, 2015,, URL , ().   Google Scholar

[19]

Hypertable Inc, Hypertable, 2014,, URL , ().   Google Scholar

[20]

S. IT, Knowledge base of relational and nosql database management systems, 2015,, URL , ().   Google Scholar

[21]

S. IT, System properties comparison neo4j vs. orientdb vs. titan, 2015,, URL , ().   Google Scholar

[22]

J. Jose, H. Subramoni, M. Luo, M. Zhang, J. Huang, M. Wasi-ur Rahman, N. S. Islam, X. Ouyang, H. Wang, S. Sur et al., Memcached design on high performance rdma capable interconnects, in Parallel Processing (ICPP), 2011 International Conference on, IEEE, 2011, 743-752. doi: 10.1109/ICPP.2011.37.  Google Scholar

[23]

S. Jouili and V. Vansteenberghe, An empirical comparison of graph databases, in Social Computing (SocialCom), 2013 International Conference on, 2013, 708-715. doi: 10.1109/SocialCom.2013.106.  Google Scholar

[24]

J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham and C. Matser, Performance evaluation of nosql databases: A case study, in Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, PABS '15, ACM, New York, NY, USA, 2015, 5-10. doi: 10.1145/2694730.2694731.  Google Scholar

[25]

LinkedIn, Project voldemort, 2015,, URL , ().   Google Scholar

[26]

R. C. McColl, D. Ediger, J. Poovey, D. Campbell and D. A. Bader, A performance evaluation of open source graph databases, in Proceedings of the First Workshop on Parallel Programming for Analytics Applications, PPAA '14, ACM, New York, NY, USA, 2014, 11-18. doi: 10.1145/2567634.2567638.  Google Scholar

[27]

MongoDB Inc., Mongodb 3.0 manual, 2015,, URL , ().   Google Scholar

[28]

A. Moniruzzaman and S. A. Hossain, Nosql database: New era of databases for big data analytics-classification, characteristics and comparison,, arXiv preprint arXiv:1307.0191., ().   Google Scholar

[29]

M. A. Olson, K. Bostic and M. I. Seltzer, Berkeley db., in USENIX Annual Technical Conference, FREENIX Track, 1999, 183-191. Google Scholar

[30]

Orient Technologies, Top 10 key advantages for going with orientdb, 2015,, URL , ().   Google Scholar

[31]

A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden and M. Stonebraker, A comparison of approaches to large-scale data analysis, in Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, ACM, 2009, 165-178. doi: 10.1145/1559845.1559865.  Google Scholar

[32]

D. Pritchett, Base: An acid alternative, Queue, 6 (2008), 48-55. doi: 10.1145/1394127.1394128.  Google Scholar

[33]

T. Rabl, A. Ghazal, M. Hu, A. Crolotte, F. Raab, M. Poess and H.-A. Jacobsen, Bigbench specification v0. 1, in Specifying Big Data Benchmarks, Springer, 2014, 164-201. Google Scholar

[34]

RedisLabs, Redis, 2015,, URL , ().   Google Scholar

[35]

SAVI, Smart Applications on Virtual Infrastructure, Cloud platform, 2015, URL http://www.savinetwork.ca. Google Scholar

[36]

S. Sivasubramanian, Amazon dynamodb: A seamlessly scalable non-relational database service, in Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, ACM, 2012, 729-730. doi: 10.1145/2213836.2213945.  Google Scholar

[37]

C. Strozzi, Nosql-a relational database management system, 2015,, URL , ().   Google Scholar

[38]

Technology, Cypher query language, 2015,, URL , ().   Google Scholar

[39]

The Apache Foundation, Apache accumulo, 2015,, URL , ().   Google Scholar

[40]

The Apache Foundation, Welcome to apache cassandra, 2015,, URL , ().   Google Scholar

[41]

The Apache Foundation, Welcome to apache hbase, 2015,, URL , ().   Google Scholar

[42]

A. Tizghadam and A. Leon-Garcia, Connected Vehicles and Smart Transportation - CVST Platform, 2015,, URL , ().   Google Scholar

[43]

G. Vaish, Getting started with NoSQL, Packt Publishing Ltd, 2013. Google Scholar

[44]

vsChart.com, The comparison wiki: Database list, 2015,, URL , ().   Google Scholar

[45]

P. Wiki, Pig mix benchmark, 2015,, URL, ().   Google Scholar

show all references

References:
[1]

Y. Abubakar, T. S. Adeyi and I. G. Auta, Performance evaluation of nosql systems using ycsb in a resource austere environment, Performance Evaluation, 7 (2014), 23-27. doi: 10.5120/ijais14-451229.  Google Scholar

[2]

P. Andlinger, 2015,, URL , ().   Google Scholar

[3]

Apache Software Foundation, Apache tinkerpop, 2015,, URL , ().   Google Scholar

[4]

Apache Software Foundation, Technical overview of apache couchdb, 2015,, URL , ().   Google Scholar

[5]

ArangoDB GmbH, Arangodb documentation, 2015,, URL , ().   Google Scholar

[6]

Aurelius LLC, Titan architecture overview, 2015,, URL , ().   Google Scholar

[7]

Basho Technologies, Inc, Riak docs, 2015,, URL , ().   Google Scholar

[8]

M. Burrows, The chubby lock service for loosely-coupled distributed systems, in Proceedings of the 7th symposium on Operating systems design and implementation, USENIX Association, 2006, 335-350. Google Scholar

[9]

R. Casado and M. Younas, Emerging trends and technologies in big data processing, Concurrency and Computation: Practice and Experience, 27 (2015), 2078-2091. doi: 10.1002/cpe.3398.  Google Scholar

[10]

R. Cattell, Scalable sql and nosql data stores, ACM SIGMOD Record, 39 (2010), 12-27. doi: 10.1145/1978915.1978919.  Google Scholar

[11]

F. Chang, J. Dean, S. Ghemawat, W. C. Hsieh, D. A. Wallach, M. Burrows, T. Chandra, A. Fikes and R. E. Gruber, Bigtable: A distributed storage system for structured data, ACM Transactions on Computer Systems (TOCS), 26 (2008), p4. doi: 10.1145/1365815.1365816.  Google Scholar

[12]

B. F. Cooper, A. Silberstein, E. Tam, R. Ramakrishnan and R. Sears, Benchmarking cloud serving systems with ycsb, in Proceedings of the 1st ACM symposium on Cloud computing, ACM, 2010, 143-154. doi: 10.1145/1807128.1807152.  Google Scholar

[13]

S. Edlich, A. Friedland, J. Hampe, B. Brauer, M. Brückner, S. Edlich, A. Friedland, J. Hampe, B. Brauer and M. Brückner, Brückner,, Nosql., ().   Google Scholar

[14]

A. Feinberg, Project voldemort: Reliable distributed storage, in Proceedings of the 10th IEEE International Conference on Data Engineering, 2011. Google Scholar

[15]

B. Fitzpatrick, Distributed caching with memcached, Linux journal, 2004 (2004), p5. Google Scholar

[16]

S. K. Gajendran, A survey on nosql databases,, University of Illinois., ().   Google Scholar

[17]

J. Gray, Graysort benchmark, 2015,, URL , ().   Google Scholar

[18]

Hibernating Rhinos, Ravendb - the open source nosql database for .NET, 2015,, URL , ().   Google Scholar

[19]

Hypertable Inc, Hypertable, 2014,, URL , ().   Google Scholar

[20]

S. IT, Knowledge base of relational and nosql database management systems, 2015,, URL , ().   Google Scholar

[21]

S. IT, System properties comparison neo4j vs. orientdb vs. titan, 2015,, URL , ().   Google Scholar

[22]

J. Jose, H. Subramoni, M. Luo, M. Zhang, J. Huang, M. Wasi-ur Rahman, N. S. Islam, X. Ouyang, H. Wang, S. Sur et al., Memcached design on high performance rdma capable interconnects, in Parallel Processing (ICPP), 2011 International Conference on, IEEE, 2011, 743-752. doi: 10.1109/ICPP.2011.37.  Google Scholar

[23]

S. Jouili and V. Vansteenberghe, An empirical comparison of graph databases, in Social Computing (SocialCom), 2013 International Conference on, 2013, 708-715. doi: 10.1109/SocialCom.2013.106.  Google Scholar

[24]

J. Klein, I. Gorton, N. Ernst, P. Donohoe, K. Pham and C. Matser, Performance evaluation of nosql databases: A case study, in Proceedings of the 1st Workshop on Performance Analysis of Big Data Systems, PABS '15, ACM, New York, NY, USA, 2015, 5-10. doi: 10.1145/2694730.2694731.  Google Scholar

[25]

LinkedIn, Project voldemort, 2015,, URL , ().   Google Scholar

[26]

R. C. McColl, D. Ediger, J. Poovey, D. Campbell and D. A. Bader, A performance evaluation of open source graph databases, in Proceedings of the First Workshop on Parallel Programming for Analytics Applications, PPAA '14, ACM, New York, NY, USA, 2014, 11-18. doi: 10.1145/2567634.2567638.  Google Scholar

[27]

MongoDB Inc., Mongodb 3.0 manual, 2015,, URL , ().   Google Scholar

[28]

A. Moniruzzaman and S. A. Hossain, Nosql database: New era of databases for big data analytics-classification, characteristics and comparison,, arXiv preprint arXiv:1307.0191., ().   Google Scholar

[29]

M. A. Olson, K. Bostic and M. I. Seltzer, Berkeley db., in USENIX Annual Technical Conference, FREENIX Track, 1999, 183-191. Google Scholar

[30]

Orient Technologies, Top 10 key advantages for going with orientdb, 2015,, URL , ().   Google Scholar

[31]

A. Pavlo, E. Paulson, A. Rasin, D. J. Abadi, D. J. DeWitt, S. Madden and M. Stonebraker, A comparison of approaches to large-scale data analysis, in Proceedings of the 2009 ACM SIGMOD International Conference on Management of data, ACM, 2009, 165-178. doi: 10.1145/1559845.1559865.  Google Scholar

[32]

D. Pritchett, Base: An acid alternative, Queue, 6 (2008), 48-55. doi: 10.1145/1394127.1394128.  Google Scholar

[33]

T. Rabl, A. Ghazal, M. Hu, A. Crolotte, F. Raab, M. Poess and H.-A. Jacobsen, Bigbench specification v0. 1, in Specifying Big Data Benchmarks, Springer, 2014, 164-201. Google Scholar

[34]

RedisLabs, Redis, 2015,, URL , ().   Google Scholar

[35]

SAVI, Smart Applications on Virtual Infrastructure, Cloud platform, 2015, URL http://www.savinetwork.ca. Google Scholar

[36]

S. Sivasubramanian, Amazon dynamodb: A seamlessly scalable non-relational database service, in Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, ACM, 2012, 729-730. doi: 10.1145/2213836.2213945.  Google Scholar

[37]

C. Strozzi, Nosql-a relational database management system, 2015,, URL , ().   Google Scholar

[38]

Technology, Cypher query language, 2015,, URL , ().   Google Scholar

[39]

The Apache Foundation, Apache accumulo, 2015,, URL , ().   Google Scholar

[40]

The Apache Foundation, Welcome to apache cassandra, 2015,, URL , ().   Google Scholar

[41]

The Apache Foundation, Welcome to apache hbase, 2015,, URL , ().   Google Scholar

[42]

A. Tizghadam and A. Leon-Garcia, Connected Vehicles and Smart Transportation - CVST Platform, 2015,, URL , ().   Google Scholar

[43]

G. Vaish, Getting started with NoSQL, Packt Publishing Ltd, 2013. Google Scholar

[44]

vsChart.com, The comparison wiki: Database list, 2015,, URL , ().   Google Scholar

[45]

P. Wiki, Pig mix benchmark, 2015,, URL, ().   Google Scholar

[1]

Mohamed Baouch, Juan Antonio López-Ramos, Blas Torrecillas, Reto Schnyder. An active attack on a distributed Group Key Exchange system. Advances in Mathematics of Communications, 2017, 11 (4) : 715-717. doi: 10.3934/amc.2017052
[2]

Shunfu Jin, Wuyi Yue, Zhanqiang Huo. Performance evaluation for connection oriented service in the next generation Internet. Numerical Algebra, Control & Optimization, 2011, 1 (4) : 749-761. doi: 10.3934/naco.2011.1.749
[3]

Shunfu Jin, Wuyi Yue, Chao Meng, Zsolt Saffer. A novel active DRX mechanism in LTE technology and its performance evaluation. Journal of Industrial & Management Optimization, 2015, 11 (3) : 849-866. doi: 10.3934/jimo.2015.11.849
[4]

Shunfu Jin, Haixing Wu, Wuyi Yue, Yutaka Takahashi. Performance evaluation and Nash equilibrium of a cloud architecture with a sleeping mechanism and an enrollment service. Journal of Industrial & Management Optimization, 2020, 16 (5) : 2407-2424. doi: 10.3934/jimo.2019060
[5]

Keiji Tatsumi, Masashi Akao, Ryo Kawachi, Tetsuzo Tanino. Performance evaluation of multiobjective multiclass support vector machines maximizing geometric margins. Numerical Algebra, Control & Optimization, 2011, 1 (1) : 151-169. doi: 10.3934/naco.2011.1.151
[6]

Tuan Phung-Duc, Wouter Rogiest, Sabine Wittevrongel. Single server retrial queues with speed scaling: Analysis and performance evaluation. Journal of Industrial & Management Optimization, 2017, 13 (4) : 1927-1943. doi: 10.3934/jimo.2017025
[7]

Yoichi Enatsu, Yukihiko Nakata, Yoshiaki Muroya. Global stability of SIR epidemic models with a wide class of nonlinear incidence rates and distributed delays. Discrete & Continuous Dynamical Systems - B, 2011, 15 (1) : 61-74. doi: 10.3934/dcdsb.2011.15.61
[8]

Sergei Avdonin, Jonathan Bell. Determining a distributed conductance parameter for a neuronal cable model defined on a tree graph. Inverse Problems & Imaging, 2015, 9 (3) : 645-659. doi: 10.3934/ipi.2015.9.645
[9]

Tsuguhito Hirai, Hiroyuki Masuyama, Shoji Kasahara, Yutaka Takahashi. Performance optimization of parallel-distributed processing with checkpointing for cloud environment. Journal of Industrial & Management Optimization, 2018, 14 (4) : 1423-1442. doi: 10.3934/jimo.2018014
[10]

Yunmei Lu, Mingyuan Yan, Meng Han, Qingliang Yang, Yanqing Zhang. Privacy preserving feature selection and Multiclass Classification for horizontally distributed data. Mathematical Foundations of Computing, 2018, 1 (4) : 331-348. doi: 10.3934/mfc.2018016
[11]

Shunfu Jin, Wuyi Yue. Performance analysis and evaluation for power saving class type III in IEEE 802.16e network. Journal of Industrial & Management Optimization, 2010, 6 (3) : 691-708. doi: 10.3934/jimo.2010.6.691
[12]

Omer Faruk Yilmaz, Mehmet Bulent Durmusoglu. A performance comparison and evaluation of metaheuristics for a batch scheduling problem in a multi-hybrid cell manufacturing system with skilled workforce assignment. Journal of Industrial & Management Optimization, 2018, 14 (3) : 1219-1249. doi: 10.3934/jimo.2018007
[13]

Yuan Zhao, Wuyi Yue. Performance evaluation and optimization of cognitive radio networks with adjustable access control for multiple secondary users. Journal of Industrial & Management Optimization, 2019, 15 (1) : 1-14. doi: 10.3934/jimo.2018029
[14]

Zhanyou Ma, Wenbo Wang, Linmin Hu. Performance evaluation and analysis of a discrete queue system with multiple working vacations and non-preemptive priority. Journal of Industrial & Management Optimization, 2020, 16 (3) : 1135-1148. doi: 10.3934/jimo.2018196
[15]

Shunfu Jin, Wuyi Yue, Xuena Yan. Performance evaluation of a power saving mechanism in IEEE 802.16 wireless MANs with bi-directional traffic. Journal of Industrial & Management Optimization, 2011, 7 (3) : 717-733. doi: 10.3934/jimo.2011.7.717
[16]

Jianping Liu, Shunfu Jin. An imperfect sensing-based channel reservation strategy in CRNs and its performance evaluation. Journal of Industrial & Management Optimization, 2020, 16 (3) : 1149-1169. doi: 10.3934/jimo.2018197
[17]

Dawei Wang, Linlin Zhao, Feng Yang, Kehong Chen. Performance evaluation of the Chinese high-tech industry: A two-stage DEA approach with feedback and shared resource. Journal of Industrial & Management Optimization, 2021  doi: 10.3934/jimo.2021114
[18]

Patrizia Pucci, Maria Cesarina Salvatori. On an initial value problem modeling evolution and selection in living systems. Discrete & Continuous Dynamical Systems - S, 2014, 7 (4) : 807-821. doi: 10.3934/dcdss.2014.7.807
[19]

Anupam Gautam, Selvamuthu Dharmaraja. Selection of DRX scheme for voice traffic in LTE-A networks: Markov modeling and performance analysis. Journal of Industrial & Management Optimization, 2019, 15 (2) : 739-756. doi: 10.3934/jimo.2018068
[20]

Shiva Moslemi, Abolfazl Mirzazadeh. Performance evaluation of four-stage blood supply chain with feedback variables using NDEA cross-efficiency and entropy measures under IER uncertainty. Numerical Algebra, Control & Optimization, 2017, 7 (4) : 379-401. doi: 10.3934/naco.2017024

 Impact Factor: 

Tools

Metrics

  • PDF downloads (425)
  • HTML views (0)
  • Cited by (7)

Other articles
by authors

[Back to Top]

Copyright © 2021 American Institute of Mathematical Sciences