IOT数据库选型——NOSQL,MemSQL,cassandra,Riak或者OpenTSDB,InfluxDB

简介:

补充:

Basho公司开源了它的时序数据库产品Riak TS 1.3

代码在github riak的riak-ts分支上!

Riak KV产品构建于Riak内核之上,提供了一种高弹性、高可用的键值数据库。Riak KV产品当前正在持续改进中,专注于数据正确性、预防数据损失和破坏等特性。

Riak TS产品源于Riak KV数据库,是一种为时序数据仓库而专门构建的产品。其中集成了Riak KV产品的所有强大功能,并使用这些功能去解决用户在处理时序数据中所遇到的问题。我们在该产品中确实地实现了哪些特性呢?这里我列出了其中的一部分:

  • 数据的快速写入路径;
  • 为数据桶建立模式;
  • 查询规划及查询子系统;
  • 对虚拟节点的并行数据抽取;
  • 灵活的复合键值;

我们也查看了时序数据库产品的市场情况,当时只见到了寥寥可数的几个解决方案,并且所有这些解决方案的质量都不足以承担企业级的生产工作负荷。已有的时序数据解决方案或者是缺乏可扩展集群或弹性,或者是管理和操作非常繁琐。所有这些使得它们成为糟糕的选择。

为讨论解决这个问题的创意,我们进而开了一次架构会议。最终,我们的一个工程师提出了一个有意思的创意,即使用量子(时间范围)将数据围绕哈希 环分布,并基于此创意构建了一个看上去运行良好的概念验证原型。依此我们开始了Riak TS产品的开发过程,力图去解决许多时序数据处理中更加困难的问题。

见:

https://elixirforum.com/t/which-database-for-time-series-data/715/6

http://db-engines.com/en/system/Graphite%3BInfluxDB%3BRiak+TS

 

IoT databases should be as flexible as required by the application. NoSQLdatabases -- especially key-value, document and column family databases -- easily accommodate different data types and structures without the need for predefined, fixed schemas. NoSQL databases are good options when an organization has multiple data types and those data types will likely change over time. In other cases, applications that collect a fixed set of data -- such as data on weather conditions -- may benefit from a relational model. In-memory SQL databases, such as MemSQL, offer this benefit.

Managing a database for IoT applications in-house

For those organizations choosing to manage their own databases, DataStax Cassandra is a highly scalable distributed database that supports a flexible big table schema and fast writes and scales to large volumes of data. Riak IoT is a distributed, highly scalable key-value data store which integrates with Apache Spark, a big data analytics platform that enables stream analytic processing. Cassandra also integrates with Spark as well as other big data analytics platforms, such as Hadoop MapReduce.

OpenTSDB is an open source database capable of running on Hadoop andHBase. The database is made up of command line interfaces and a Time Series Daemon (TSD). TSDs, which are responsible for processing all database requests, run independently of one another. Even though TSDs use HBase to store time-series data, TSD users have little to no contact with HBase itself.

MemSQL is a relational database tuned for real-time data streaming. With MemSQL, streamed data, transactions and historical data can be kept within the same database. The database also has the capacity to work well with geospatial data out of the box, which could be useful for location-based IoT applications. MemSQL supports integration with Hadoop Distributed File System and Apache Spark, as well as other data warehousing solutions.

 

摘自:http://internetofthingsagenda.techtarget.com/feature/Find-the-IoT-database-that-best-fits-your-enterprises-needs

 

You’ve heard the hype, the Internet of Things (IoT) is going to connect more people to devices, more devices to the Internet and generate more data than any major IT shift in history. IoT is going to be bigger than the web, mobile and the cloud, right? It’s still too early to tell for sure, but at InfluxData we are helping startups and enterprises everyday bring an interconnected world closer to reality.

What does time-series have to do with IoT? Everything, actually. Sensors and devices used in IoT architectures emit time-series data, and a lot of it.

Why are companies building IoT and sensor data solutions?

Whether it’s pH and humidity readings from an agri-sensor, depth and fluid readings from a geo-sensor or voltage and temperature from a power control sensor, these metrics are forming the basis of intelligent businesses. Common use cases we run across are:

  • Agro industries are monitoring and trying to control environmental conditions for optimal plant growth.
  • Power and utility companies are building smart solutions to reduce resource wastage for residential and commercial customers.
  • Research labs and heavy industries are tracking the resources, usage and health of millions of tiny valves and instruments that go into their massive production plants, factories and manufacturing facilities.
  • Smart cars are now powerful computers making runtime decisions based on data collected by 100s of sensors on every vehicle.

get-started__graphic-3

Challenges in building IoT and sensor data solutions

The key challenges organizations face while building an IoT solution are:

  • Bandwidth – As sensors are generally deployed on-premise and need to communicate over wireless networks, bandwidth constraints prevent sending large packets of data in real-time
  • Horsepower – Compute power on sensors are generally limited. Hence analytics software – programs or databases or even processing logic needs to have a tiny footprint.
  • Concurrency – In case of industrial IoT, number of sensors could easily range in 100s of 1000s, each transmitting metrics every minute or so. Anticipating backend database’s concurrency limits is crucial in the design of such solutions
  • Protocol – As this space is rapidly evolving, there aren’t any definitive standards for communication protocols. MQTT, AMQPP, CoAP etc are being used based on use cases. Hence IoT analytics solutions need to support many communication protocols.
  • Scale – Data retention, compression and visualization has it’s own challenges in such a large data footprint solution. Businesses want to plot trends (WoW, MoM, YoY) and aggregation of massive data sets can be very compute heavy.

 摘自:https://www.influxdata.com/use-cases/iot-and-sensor-data/

 

 

NoSQL Database: The NoSQL database is typically used to address the fast data ingest problem for device data. In some cases, there may be a stream processor—e.g. Storm, Samza, Kinesis, etc.—addressing data filtering and routing and some lightweight processing, such as counts. However, the NoSQL database is typically used because, unlike most SQL databases, which top out at about 5,000 inserts/second, you can get up to 50,000 inserts/second from NoSQL databases. However, NoSQL databases are not designed to handle the analytic processing of the data or joins, which are common requirements for Internet of Things applications. NoSQL effectively provides a real-time data ingest engine for data that is then moved to Hadoop using an extract, transform and load (ETL) process.——NOSQL写入快,但是数据分析,联合查询不方便!










本文转自张昺华-sky博客园博客,原文链接:http://www.cnblogs.com/bonelee/p/6265106.html,如需转载请自行联系原作者

相关文章
|
1月前
|
存储 监控 NoSQL
九大核心NoSQL数据库及使用场景详解
【10月更文挑战第6天】在当今大数据与云计算飞速发展的时代,NoSQL数据库以其灵活的数据模型、可扩展性和高性能,成为了众多应用场景下的首选。本文将为您详细介绍九大核心NoSQL数据库及其典型使用场景,帮助您在工作和学习中更好地选择和应用。
55 3
|
2月前
|
NoSQL 关系型数据库 MySQL
微服务架构下的数据库选择:MySQL、PostgreSQL 还是 NoSQL?
在微服务架构中,数据库的选择至关重要。不同类型的数据库适用于不同的需求和场景。在本文章中,我们将深入探讨传统的关系型数据库(如 MySQL 和 PostgreSQL)与现代 NoSQL 数据库的优劣势,并分析在微服务架构下的最佳实践。
|
5天前
|
存储 缓存 NoSQL
常见的 NoSQL 数据库有哪些?
常见的 NoSQL 数据库有哪些?
9 2
|
1月前
|
SQL NoSQL 数据库
Cassandra数据库与Cql实战笔记
Cassandra数据库与Cql实战笔记
18 1
Cassandra数据库与Cql实战笔记
|
22天前
|
存储 SQL JSON
介绍一下RDBMS和NoSQL数据库之间的区别
【10月更文挑战第21天】介绍一下RDBMS和NoSQL数据库之间的区别
47 2
|
22天前
|
存储 SQL NoSQL
数据库技术深度探索:从关系型到NoSQL的演变
【10月更文挑战第21天】数据库技术深度探索:从关系型到NoSQL的演变
30 1
|
29天前
|
存储 NoSQL 搜索推荐
nosql
【10月更文挑战第14天】nosql
18 2
|
1月前
|
NoSQL MongoDB 数据库
MongoDB是一个NoSQL数据库,有着多种不同的命令和操作。以下是一些常见的MongoDB命令:
一些常用的MongoDB命令,如数据库和集合的管理、数据的插入、查询、更新、删除以及聚合操作等。
24 1
|
23天前
|
NoSQL 前端开发 MongoDB
前端的全栈之路Meteor篇(三):运行在浏览器端的NoSQL数据库副本-MiniMongo介绍及其前后端数据实时同步示例
MiniMongo 是 Meteor 框架中的客户端数据库组件,模拟了 MongoDB 的核心功能,允许前端开发者使用类似 MongoDB 的 API 进行数据操作。通过 Meteor 的数据同步机制,MiniMongo 与服务器端的 MongoDB 实现实时数据同步,确保数据一致性,支持发布/订阅模型和响应式数据源,适用于实时聊天、项目管理和协作工具等应用场景。
|
1月前
|
存储 SQL 分布式计算
NoSQL 简介
10月更文挑战第10天
28 0