IOT数据库选型——NOSQL,MemSQL,cassandra,Riak或者OpenTSDB,InfluxDB

简介:

补充:

Basho公司开源了它的时序数据库产品Riak TS 1.3

代码在github riak的riak-ts分支上!

Riak KV产品构建于Riak内核之上,提供了一种高弹性、高可用的键值数据库。Riak KV产品当前正在持续改进中,专注于数据正确性、预防数据损失和破坏等特性。

Riak TS产品源于Riak KV数据库,是一种为时序数据仓库而专门构建的产品。其中集成了Riak KV产品的所有强大功能,并使用这些功能去解决用户在处理时序数据中所遇到的问题。我们在该产品中确实地实现了哪些特性呢?这里我列出了其中的一部分:

  • 数据的快速写入路径;
  • 为数据桶建立模式;
  • 查询规划及查询子系统;
  • 对虚拟节点的并行数据抽取;
  • 灵活的复合键值;

我们也查看了时序数据库产品的市场情况,当时只见到了寥寥可数的几个解决方案,并且所有这些解决方案的质量都不足以承担企业级的生产工作负荷。已有的时序数据解决方案或者是缺乏可扩展集群或弹性,或者是管理和操作非常繁琐。所有这些使得它们成为糟糕的选择。

为讨论解决这个问题的创意,我们进而开了一次架构会议。最终,我们的一个工程师提出了一个有意思的创意,即使用量子(时间范围)将数据围绕哈希 环分布,并基于此创意构建了一个看上去运行良好的概念验证原型。依此我们开始了Riak TS产品的开发过程,力图去解决许多时序数据处理中更加困难的问题。

见:

https://elixirforum.com/t/which-database-for-time-series-data/715/6

http://db-engines.com/en/system/Graphite%3BInfluxDB%3BRiak+TS

 

IoT databases should be as flexible as required by the application. NoSQLdatabases -- especially key-value, document and column family databases -- easily accommodate different data types and structures without the need for predefined, fixed schemas. NoSQL databases are good options when an organization has multiple data types and those data types will likely change over time. In other cases, applications that collect a fixed set of data -- such as data on weather conditions -- may benefit from a relational model. In-memory SQL databases, such as MemSQL, offer this benefit.

Managing a database for IoT applications in-house

For those organizations choosing to manage their own databases, DataStax Cassandra is a highly scalable distributed database that supports a flexible big table schema and fast writes and scales to large volumes of data. Riak IoT is a distributed, highly scalable key-value data store which integrates with Apache Spark, a big data analytics platform that enables stream analytic processing. Cassandra also integrates with Spark as well as other big data analytics platforms, such as Hadoop MapReduce.

OpenTSDB is an open source database capable of running on Hadoop andHBase. The database is made up of command line interfaces and a Time Series Daemon (TSD). TSDs, which are responsible for processing all database requests, run independently of one another. Even though TSDs use HBase to store time-series data, TSD users have little to no contact with HBase itself.

MemSQL is a relational database tuned for real-time data streaming. With MemSQL, streamed data, transactions and historical data can be kept within the same database. The database also has the capacity to work well with geospatial data out of the box, which could be useful for location-based IoT applications. MemSQL supports integration with Hadoop Distributed File System and Apache Spark, as well as other data warehousing solutions.

 

摘自:http://internetofthingsagenda.techtarget.com/feature/Find-the-IoT-database-that-best-fits-your-enterprises-needs

 

You’ve heard the hype, the Internet of Things (IoT) is going to connect more people to devices, more devices to the Internet and generate more data than any major IT shift in history. IoT is going to be bigger than the web, mobile and the cloud, right? It’s still too early to tell for sure, but at InfluxData we are helping startups and enterprises everyday bring an interconnected world closer to reality.

What does time-series have to do with IoT? Everything, actually. Sensors and devices used in IoT architectures emit time-series data, and a lot of it.

Why are companies building IoT and sensor data solutions?

Whether it’s pH and humidity readings from an agri-sensor, depth and fluid readings from a geo-sensor or voltage and temperature from a power control sensor, these metrics are forming the basis of intelligent businesses. Common use cases we run across are:

  • Agro industries are monitoring and trying to control environmental conditions for optimal plant growth.
  • Power and utility companies are building smart solutions to reduce resource wastage for residential and commercial customers.
  • Research labs and heavy industries are tracking the resources, usage and health of millions of tiny valves and instruments that go into their massive production plants, factories and manufacturing facilities.
  • Smart cars are now powerful computers making runtime decisions based on data collected by 100s of sensors on every vehicle.

get-started__graphic-3

Challenges in building IoT and sensor data solutions

The key challenges organizations face while building an IoT solution are:

  • Bandwidth – As sensors are generally deployed on-premise and need to communicate over wireless networks, bandwidth constraints prevent sending large packets of data in real-time
  • Horsepower – Compute power on sensors are generally limited. Hence analytics software – programs or databases or even processing logic needs to have a tiny footprint.
  • Concurrency – In case of industrial IoT, number of sensors could easily range in 100s of 1000s, each transmitting metrics every minute or so. Anticipating backend database’s concurrency limits is crucial in the design of such solutions
  • Protocol – As this space is rapidly evolving, there aren’t any definitive standards for communication protocols. MQTT, AMQPP, CoAP etc are being used based on use cases. Hence IoT analytics solutions need to support many communication protocols.
  • Scale – Data retention, compression and visualization has it’s own challenges in such a large data footprint solution. Businesses want to plot trends (WoW, MoM, YoY) and aggregation of massive data sets can be very compute heavy.

 摘自:https://www.influxdata.com/use-cases/iot-and-sensor-data/

 

 

NoSQL Database: The NoSQL database is typically used to address the fast data ingest problem for device data. In some cases, there may be a stream processor—e.g. Storm, Samza, Kinesis, etc.—addressing data filtering and routing and some lightweight processing, such as counts. However, the NoSQL database is typically used because, unlike most SQL databases, which top out at about 5,000 inserts/second, you can get up to 50,000 inserts/second from NoSQL databases. However, NoSQL databases are not designed to handle the analytic processing of the data or joins, which are common requirements for Internet of Things applications. NoSQL effectively provides a real-time data ingest engine for data that is then moved to Hadoop using an extract, transform and load (ETL) process.——NOSQL写入快,但是数据分析,联合查询不方便!










本文转自张昺华-sky博客园博客,原文链接:http://www.cnblogs.com/bonelee/p/6265106.html,如需转载请自行联系原作者

相关文章
|
25天前
|
存储 NoSQL 关系型数据库
面试题18: NOSQL数据库
面试题18: NOSQL数据库
|
30天前
|
存储 NoSQL API
一个小巧、快速、轻量级的 .NET NoSQL 嵌入式数据库
一个小巧、快速、轻量级的 .NET NoSQL 嵌入式数据库
|
29天前
|
多模数据库 Cloud Native NoSQL
Nosql学习之路:云原生多模数据库Lindorm训练营第一弹来啦
Lindorm训练营系列将通过一系列由浅入深的高质量课程和丰富的动手实验,将理论与实践结合,带你从入门到成为高阶开发者。参营学习还有机会获得惊喜彩蛋~
|
1月前
|
缓存 NoSQL MongoDB
在使用NoSQL数据库时,你遇到过哪些挑战?如何解决这些挑战?
在使用NoSQL数据库时,你遇到过哪些挑战?如何解决这些挑战?
20 0
|
1月前
|
存储 JSON NoSQL
请列举一些常见的NoSQL数据库类型和其特点。
请列举一些常见的NoSQL数据库类型和其特点。
27 0
|
1月前
|
存储 SQL NoSQL
NoSQL数据库的优点和缺点是什么?
NoSQL数据库的优点和缺点是什么?
32 0
|
1月前
|
存储 NoSQL 关系型数据库
什么是NoSQL数据库?它与传统关系型数据库有什么区别?
什么是NoSQL数据库?它与传统关系型数据库有什么区别?
24 0
|
1月前
|
存储 NoSQL 关系型数据库
【MySQL】为什么需要NOSQL数据库
`RDBMS`和`NOSQL`数据库的优缺点
|
2月前
|
NoSQL 安全 Java
基于内存的分布式NoSQL数据库Redis(六)AOF设计
基于内存的分布式NoSQL数据库Redis(六)AOF设计
158 0
|
2月前
|
存储 分布式计算 NoSQL
基于内存的分布式NoSQL数据库Redis(五)数据存储与RDB设计
基于内存的分布式NoSQL数据库Redis(五)数据存储与RDB设计
142 0

热门文章

最新文章