IOT数据库选型——NOSQL,MemSQL,cassandra,Riak或者OpenTSDB,InfluxDB

简介:

补充:

Basho公司开源了它的时序数据库产品Riak TS 1.3

代码在github riak的riak-ts分支上!

Riak KV产品构建于Riak内核之上,提供了一种高弹性、高可用的键值数据库。Riak KV产品当前正在持续改进中,专注于数据正确性、预防数据损失和破坏等特性。

Riak TS产品源于Riak KV数据库,是一种为时序数据仓库而专门构建的产品。其中集成了Riak KV产品的所有强大功能,并使用这些功能去解决用户在处理时序数据中所遇到的问题。我们在该产品中确实地实现了哪些特性呢?这里我列出了其中的一部分:

  • 数据的快速写入路径;
  • 为数据桶建立模式;
  • 查询规划及查询子系统;
  • 对虚拟节点的并行数据抽取;
  • 灵活的复合键值;

我们也查看了时序数据库产品的市场情况,当时只见到了寥寥可数的几个解决方案,并且所有这些解决方案的质量都不足以承担企业级的生产工作负荷。已有的时序数据解决方案或者是缺乏可扩展集群或弹性,或者是管理和操作非常繁琐。所有这些使得它们成为糟糕的选择。

为讨论解决这个问题的创意,我们进而开了一次架构会议。最终,我们的一个工程师提出了一个有意思的创意,即使用量子(时间范围)将数据围绕哈希 环分布,并基于此创意构建了一个看上去运行良好的概念验证原型。依此我们开始了Riak TS产品的开发过程,力图去解决许多时序数据处理中更加困难的问题。

见:

https://elixirforum.com/t/which-database-for-time-series-data/715/6

http://db-engines.com/en/system/Graphite%3BInfluxDB%3BRiak+TS

 

IoT databases should be as flexible as required by the application. NoSQLdatabases -- especially key-value, document and column family databases -- easily accommodate different data types and structures without the need for predefined, fixed schemas. NoSQL databases are good options when an organization has multiple data types and those data types will likely change over time. In other cases, applications that collect a fixed set of data -- such as data on weather conditions -- may benefit from a relational model. In-memory SQL databases, such as MemSQL, offer this benefit.

Managing a database for IoT applications in-house

For those organizations choosing to manage their own databases, DataStax Cassandra is a highly scalable distributed database that supports a flexible big table schema and fast writes and scales to large volumes of data. Riak IoT is a distributed, highly scalable key-value data store which integrates with Apache Spark, a big data analytics platform that enables stream analytic processing. Cassandra also integrates with Spark as well as other big data analytics platforms, such as Hadoop MapReduce.

OpenTSDB is an open source database capable of running on Hadoop andHBase. The database is made up of command line interfaces and a Time Series Daemon (TSD). TSDs, which are responsible for processing all database requests, run independently of one another. Even though TSDs use HBase to store time-series data, TSD users have little to no contact with HBase itself.

MemSQL is a relational database tuned for real-time data streaming. With MemSQL, streamed data, transactions and historical data can be kept within the same database. The database also has the capacity to work well with geospatial data out of the box, which could be useful for location-based IoT applications. MemSQL supports integration with Hadoop Distributed File System and Apache Spark, as well as other data warehousing solutions.

 

摘自:http://internetofthingsagenda.techtarget.com/feature/Find-the-IoT-database-that-best-fits-your-enterprises-needs

 

You’ve heard the hype, the Internet of Things (IoT) is going to connect more people to devices, more devices to the Internet and generate more data than any major IT shift in history. IoT is going to be bigger than the web, mobile and the cloud, right? It’s still too early to tell for sure, but at InfluxData we are helping startups and enterprises everyday bring an interconnected world closer to reality.

What does time-series have to do with IoT? Everything, actually. Sensors and devices used in IoT architectures emit time-series data, and a lot of it.

Why are companies building IoT and sensor data solutions?

Whether it’s pH and humidity readings from an agri-sensor, depth and fluid readings from a geo-sensor or voltage and temperature from a power control sensor, these metrics are forming the basis of intelligent businesses. Common use cases we run across are:

  • Agro industries are monitoring and trying to control environmental conditions for optimal plant growth.
  • Power and utility companies are building smart solutions to reduce resource wastage for residential and commercial customers.
  • Research labs and heavy industries are tracking the resources, usage and health of millions of tiny valves and instruments that go into their massive production plants, factories and manufacturing facilities.
  • Smart cars are now powerful computers making runtime decisions based on data collected by 100s of sensors on every vehicle.

get-started__graphic-3

Challenges in building IoT and sensor data solutions

The key challenges organizations face while building an IoT solution are:

  • Bandwidth – As sensors are generally deployed on-premise and need to communicate over wireless networks, bandwidth constraints prevent sending large packets of data in real-time
  • Horsepower – Compute power on sensors are generally limited. Hence analytics software – programs or databases or even processing logic needs to have a tiny footprint.
  • Concurrency – In case of industrial IoT, number of sensors could easily range in 100s of 1000s, each transmitting metrics every minute or so. Anticipating backend database’s concurrency limits is crucial in the design of such solutions
  • Protocol – As this space is rapidly evolving, there aren’t any definitive standards for communication protocols. MQTT, AMQPP, CoAP etc are being used based on use cases. Hence IoT analytics solutions need to support many communication protocols.
  • Scale – Data retention, compression and visualization has it’s own challenges in such a large data footprint solution. Businesses want to plot trends (WoW, MoM, YoY) and aggregation of massive data sets can be very compute heavy.

 摘自:https://www.influxdata.com/use-cases/iot-and-sensor-data/

 

 

NoSQL Database: The NoSQL database is typically used to address the fast data ingest problem for device data. In some cases, there may be a stream processor—e.g. Storm, Samza, Kinesis, etc.—addressing data filtering and routing and some lightweight processing, such as counts. However, the NoSQL database is typically used because, unlike most SQL databases, which top out at about 5,000 inserts/second, you can get up to 50,000 inserts/second from NoSQL databases. However, NoSQL databases are not designed to handle the analytic processing of the data or joins, which are common requirements for Internet of Things applications. NoSQL effectively provides a real-time data ingest engine for data that is then moved to Hadoop using an extract, transform and load (ETL) process.——NOSQL写入快,但是数据分析,联合查询不方便!










本文转自张昺华-sky博客园博客,原文链接:http://www.cnblogs.com/bonelee/p/6265106.html,如需转载请自行联系原作者

相关文章
|
5天前
|
人工智能 运维 NoSQL
云栖大会|AI浪潮下的NoSQL演进:下一代数据库的破局之道
AI浪潮下的NoSQL演进:下一代数据库的破局之道
|
4月前
|
存储 NoSQL 搜索推荐
NoSQL数据库分类概览
以上就是我们的NoSQL数据库奇幻之旅。每一种NoSQL数据库都有自己独特的魅力和专长,择选合适的数据库,就像在魔法世界中挑选最适合自己的魔杖,使你的数据管理变得更加高效和神奇。在当今数据驱动的时代,懂得这些数据库的秘密,就掌握了处理各种数据挑战的关键。
246 61
|
存储 监控 NoSQL
九大核心NoSQL数据库及使用场景详解
【10月更文挑战第6天】在当今大数据与云计算飞速发展的时代,NoSQL数据库以其灵活的数据模型、可扩展性和高性能,成为了众多应用场景下的首选。本文将为您详细介绍九大核心NoSQL数据库及其典型使用场景,帮助您在工作和学习中更好地选择和应用。
485 3
|
9月前
|
存储 NoSQL 关系型数据库
【赵渝强老师】什么是NoSQL数据库?
随着大数据技术的兴起,NoSQL数据库(Not Only SQL)得到广泛应用。它不局限于二维表结构,允许数据冗余。常见的NoSQL数据库包括Redis、MongoDB和HBase。Redis是基于内存的高性能数据库,采用单线程模型和多路复用I/O,支持高效的数据结构。MongoDB使用BSON格式存储文档,查询语言强大,类似关系型数据库。HBase基于HDFS,适合数据分析,采用列式存储,支持灵活的列族设计。视频讲解及更多内容见下文。
473 79
|
6月前
|
SQL 运维 监控
数据库国产化选型?YashanDB 给中大型企业的五个答案
近两年,“国产数据库”成为企业数字化升级的重要议题。YashanDB作为新一代国产关系型数据库,以完整产品矩阵解决企业核心关切:通过图形化工具提升开发效率;提供自动化迁移平台简化数据库切换;云管理工具减轻运维负担;支持复杂业务场景;拥有成熟团队保障企业级服务。YashanDB不仅是“可替代”,更是“更可靠”的选择,助力企业实现高效、可控的数据库升级。
|
7月前
|
物联网 测试技术 API
时序数据库 InfluxDB 3.0 版本性能实测报告:写入吞吐量提升效果验证
TSBS 测试表明,对于少于 100 万台设备的数据集,InfluxDB OSS 3.0 的数据写入速度实际上比 InfluxDB OSS 1.8 更慢。 对于 100 万台及以上设备的数据集,InfluxDB OSS 3.0 的数据写入性能才开始超过 InfluxDB OSS 1.8。 InfluxDB OSS 3.0 的数据写入接口与 InfluxDB 1.8 并不兼容,用户无法顺利迁移。
534 7
|
11月前
|
存储 缓存 NoSQL
常见的 NoSQL 数据库有哪些?
常见的 NoSQL 数据库有哪些?
727 59
|
10月前
|
缓存 物联网 数据库
InfluxDB vs TDengine :2025 年了,谁家用的数据库还不能高效读缓存?
在工业互联网和物联网的大数据应用场景中,实时数据的写入和查询性能至关重要。如何快速获取最新设备状态并实时处理数据,直接影响到业务的高效运转。本文将深入分析 TDengine 和 InfluxDB 在缓存机制上的差异,帮助读者更好地理解这两款主流时序数据库在性能优化方面的优劣。
918 1
|
SQL NoSQL 数据库
Cassandra数据库与Cql实战笔记
Cassandra数据库与Cql实战笔记
185 1
Cassandra数据库与Cql实战笔记
|
12月前
|
存储 SQL JSON
介绍一下RDBMS和NoSQL数据库之间的区别
【10月更文挑战第21天】介绍一下RDBMS和NoSQL数据库之间的区别
416 2

热门文章

最新文章