大数据情报第四期(2018-07-16)

本文涉及的产品
实时计算 Flink 版,5000CU*H 3个月
简介: 《使用Kafka Streams构建事件溯源系统的经验分享》近期在乌克兰基辅举行的JEEConf大会上,Amitay Horwitz介绍了他的团队是如何实现一个事件溯源的发票系统、系统两年半生产环境运行期间所遇到的挑战,以及团队是如何使用Kafka Streams实现新的设计。

《使用Kafka Streams构建事件溯源系统的经验分享》近期在乌克兰基辅举行的JEEConf大会上,Amitay Horwitz介绍了他的团队是如何实现一个事件溯源的发票系统、系统两年半生产环境运行期间所遇到的挑战,以及团队是如何使用Kafka Streams实现新的设计。

《使用Apache Kafka和KSQL实现普及化流处理》大多数的流处理技术,需要开发人员使用Java或Scala等编程语言编写代码。KSQL是Apache Kafka的数据流SQL引擎,它使用SQL语句替代编写大量代码去实现流处理任务。KSQL基于Kafka的Stream API构建,它支持过滤、转换、聚合、连接、加窗操作和Sessionization(即捕获单一会话期间的所有的流事件)等流处理操作。KSQL的用例涉及实现实时报表和仪表盘、基础设施和物联网设备监控、异常检测和欺骗行为报警等。

《谷歌Kubernetes引擎上的GPU现已普遍可用》谷歌宣布可在Kubernetes引擎(GKE)中普遍使用GPU。与最近发布的1.10正式版GKE一起,用户可以将机器学习(ML)工作负载放在上面,并利用GPU的强大处理能力。

《Announcing the general availability of Azure SQL Data Sync》We are delighted to announce the general availability (GA) of Azure SQL Data Sync! Azure SQL Data Sync allows you to synchronize data between Azure SQL Database and any other SQL endpoints unidirectionally or bidirectionally. It enables hybrid SQL deployment and allows local data access from both Azure and on-premises application. It also allows you to deploy your data applications globally with a local copy of data in each region, and keep data synchronized across all the regions. It will significantly improve the application response time and reliability by eliminating the impact of network latency and connection failure rate.

《Apache Flink 1.5.1 Released》The Apache Flink community released the first bugfix version of the Apache Flink 1.5 series.This release includes more than 60 fixes and minor improvements for Flink 1.5.0. The list below includes a detailed list of all fixes.

《June Preview Release: Packing Confluent Platform with the Features You Requested!》We are very excited to announce the Confluent Platform June 2018 Preview. This is our most feature-packed preview release for Confluent Platform since we started doing our monthly preview releases in April 2018.

《New Azure innovation advances customer success for the cloud- and AI-powered future》Organizations around the world are gearing up for a future powered by the intelligent cloud and AI. As these technologies become increasingly central to business strategy and transformation, Microsoft is committed to delivering cutting-edge innovations, programs and expertise that help our customers navigate these technological and business shifts.

《Azure sets new performance benchmarks with SQL Data Warehouse》As the amount of data grows exponentially, the pressure to quickly harness it for insights to share across the organization also increases rapidly. As Microsoft continues to evolve our analytics portfolio, we are committed to delivering a data warehouse solution that provides a fast, flexible, and secure analytics platform in the cloud.

《Kafka 1.0 on HDInsight lights up real time analytics scenarios》Data engineers love Kafka on HDInsight as a high-throughput, low-latency ingestion platform in their real time data pipeline. They already leverage Kafka features such as message compressionconfigurable retention policy, and log compaction. With the release of Apache Kafka 1.0 on HDInsight, customers now get key features that make it easy to implement the most demanding scenarios.

《Using Apache Spark DStreams with Cloud Dataproc and Cloud Pub/Sub》Apache Spark offers two APIs for streaming: the original Discretized Streams API, or DStreams, and the more recent Structured Streaming API, which was released as an alpha in Spark 2.0 and as a stable release in Spark 2.2. While Structured Streaming offers several new, important features like event time operations and the Datasets and DataFrames abstractions, it also has some limitations. For example, Structured Streaming does not yet support operations such as sorting or multiple streaming aggregations.

《A one size fits all database doesn't fit anyone》Seldom can one database fit the needs of multiple distinct use cases. The days of the one-size-fits-all monolithic database are behind us, and developers are now building highly distributed applications using a multitude of purpose-built databases. Developers are doing what they do best: breaking complex applications into smaller pieces and then picking the best tool to solve each problem. The best tool for a job usually differs by use case.

相关实践学习
基于Hologres轻松玩转一站式实时仓库
本场景介绍如何利用阿里云MaxCompute、实时计算Flink和交互式分析服务Hologres开发离线、实时数据融合分析的数据大屏应用。
基于MaxCompute的热门话题分析
Apsara Clouder大数据专项技能认证配套课程:基于MaxCompute的热门话题分析
目录
相关文章
|
数据采集 BI
智慧公安情报研判分析系统开发,大数据分析平台建设
智慧公安情报研判分析系统是集基础信息采集、情报信息研判、数据查询、从底层数据采集到高端研判应用自上至下贯穿整个公安局情报信息化业务。
659 0
|
存储 数据可视化 Oracle
公安情报研判平台建设,大数据可视化系统开发方案
情报研判平台,首先在公安各警种情报工作需求之上建立统一的研判基础平台,提供研判信息资源整合和分析、研判、发布平台,通过统一的基础平台确保各警种研判信息来源的丰富和统一。
491 0
|
数据可视化 大数据 数据挖掘
公安情报研判系统开发,大数据可视化平台建设方案
公安情报研判系统开发,面向公安情报部门,通过对海量非结构化原始情报文本进行深度语义理解、自动价值分拣、智能标签提取,实现情报分拣自动化、标签提取全面化、串并研判智能化、风险预警实时化,生成以人-群-事为核心的立体式数据统计与分析,辅助情报深度研判与风险预警,为事件-人员-指令全流程提供。
403 0
|
大数据 Apache 数据库
大数据情报第三期(2018-07-02)
《OpenAI Dota2 5v5模式击败人类,AI每天训练量抵人类180年》今天凌晨,OpenAI通过官方博客宣布了其在Dota对抗上的新进展——由五个神经网络组成的团战AI团队,在5v5中击败了业余人类玩家,并表示,将有望挑战顶级专业团队。
1533 0
|
监控 数据可视化 大数据
利用“大数据、云计算”提高情报分析以打击反恐
导读:  美国国防部长卡特曾赴硅谷招募顶尖科技人才。近年来的信息大爆炸使得五角大楼不得不将目光聚焦硅谷,以打击反恐。神秘的大数据平台Palantir就是美国CIA、FBI等寻求的合作对象。Palantir最为人津津乐道的案例有两个,一是此前美国政府追捕本拉登行动中,Palantir扮演了重要的情报分析的角色;二是Palantir协助多家银行追回了纳斯达克前主席麦道夫BernieMadoff所隐藏起来的数十亿美元巨款。
2368 0
|
2月前
|
数据采集 分布式计算 DataWorks
ODPS在某公共数据项目上的实践
本项目基于公共数据定义及ODPS与DataWorks技术,构建一体化智能化数据平台,涵盖数据目录、归集、治理、共享与开放六大目标。通过十大子系统实现全流程管理,强化数据安全与流通,提升业务效率与决策能力,助力数字化改革。
68 4

热门文章

最新文章