Breakthrough in Alibaba Cloud Computing Capabilities - BigBench Reaches 100 TB World Record

简介: In the first day of the 2017 Hangzhou Computing Conference on Oct. 11, Alibaba Cloud President Hu Xiaoming introduced a next-generation computing platform MaxCompute + PAI.
+关注继续查看

AW7835J_AW7835J_ComparisonofSmallMessageSendingPerformancesofKafkaRabbitMQandRocketMQ

In the first day of the 2017 Hangzhou Computing Conference on Oct. 11, Alibaba Cloud President Hu Xiaoming introduced a next-generation computing platform MaxCompute + PAI.

01

In the main forum on the 12th, Zhou Jingren, Alibaba Group Vice President and director from the Search Division and Computing Platform Division, said that data lays the foundation for artificial intelligence innovation, and possessing plenty of computing capabilities to help fully release the value of the data. Later, Zhou Jingren released BigBench On MaxCompute[1] 2.0 + PAI with Rob Hays, Vice President of Intel's Data Center Division. The release broke the best records set by TPCx-BB[2] and reflected the extremely robust data processing capabilities of MaxCompute and the absolute strength of public cloud compared to the traditional model.

02

At present, the maximum capacity publicized by TPC is 10 TB, the best performance is 1491.23 BBQpm, and the best price/performance ratio is 589 Price/BBQpm. Alibaba Cloud's BigBench on MaxCompute 2.0+PAI expands that capacity to 100 TB for the first time in the world, which is also the first benchmark to be based on public cloud services. Engines running on this platform achieve 7000 points.

It was reported that MaxCompute test environment would be open for one month on public cloud after the conference and that the BigBench On MaxCompute+PAI SDK (inherited from TPCx-BigBench and enabling it to run on the big data environment of Alibaba Cloud) would be open-source for developers to use.

The great capacity breakthrough of BigBench on MaxCompute owes to MaxCompute's mass data processing capabilities and machine learning algorithm efficiency. MaxCompute, based on the Apsara distributed OS developed by Alibaba Cloud, can connect more than 10,000 servers in a single cluster and process Exabytes of data.

MaxCompute next-generation engines get continuous and in-depth performance optimization in the Compiler, Optimizer, and runtime. In addition to high-performance computing, Alibaba Cloud PAI provides users with a robust algorithm experiment platform which includes traditional machine learning as well as the latest in deep learning and enhanced learning. PAI provides a great number of algorithms and tools to meet algorithm requirements in different business scenarios. The platform is also optimized for performance and data capacity.

Furthermore, MaxCompute and Intel processor integration and in-depth optimization enable full use of Intel Xeon® Scalable Processor's structural strengths. Rob Hays, Vice President of Intel's data center division, said "We are delighted to be working with Alibaba Cloud to optimize MaxCompute on the latest Intel® Xeon® Scalable processor platform and to witness the excellent performance of MaxCompute in the BigBench test."

Well, What computing bonuses does BigBench on MaxCompute2.0+PAI bring for developers to help them seize more market opportunities?

  1. Break through the capacity bottleneck. When BigBench data capacity exceeds 10 TB, most products will be bottlenecked and unable to expand. BigBench on MaxCompute enables data capacity to be expanded to 100 TB, which meets users' increasing data capacity requirements.
  2. Lower cost. Conventional hardware + software building mode requires servers. Though the server cost can be apportioned throughout the lifespan of the servers, purchasing hardware means that your future computing resources come at an increased relative cost, as hardware inevitably drops in price year by year. BigBench employs price/QPM to calculate the price/performance ratio. Compared with the conventional hardware mode, MaxCompute supports prepayment and data-based payment, which offers pricing flexibility and competitive price/performance ratios.
  3. Meet scalability requirement. The demand of data on the internet means that an explosion of traffic could happen at any time. The traditional hardware model requires a long adjustment period to meet increased demand. BigBench on MaxCompute enables on-demand computing capacity expansion, satisfying enterprises' capacity expansion requirements at any particular time.
  4. Save O&M workload. Traditionally, a data room needs to be maintained by an O&M team. Usually, the maintenance quality cannot be guaranteed. BigBench on MaxCompute runs on public cloud, saving enterprise customers from investing additional manpower to carry out maintenance.
    BigBench on MaxCompute is modified based on TPCx-BB, so it is compatible with all TPCx-BB semantics. As an industrial benchmark, TPCx-BB covers all operation types of big data processing, including SQL, MapReduce, Streamling, and MachineLearning. The full coverage capability of BigBench on MaxCompute reflects MaxCompute' software stack integrity in big data processing. The following table lists the software stacks of BigBench on MaxCompute:

03

BigBench on MaxCompute is also an industrial benchmark, which demonstrates the software stack integrity of MaxCompute in big data processing and the superior performance in capacity, cost, and scalability.

BigBench on MaxCompute is very easy to access. Enterprise customers can connect to the platform provided they have prepared:

  1. Alibaba Cloud account;
  2. BigBench on MaxCompute toolkit;
  3. and MaxCompute client.
    For details on the platform use, see MaxCompute Official Documentations.

[1] BigBench on MaxCompute is derived from TPCx-BB, so it is compatible with all TPCx-BB semantics.
[2] TPCx-BB (BigBench) was released by Transaction Processing Performance Council (TPC) in Feb. 2016. First E2E big data analysis app-level benchmark.

相关实践学习
简单用户画像分析
本场景主要介绍基于海量日志数据进行简单用户画像分析为背景,如何通过使用DataWorks完成数据采集 、加工数据、配置数据质量监控和数据可视化展现等任务。
SaaS 模式云数据仓库必修课
本课程由阿里云开发者社区和阿里云大数据团队共同出品,是SaaS模式云原生数据仓库领导者MaxCompute核心课程。本课程由阿里云资深产品和技术专家们从概念到方法,从场景到实践,体系化的将阿里巴巴飞天大数据平台10多年的经过验证的方法与实践深入浅出的讲给开发者们。帮助大数据开发者快速了解并掌握SaaS模式的云原生的数据仓库,助力开发者学习了解先进的技术栈,并能在实际业务中敏捷的进行大数据分析,赋能企业业务。 通过本课程可以了解SaaS模式云原生数据仓库领导者MaxCompute核心功能及典型适用场景,可应用MaxCompute实现数仓搭建,快速进行大数据分析。适合大数据工程师、大数据分析师 大量数据需要处理、存储和管理,需要搭建数据仓库?学它! 没有足够人员和经验来运维大数据平台,不想自建IDC买机器,需要免运维的大数据平台?会SQL就等于会大数据?学它! 想知道大数据用得对不对,想用更少的钱得到持续演进的数仓能力?获得极致弹性的计算资源和更好的性能,以及持续保护数据安全的生产环境?学它! 想要获得灵活的分析能力,快速洞察数据规律特征?想要兼得数据湖的灵活性与数据仓库的成长性?学它! 出品人:阿里云大数据产品及研发团队专家 产品 MaxCompute 官网 https://www.aliyun.com/product/odps 
目录
相关文章
|
7月前
|
Serverless 容器
s to describe what capacity cloud native database serv
No less than 800 words to describe what capacity cloud native database serverless, and cloud native database serverless how to help the business "cost reduction and efficiency"
28 0
|
7月前
《Alibaba_Cloud_Whitepaper_-_The_Cloud_is_Taking_Over_Ecommerce》电子版地址
Alibaba_Cloud_Whitepaper_-_The_Cloud_is_Taking_Over_Ecommerce
25 0
《Alibaba_Cloud_Whitepaper_-_The_Cloud_is_Taking_Over_Ecommerce》电子版地址
|
8月前
|
网络协议 关系型数据库 MySQL
|
8月前
|
监控 网络协议 安全
Security Topic | Cloud computing (FREE)
云计算 Security 习题(试读)
67 0
|
8月前
|
Java 应用服务中间件 Linux
Operation Topic | Cloud computing (FREE)
云计算 Operation 习题(试读)
101 0
|
8月前
|
网络协议 安全 Unix
Admin & Engineer & Services Topic | Cloud computing (FREE)
云计算 Admin & Engineer & Services 习题(试读)
63 0
|
9月前
《CHOOSING PARTNERS AND SCENARIOS IN A MULTI-CLOUD WORLD》电子版地址
CHOOSING PARTNERS AND SCENARIOS IN A MULTI-CLOUD WORLD
33 0
《CHOOSING PARTNERS AND SCENARIOS IN A MULTI-CLOUD WORLD》电子版地址
|
9月前
|
安全
|
10月前
《Alibaba_Cloud_Whitepaper_-_Data_Security_is_Now_More_Important_Than_Ever.pdf》电子版地址
Alibaba_Cloud_Whitepaper_-_Data_Security_is_Now_More_Important_Than_Ever.pdf
45 0
《Alibaba_Cloud_Whitepaper_-_Data_Security_is_Now_More_Important_Than_Ever.pdf》电子版地址
推荐文章
更多