Services and State with Ambari REST API

本文涉及的产品
服务治理 MSE Sentinel/OpenSergo,Agent数量 不受限
注册配置 MSE Nacos/ZooKeeper,118元/月
云原生网关 MSE Higress,422元/月
简介:

The Ambari management tool for Hadoop offers among other handy tools a comprehensive REST API for cluster administration. Logically a cluster is divided into hosts, services and service components. While the UI might not always has support for all needed scenarios sure the REST API can be used to achieve it. For example moving a master component of a service from one host to another.

In this post we are going to look a little closer at the way the Ambari API can be used to manage Hadoop services. At the end of this post you will find a list of all the currently supported Hadoop services with all the needed master, slave and client components that can be manged and administrated within your HDP stack. Also this posts contains the possible states and state transitions a component might have which could become useful when facing problems like Host config is in invalid state.

The possible states a service component can be in is listed below. To change or get the state of a service component the following API calls can be issued.

1
2
3
4
5
6
7
8
// Getting the state of the component
curl -k -u admin:admin -H  "X-Requested-By:ambari"  -i -X GET
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT 
// Setting the state of the component to INSTALLED
curl -k -u admin:admin -H  "X-Requested-By:ambari"  -i -X PUT 
-d '{ "RequestInfo" :{ "context" : "Install YARN_CLIENT" },
"Body" :{ "HostRoles" :{ "state" : "INSTALLED" }}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/YARN_CLIENT

Among the need to change the state of a service component a very common issue is the need to actually move one service – most likely a master service – to different host. Users faced with this problems often need to issue the existing API due to a lack of support by the existing UI – at least with older releases of Ambari Web UI. This is an example of moving the OOZIE_SERVER to a new host.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Stop the old component by putting it back in state INSTALLED
curl -u admin:admin -H  "X-Requested-By:ambari"  -i -X PUT 
-d  '{"RequestInfo": {"context": "Stop Oozie","query":"HostRoles/component_name.in(' OOZIE_SERVER')"}, 
"Body" :{ "HostRoles" : { "state" "INSTALLED" }}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER
// Add component to host
curl -u admin:admin -H  "X-Requested-By:ambari"  -i -X POST -d 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER
// Install component by achieving stated INSTALLED
curl -u admin:admin -H  "X-Requested-By:ambari"  -i -X PUT 
-d  '{"RequestInfo": {"context": "Install Oozie","query":"HostRoles/component_name.in(' OOZIE_SERVER')"}, 
"Body" :{ "HostRoles" : { "state" "INSTALLED" }}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER
// Delete old component
curl -u admin:admin -H  "X-Requested-By:ambari"  -i -X DELETE 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<OLD_HOST_FQDN>/host_components/OOZIE_SERVER
// Starting the new component
curl -u admin:admin -H  "X-Requested-By:ambari"  -i -X PUT 
-d  '{"RequestInfo": {"context": "Start Oozie","query":"HostRoles/component_name.in(' OOZIE_SERVER')"}, 
"Body" :{ "HostRoles" : { "state" "STARTED" }}}' 
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts
/<NEW_HOST_FQDN>/host_components/OOZIE_SERVER

Removing a complete service from the cluster would require all the removal of all it’s components together with the service itself. For example for removing the Knox service we would need to remove the KNOX_GATEWAY component from the cluster, but before let’s get their state:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
// Get service info
curl -u admin:admin -H  "X-Requested-by:ambari"  -i -k -X GET
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/
// Stop component KNOX_GATEWAY
curl -u admin:admin -H  "X-Requested-by:ambari"  -i -k -X PUT 
-d  '{"ServiceComponentInfo": {"state": "INSTALLED"}}'
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX/components/KNOX_GATEWAY
// Stop service KNOX
curl -u admin:admin -H  "X-Requested-by:ambari"  -i -k -X PUT 
-d  '{"ServiceInfo": {"state": "INSTALLED"}}'
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX
// Delete component
curl -u admin:admin -H  "X-Requested-by:ambari"  -i -k -X DELETE
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/hosts/<HOST_FQDN>/host_components/KNOX_GATEWAY
// Delete service
curl -u admin:admin -H  "X-Requested-by:ambari"  -i -k -X DELETE
http://<HOST>:8080/api/v1/clusters/<CLUSTER_NAME>/services/KNOX

Possible states of a component:


StateDescription

INITInitial/Clean state.

INSTALLINGIn the process of installing.

INSTALL_FAILEDInstall failed.

INSTALLEDState when install completed successfully.

STARTINGIn the process of starting.

STARTEDState when start completed successfully.

STOPPINGIn the process of stopping.

UNINSTALLINGIn the process of uninstalling.

UNINSTALLEDState when uninstall completed successfully.

WIPING_OUTIn the process of wiping out the install.

UPGRADINGIn the process of upgrading the deployed bits.

DISABLEDDisabled master’s backup state.

UNKNOWNState could not be determined.



Allowed State transitions of a component:


Desired StateStart State

INSTALLED <-INIT, UNINSTALLED, INSTALLED, INSTALLING, STARTED, INSTALL_FAILED, UPGRADING, STOPPING, UNKNOWN, DISABLED

STARTED <-INSTALLED, STARTING, STARTED

UNINSTALLED <-INSTALLED, UNINSTALLED, UNINSTALLING

INIT <-UNINSTALLED, INIT, WIPING_OUT

DISABLED <-INSTALLED, INSTALL_FAILED, UNKNOWN



Services

A Hadoop cluster contains of multiple services that individually have either master, slave or client components. Below you will find a list of all currently supported components as part of a HDP stack divided in master, slave, or client groups. The cardinality notates the number of components that can exist at the same time in one cluster.

Master Services

NameAmbari Component Name

Service

Cardinality

NameNodeNAMENODEHDFS1-2

Secondary NameNodeSECONDARY_NAMENODEHDFS1

ResourceMangerRESOURCEMANAGERYARN1-2

Application Timeline ServerAPP_TIMELINE_SERVERYARN1

HistoryServerHISTORYSERVERMAPREDUCE2 1

Hive MetastoreHIVE_METASTOREHIVE1-2

HiveServer2HIVE_SERVERHIVE1-2

WebHcat ServerWEBHCAT_SERVERHIVE1

HBase MasterHBASE_MASTERHBASE1+

Spark Job History ServerSPARK_JOBHISTORYSERVERSPARK1

Nimbus ServerNIMBUSSTORM1

Storm REST ServerSTORM_REST_APISTORM1

Storm UISTORM_UI_SERVERSTORM1

DRPC ServerDRPC_SERVERSTORM1

Falcon ServerFALCON_SERVERFALCON1

ZookeeperZOOKEEPER_SERVERZOOKEEPER1+ (odd #)

Kafka BrokerKAFKA_BROKERKAFKA1+

Knox GatewayKNOX_GATEWAYKNOX1+

Ranger Admin ServerRANGER_ADMINRANGER1-3

Ranger User SyncRANGER_USERSYNCRANGER1

Ranger Key Management ServerRANGER_KMS_SERVERRANGER_KMS1+

Oozie ServerOOZIE_SERVEROOZIE1

Ganglia ServerGANGLIA_SERVERGANGLIA1

Nagios ServerNAGIOS_SERVERNAGIOS1

Ambari Metrics ServiceMETRICS_COLLECTORAMS1

Zeppelin ServerZEPPELIN_MASTERSPARK / HIVE1



Slave Services

NameAmbari Component Name

ServiceCardinality

DataNodeDATANODEHDFS1+

Journale Nodes for NameNode HAJOURNALNODEHDFS0+ (odd #)

Zookeeper Failover ServiceZKFCHDFS0+

Secondary NameNodeNFS_GATEWAYHDFS0+

Node ManagerNODEMANAGERYARN1+

HBase RegionServerHBASE_REGIONSERVERHBASE1+

Phoneix Query ServerPHOENIX_QUERY_SERVERHBASE0+

Storm SupervisorSUPERVISORSTORM1+

Ganglia Metrics CollectorGANGLIA_MONITORGANGLIAALL

Ambari Metrics CollectorMETRICS_MONITORAMSALL


Clients

NameAmbari Component Name

ServiceCardinality

HDFS ClientHDFS_CLIENTHDFS1+

YARN ClientYARN_CLIENTYARN1+

MapReduce ClientMAPREDUCE2_CLIENTMAPREDUCE21+

Spark ClientSPARK_CLIENTSPARK1+

Falcon ClientFALCON_CLIENTFALCON1+

HBase ClientHBASE_CLIENTHBASE1+

Hive ClientHIVE_CLIENTHIVE1+

HCat ClientHCATHIVE1+

Mahout ClientMAHOUTMAHOUT0+

Oozie ClientOOZIE_CLIENTOOZIE1+

Sqoop ClientSQOOPSQOOP1+

Zookeeper ClientZOOKEEPER_CLIENTZOOKEEPER1+










本文转自 yntmdr 51CTO博客,原文链接:http://blog.51cto.com/yntmdr/1775098,如需转载请自行联系原作者
相关实践学习
基于MSE实现微服务的全链路灰度
通过本场景的实验操作,您将了解并实现在线业务的微服务全链路灰度能力。
目录
相关文章
|
5天前
|
JSON API 网络架构
gRPC 与 REST 的比较分析:哪种 API 适合您的开发需求?
gRPC, 由 Google 推出的开源远程过程调用(RPC)框架, 使两个应用程序间的方法调用变得简单,支持结构化数据的交换。通过采用 Protocol Buffers (Protobuf) ——一种与语言无关的接口定义语言,gRPC 体现了许多现代网络通信技术的优势
gRPC 与 REST 的比较分析:哪种 API 适合您的开发需求?
|
6天前
|
数据采集 DataWorks 测试技术
DataWorks产品使用合集之如何通过REST API进行数据采集,并且自定义传入API的参数
DataWorks作为一站式的数据开发与治理平台,提供了从数据采集、清洗、开发、调度、服务化、质量监控到安全管理的全套解决方案,帮助企业构建高效、规范、安全的大数据处理体系。以下是对DataWorks产品使用合集的概述,涵盖数据处理的各个环节。
|
6天前
|
JavaScript 应用服务中间件 API
Node.js搭建REST API实战:从基础到部署
【7月更文挑战第18天】通过以上步骤,你可以将你的Node.js REST API从开发环境顺利迁移到生产环境,并利用各种工具和技术来确保应用的稳定性、安全性和可扩展性。
|
6天前
|
分布式计算 DataWorks API
DataWorks产品使用合集之使用REST API Reader往ODPS写数据时,如何获取入库时间
DataWorks作为一站式的数据开发与治理平台,提供了从数据采集、清洗、开发、调度、服务化、质量监控到安全管理的全套解决方案,帮助企业构建高效、规范、安全的大数据处理体系。以下是对DataWorks产品使用合集的概述,涵盖数据处理的各个环节。
|
5天前
|
前端开发 JavaScript API
SharePoint Search REST API 获取数据
【7月更文挑战第6天】使用SharePoint Search REST API进行搜索涉及发送AJAX请求到`/_api/search/query`,其中`querytext`包含搜索关键词,`sourceid`指定结果源ID。示例代码展示了使用jQuery的`.ajax()`方法,成功后通过`success`回调处理返回的搜索结果数据。记得替换URL占位符并确保有相应权限。返回数据的结构可能因配置而异,可能需要进一步解析。还要考虑身份验证、分页和其他查询参数。查阅官方文档可了解更多复杂操作和API使用。
|
15天前
|
安全 Java API
构建基于Spring Boot的REST API安全机制
构建基于Spring Boot的REST API安全机制
|
19天前
|
开发框架 Java API
Java中的REST API开发详解
Java中的REST API开发详解
|
22天前
|
开发框架 安全 .NET
使用VB.NET构建Web服务和REST API的指南
【7月更文挑战第2天】使用VB.NET构建Web服务和REST API的指南:从Web服务基础到ASP.NET Core实践,涵盖控制器、路由、模型绑定、安全措施(如JWT、HTTPS)及测试、部署(Azure、Docker)与监控工具。了解如何利用VB.NET在现代云环境中创建高效、安全的API。开始你的VB.NET Web服务开发之旅!**
25 1
|
21天前
|
开发框架 Java API
Java中的REST API开发详解
Java中的REST API开发详解
|
22天前
|
缓存 Java API
Java中设计和优化REST API的技巧
Java中设计和优化REST API的技巧

热门文章

最新文章