问题描述
按照我们之前的配置，在CDH7.1.1上为Ranger集成OpenLDAP认证这边文章中，我们为Ranger集成了OpenLDAP认证，刚开始给Hive、HDFS、HBase授权的时候，没发现有什么毛病，ldap用户都能正常登录和同步。后来使用Ranger API给用户批量授权时，将大量用户放在同一用户组里，比较好管理。这时我们才发现ldap的用户组没有被Ranger同步过来。
查看openldap，我们可以看到openldap已经创建etl_user用户和用户组。
# etl_user, Group, macro.com dn: cn=etl_user,ou=Group,dc=macro,dc=com objectClass: posixGroup objectClass: top cn: etl_user userPassword:: e2NyeXB0fXg= gidNumber: 50001 memberUid: etl_user # etl_user, People, macro.com dn: uid=etl_user,ou=People,dc=macro,dc=com uid: etl_user cn: etl_user objectClass: account objectClass: posixAccount objectClass: top objectClass: shadowAccount userPassword:: MTIzNDU2 shadowLastChange: 18235 shadowMin: 0 shadowMax: 99999 shadowWarning: 7 loginShell: /bin/bash uidNumber: 50001 gidNumber: 50001 homeDirectory: /home/etl_user # search result search: 2 result: 0 Success # numResponses: 143 # numEntries: 142
通过之前的配置，我们在ranger中也可以看到etl_user用户。
但是我们发现etl_user用户组却没有被同步过来。
问题分析
仔细查看下cloudera官网，发现ranger的ldap用户组配置有误，需要补充如下
修改配置，重启Ranger，发现还是无法同步ldap用户组。这时我们来看一看日志有没有相应的报错
我们可以看到类似的信息：
2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for etl_user 2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000 2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for oozie 2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000 2020-09-21 17:15:13,204 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for hbase 2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000 2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for sentry 2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000 2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No members available for impala 2020-09-21 17:15:13,205 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: timeStampVal = 20200209030017Zand currentDeltaSyncTime = 1581188417000
用ldap命令搜索etl_user的用户和用户组，看一下有什么问题：
etl_user用户
[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=People,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W Enter LDAP Password: # extended LDIF # # LDAPv3 # base <ou=People,dc=macro,dc=com> with scope subtree # filter: cn=etl_user # requesting: ALL # # etl_user, People, macro.com dn: uid=etl_user,ou=People,dc=macro,dc=com uid: etl_user cn: etl_user objectClass: account objectClass: posixAccount objectClass: top objectClass: shadowAccount userPassword:: MTIzNDU2 shadowLastChange: 18235 shadowMin: 0 shadowMax: 99999 shadowWarning: 7 loginShell: /bin/bash uidNumber: 50001 gidNumber: 50001 homeDirectory: /home/etl_user # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1
etl_user用户组
[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W Enter LDAP Password: # extended LDIF # # LDAPv3 # base <ou=Group,dc=macro,dc=com> with scope subtree # filter: cn=etl_user # requesting: ALL # # etl_user, Group, macro.com dn: cn=etl_user,ou=Group,dc=macro,dc=com objectClass: posixGroup objectClass: top cn: etl_user userPassword:: e2NyeXB0fXg= gidNumber: 50001 # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1
看样子etluser用户组似乎没有拥有etl_user这个用户。我们在etl_user用户组中添加memberUid这个属性
[root@cdh1 ~]# cat etl_user.ldif dn: cn=etl_user,ou=Group,dc=macro,dc=com changetype: modify add: memberUid memberUid: etl_user
然后修改用户
[root@cdh1 ~]# ldapmodify -Z -x -W -D "cn=Manager,dc=macro,dc=com" -f etl_user.ldif Enter LDAP Password: modifying entry "cn=etl_user,ou=Group,dc=macro,dc=com"
再看看etl_user用户组，用户组属性已经被添加进去了。
[root@cdh1 ~]# ldapsearch -h cdh1.macro.com -b "ou=Group,dc=macro,dc=com" -D "cn=Manager,dc=macro,dc=com" "cn=etl_user" -W Enter LDAP Password: # extended LDIF # # LDAPv3 # base <ou=Group,dc=macro,dc=com> with scope subtree # filter: cn=etl_user # requesting: ALL # # etl_user, Group, macro.com dn: cn=etl_user,ou=Group,dc=macro,dc=com objectClass: posixGroup objectClass: top cn: etl_user userPassword:: e2NyeXB0fXg= gidNumber: 50001 memberUid: etl_user # search result search: 2 result: 0 Success # numResponses: 2 # numEntries: 1
再重启ranger,我们可以发现etl_user用户组已经被同步了
同时，日志的信息已经正确了
2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: longName: etl_user, userName: etl_user 2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: No. of members in the group etl_user = 1 2020-09-22 00:38:14,436 INFO org.apache.ranger.ldapusersync.process.LdapDeltaUserGroupBuilder: LdapDeltaUserGroupBuilder.getGroups() completed with group count: 77 2020-09-22 00:38:18,281 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Unix Auth Service! 2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [SSLv2Hello] 2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1] 2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.1] 2020-09-22 00:38:18,546 INFO org.apache.ranger.authentication.UnixAuthenticationService: Enabling Protocol: [TLSv1.2] 2020-09-22 00:38:28,755 INFO org.apache.ranger.ldapusersync.process.LdapPolicyMgrUserGroupBuilder: valid cookie saved 2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: End: initial load of user/group from source==>sink 2020-09-22 00:38:29,254 INFO org.apache.ranger.usergroupsync.UserGroupSync: Done initializing user/group source and sink
小问题
还有个小问题忘了提及，第一次启动Ranger时会出现以下错误，
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes ErrorCode: 1071 2020-09-17 11:04:39,363 [E] ranger_core_db_mysql.sql file import failed! 2020-09-17 11:04:39,364 [I] Unable to create DB schema, Please drop the database and try again
这是因为建立索引的时候，字符长度超过255，由于uef-8字符最长只能为255，超长之后会报错，所以需要对MySQL做如下配置:
set global innodb_file_per_table = on, innodb_file_format = Barracuda, innodb_large_prefix = on;
