在前面hadoop+hive+hbase环境里面,hive部分简单的配置了基于MySQL的本地模式安装
但是考虑到安全,需要给hive添加认证登陆
而且,使用hive命令beeline链接hive,也是强行需要密码的
在之前的hive-site.xml配置中,hive.server2.authentication
的配置为NONE,表示没有用户认证。
而HiveServer2支持多种用户安全认证方式:NONE,NOSASL, KERBEROS, LDAP, PAM ,CUSTOM等等,在此,我们使用CUSTOM自定义安全认证
配置
安全认证代码
package org.apache.hive;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.hive.conf.HiveConf;
import org.apache.hive.service.auth.PasswdAuthenticationProvider;
import org.slf4j.Logger;
import javax.security.sasl.AuthenticationException;
/**
* @author SakuraGaara
* @date 2019/07/01
*/
public class CustomPasswdAuthenticator implements PasswdAuthenticationProvider {
private Logger LOG = org.slf4j.LoggerFactory.getLogger(CustomPasswdAuthenticator.class);
private static final String HIVE_JDBC_PASSWD_AUTH_PREFIX = "hive.jdbc_passwd.auth.%s";
private Configuration conf = null;
@Override
public void Authenticate(String userName, String passwd)
throws AuthenticationException {
LOG.info("user: " + userName + " try login.");
String passwdConf = getConf().get(String.format(HIVE_JDBC_PASSWD_AUTH_PREFIX, userName));
if (passwdConf == null) {
String message = "user's ACL configration is not found. user:" + userName;
LOG.info(message);
throw new AuthenticationException(message);
}
if (!passwd.equals(passwdConf)) {
String message = "user name and password is mismatch. user:" + userName;
throw new AuthenticationException(message);
}
}
public Configuration getConf() {
if (conf == null) {
this.conf = new Configuration(new HiveConf());
}
return conf;
}
public void setConf(Configuration conf) {
this.conf = conf;
}
}
将以上代码打包hive-1.0-SNAPSHOT.jar,将jar放置$HIVE_HOME/lib
目录
修改hive-site.xml文件配置
hive.server2.authentication // 指定认证方式CUSTOM
hive.server2.custom.authentication.class // 指定认证方式接口
hive.jdbc_passwd.auth // 设置用户为admin,value为用户密码
<property>
<name>hive.server2.authentication</name>
<value>CUSTOM</value>
<description>
Expects one of [nosasl, none, ldap, kerberos, pam, custom].
</description>
</property>
<property>
<name>hive.server2.custom.authentication.class</name>
<value>org.apache.hive.CustomPasswdAuthenticator</value>
<description>
Custom authentication class. Used when property
</description>
</property>
<property>
<name>hive.jdbc_passwd.auth.admin</name>
<value>12345678</value>
</property>
重新启动hiveserver2,此时可jps看出RunJar进程,则启动成功
nohup ./bin/hiveserver2 > /dev/null 2>&1 &
验证
使用beeline命令链接,此时需要用户密码
shell> beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/yibao/data/app/hive-2.3.5/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/yibao/data/app/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 2.3.5 by Apache Hive
beeline> !connect jdbc:hive2://master:10000
Connecting to jdbc:hive2://master:10000
Enter username for jdbc:hive2://master:10000: admin
Enter password for jdbc:hive2://master:10000: ********
Error: Could not open client transport with JDBC Uri: jdbc:hive2://master:10000: Failed to open new session: java.lang.RuntimeException: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.authorize.AuthorizationException): User: admin is not allowed to impersonate (state=08S01,code=0)
此时会发现,虽然配置了hive认证,但是使用beeline或者其他客户端,依旧无法连接hive
解决方案
1,首先确定hive是否开启
2,然后再hadoop下的core-site.xml加入配置
<property>
<name>hadoop.proxyuser.admin.hosts</name>
<value>*</value>
</property>
<property>
<name>hadoop.proxyuser.admin.groups</name>
<value>*</value>
</property>
注意配置中的“admin”代表的是一个用户,你异常中User后面的用户写的是哪个,你在这里就配置哪个。
hadoop.proxyuser.admin.hosts 配置成*的意义,表示任意节点使用hadoop集群的代理用户admin都能访问hdfs集群
hadoop.proxyuser.admin.groups 表示代理用户的组所属
3,然后在hadoop下的hdfs-site.xml中加入配置
<property>
<name>dfs.webhdfs.enabled</name>
<value>true</value>
</property>
4, 重启hadoop集群和hive,然后beeline重新连接hive
shell> beeline
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/yibao/data/app/hive-2.3.5/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/yibao/data/app/hadoop-2.7.7/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Beeline version 2.3.5 by Apache Hive
beeline> !connect jdbc:hive2://master:10000
Connecting to jdbc:hive2://master:10000
Enter username for jdbc:hive2://master:10000: admin
Enter password for jdbc:hive2://master:10000: ********
Connected to: Apache Hive (version 2.3.5)
Driver: Hive JDBC (version 2.3.5)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://master:10000> show databases;
+----------------+
| database_name |
+----------------+
| default |
| sakura |
+----------------+
2 rows selected (0.341 seconds)
0: jdbc:hive2://master:10000>