mvn 编译Spark代码的时候常见错误:
[ERROR] Failed to execute goal on project spark-tools_2.11: Could not resolve dependencies for project org.apache.spark:spark-tools_2.11:jar:2.4.3: Failed to collect dependencies at org.clapper:classutil_2.11:jar:1.1.2: Failed to read artifact descriptor for org.clapper:classutil_2.11:jar:1.1.2: Could not transfer artifact org.clapper:classutil_2.11:pom:1.1.2 from/to alimaven (http://maven.aliyun.com/nexus/content/groups/public/): java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty -> [Help 1]
[ERROR]
截图如下:
单独切换到tools模块
获取详细的报错信息
cd tools mvn -Pkubernetes -Phadoop-2.7 -Dhadoop.version=2.7.3 -Phive -Phive-thriftserver -DskipTests clean package -e
Caused by: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at java.security.cert.PKIXParameters.setTrustAnchors (PKIXParameters.java:200) at java.security.cert.PKIXParameters.<init> (PKIXParameters.java:120) at java.security.cert.PKIXBuilderParameters.<init> (PKIXBuilderParameters.java:104) at sun.security.validator.PKIXValidator.<init> (PKIXValidator.java:88) at sun.security.validator.Validator.getInstance (Validator.java:179) at sun.security.ssl.X509TrustManagerImpl.getValidator (X509TrustManagerImpl.java:312) at sun.security.ssl.X509TrustManagerImpl.checkTrustedInit (X509TrustManagerImpl.java:171) at sun.security.ssl.X509TrustManagerImpl.checkTrusted (X509TrustManagerImpl.java:184) at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted (X509TrustManagerImpl.java:124) at sun.security.ssl.ClientHandshaker.serverCertificate (ClientHandshaker.java:1508) at sun.security.ssl.ClientHandshaker.processMessage (ClientHandshaker.java:215) at sun.security.ssl.Handshaker.processLoop (Handshaker.java:1024) at sun.security.ssl.Handshaker.process_record (Handshaker.java:954) at sun.security.ssl.SSLSocketImpl.readRecord (SSLSocketImpl.java:1065) at sun.security.ssl.SSLSocketImpl.performInitialHandshake (SSLSocketImpl.java:1384) at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1412) at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1396)
Caused by: javax.net.ssl.SSLException: java.lang.RuntimeException: Unexpected error: java.security.InvalidAlgorithmParameterException: the trustAnchors parameter must be non-empty at sun.security.ssl.Alerts.getSSLException (Alerts.java:214) at sun.security.ssl.SSLSocketImpl.fatal (SSLSocketImpl.java:1958) at sun.security.ssl.SSLSocketImpl.fatal (SSLSocketImpl.java:1915) at sun.security.ssl.SSLSocketImpl.handleException (SSLSocketImpl.java:1898) at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1419) at sun.security.ssl.SSLSocketImpl.startHandshake (SSLSocketImpl.java:1396) at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket (SSLConnectionSocketFactory.java:396) at org.apache.maven.wagon.providers.http.httpclient.conn.ssl.SSLConnectionSocketFactory.connectSocket (SSLConnectionSocketFactory.java:355) at org.apache.maven.wagon.providers.http.httpclient.impl.conn.DefaultHttpClientConnectionOperator.connect (DefaultHttpClientConnectionOperator.java:142) at org.apache.maven.wagon.providers.http.httpclient.impl.conn.PoolingHttpClientConnectionManager.connect (PoolingHttpClientConnectionManager.java:373) at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.establishRoute (MainClientExec.java:381) at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.MainClientExec.execute (MainClientExec.java:237) at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.ProtocolExec.execute (ProtocolExec.java:185) at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RetryExec.execute (RetryExec.java:89) at org.apache.maven.wagon.providers.http.httpclient.impl.execchain.RedirectExec.execute (RedirectExec.java:111) at org.apache.maven.wagon.providers.http.httpclient.impl.client.InternalHttpClient.doExecute (InternalHttpClient.java:185) at org.apache.maven.wagon.providers.http.httpclient.impl.client.CloseableHttpClient.execute (CloseableHttpClient.java:83) at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.execute (AbstractHttpClientWagon.java:825) at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:986) at org.apache.maven.wagon.providers.http.wagon.shared.AbstractHttpClientWagon.fillInputData (AbstractHttpClientWagon.java:963) at org.apache.maven.wagon.StreamWagon.getInputStream (StreamWagon.java:126) at org.apache.maven.wagon.StreamWagon.getIfNewer (StreamWagon.java:88) at org.apache.maven.wagon.StreamWagon.get (StreamWagon.java:61) at org.eclipse.aether.transport.wagon.WagonTransporter$GetTaskRunne
这里需要分析底层的调用,其实就是java请求http的时候出现安全检查问题
我们忽略安全检查
mvn -Pkubernetes -Phadoop-2.7 -Dhadoop.version=2.7.3 -Phive -Phive-thriftserver -DskipTests clean package -Dmaven.wagon.http.ssl.insecure=true -Dmaven.wagon.http.ssl.allowall=true
即可成功