Certificates used by CB's internal Lucee server

Hi Seb,

Have you installed the GoDaddy intermediary certificate on the GitLab server, which would have come with the certificate when it was issued? I’ve seen issues before with Lucee and GoDaddy certs and it has always been because either the intermediary certificate was not installed or was installed incorrectly, essentially meaning it was not installed. Java seems to have something against GoDaddy certs which means it doesn’t validate them without the intermediary certificate.

I have just run into what looks like a similar issue trying to start Commandbox 4.4.0 on Ubuntu 16.04.

I am on an enterprise network, running under a VirtualBox VM.
What do I request of my VM builder/network admins?

Commandbox installed with no errors, but attempting to run it with box results in the error following:

$ box
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at lucee.loader.engine.CFMLEngineFactory.downloadBundle(CFMLEngineFactory.java:800)
at lucee.loader.osgi.BundleLoader.loadBundles(BundleLoader.java:152)
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:388)
at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:278)
at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:174)
at lucee.runtime.script.BaseScriptEngineFactory.(BaseScriptEngineFactory.java:61)
at lucee.runtime.script.LuceeScriptEngineFactory.(LuceeScriptEngineFactory.java:27)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:122)
at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:84)
at javax.script.ScriptEngineManager.(ScriptEngineManager.java:75)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:261)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:150)
at cliloader.LoaderCLIMain.main(LoaderCLIMain.java:506)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
… 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
… 37 more
javax.servlet.ServletException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:407)
at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:278)
at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:174)
at lucee.runtime.script.BaseScriptEngineFactory.(BaseScriptEngineFactory.java:61)
at lucee.runtime.script.LuceeScriptEngineFactory.(LuceeScriptEngineFactory.java:27)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:122)
at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:84)
at javax.script.ScriptEngineManager.(ScriptEngineManager.java:75)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:261)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:150)
at cliloader.LoaderCLIMain.main(LoaderCLIMain.java:506)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at lucee.loader.engine.CFMLEngineFactory.downloadBundle(CFMLEngineFactory.java:800)
at lucee.loader.osgi.BundleLoader.loadBundles(BundleLoader.java:152)
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:388)
… 18 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
… 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
… 37 more
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider lucee.runtime.script.LuceeScriptEngineFactory could not be instantiated
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at lucee.loader.engine.CFMLEngineFactory.downloadBundle(CFMLEngineFactory.java:800)
at lucee.loader.osgi.BundleLoader.loadBundles(BundleLoader.java:152)
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:388)
at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:278)
at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:174)
at lucee.runtime.script.BaseScriptEngineFactory.(BaseScriptEngineFactory.java:61)
at lucee.runtime.script.CFMLScriptEngineFactory.(CFMLScriptEngineFactory.java:27)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:122)
at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:84)
at javax.script.ScriptEngineManager.(ScriptEngineManager.java:75)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:261)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:150)
at cliloader.LoaderCLIMain.main(LoaderCLIMain.java:506)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
… 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
… 37 more
javax.servlet.ServletException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:407)
at lucee.loader.engine.CFMLEngineFactory.initEngineIfNecessary(CFMLEngineFactory.java:278)
at lucee.loader.engine.CFMLEngineFactory.getInstance(CFMLEngineFactory.java:174)
at lucee.runtime.script.BaseScriptEngineFactory.(BaseScriptEngineFactory.java:61)
at lucee.runtime.script.CFMLScriptEngineFactory.(CFMLScriptEngineFactory.java:27)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:380)
at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
at javax.script.ScriptEngineManager.initEngines(ScriptEngineManager.java:122)
at javax.script.ScriptEngineManager.init(ScriptEngineManager.java:84)
at javax.script.ScriptEngineManager.(ScriptEngineManager.java:75)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:261)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:150)
at cliloader.LoaderCLIMain.main(LoaderCLIMain.java:506)
Caused by: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1946)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:316)
at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:310)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1639)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:223)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1037)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:965)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1064)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:559)
at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185)
at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162)
at lucee.loader.engine.CFMLEngineFactory.downloadBundle(CFMLEngineFactory.java:800)
at lucee.loader.osgi.BundleLoader.loadBundles(BundleLoader.java:152)
at lucee.loader.engine.CFMLEngineFactory.initEngine(CFMLEngineFactory.java:388)
… 18 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
at sun.security.validator.Validator.validate(Validator.java:262)
at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
… 31 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
… 37 more
ScriptEngineManager providers.next(): javax.script.ScriptEngineFactory: Provider lucee.runtime.script.CFMLScriptEngineFactory could not be instantiated
java.lang.NullPointerException
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:283)
at cliloader.LoaderCLIMain.execute(LoaderCLIMain.java:150)
at cliloader.LoaderCLIMain.main(LoaderCLIMain.java:506)
vagrant@vagrant ruby-2.4.3 ~/.CommandBox
$

That error stack trace shows that Lucee in the CLI was trying to download OSGI bundles on first start. Now, to start with, I don’t know why the heck Lucee is trying to do this. Micha has promised me over and over that every stable release of Lucee should have all the bundles it needs and shouldn’t need to download anything. That said, he also doesn’t have any flag in Lucee for me to disable that feature. Also, I’ve never been able to figure out how to enable the logging for the Lucee engine in JSR-223 to see the output from the Lucee loader that explains what it was downloading. I’ve asked, but never gotten a reply on that.

So all that said, I’m guessing the SSL cert has an issue on the Lucee update site that is being downloaded from. I’m not sure what URL is being hit right off without those logs that don’t work so I’m not sure how to troubleshoot this unless you can run some network diagnostics to find out what URLs are being hit.

I’m running into this error with downloadBundle getting called by Lucee, and it’s not able to because it’s on a machine with no internet access. Is there any update on how this can be addressed? We have to deliver a much larger ZIP deliverable with the server pre-built because of this issue.

I know of no workaround to this. I can only suggest you bug the Lucee dev team. Lord knows, I have for years but the download “feature” has never been removed. I’m sort of at a loss. Firstly, Lucee isn’t even supposed to be downloading anything so it’s a mystery why it’s even trying. But when Lucee does try to download OSGI bundles, there’s nothing you can do to prevent it. II’s all hard-coded in.

This is one of the most recent tickets on the issue (it affects more than just CommandBox users)

https://luceeserver.atlassian.net/browse/LDEV-2134

Thanks I posted a nag comment there as well, may get others to do so soon. Very frustrating issue that has cost us hours.

I ran into this issue today with a networksolutions certificate. It can be resolved by installing the certificate authority certificate they give you into the lucee cacerts keystore. I’m looking into how to automate this. Unfortunately, adding -Djavax.net.ssl.keyStore and password arguments don’t seem to resolve this either.