Apparent issue with CommandBox and POI 4.0

Communities CommandBox CLI
1

Hi. I’ve been working on a new version of my standalone Lucee Spreadsheet library which uses the newly released version 4.0 of POI.

POI 4.0 is a major, non-backwards compatible release but I’ve managed to work through the changes and all tests pass when running on my standard Lucee 5.2/'Tomcat 8.5 installation.

However when I next turned to use the awesomeness of CommandBox to run the suite against the other engines I support (which include ACF), a number of tests failed with the following exception:

java.lang.NoClassDefFoundError: Could not initialize class org.apache.poi.ooxml.util.DocumentHelper

At first I thought this was an ACF-specific issue as that’s the engine I first tested using CommandBox, but the same happened with my Lucee 4.5 and Lucee 5 embedded servers.

I won’t post the full stack trace because you can see it on this SO question reporting the same problem with POI 4.0:

Here the finger is pointed at an outdated XML parser in Wildfly, and as I believe that’s what CommandBox uses, my conclusion is that the issue is specific to CommandBox.

Unfortunately, adding the updated Xerces 2.12 jar to my JavaLoader paths as a “direct dependency” doesn’t fix things as it seems to have done for the SO questioner (who is working directly in Java not CFML obviously). Nor does replacing the jar in the embedded ACF’s lib path.

I’ve also tried updating the XmlBeans jar to 3.0.2 in the hope that the recent patch there would address the issue, but no joy.

Does anyone have any suggestions as to how I might manually patch my CommandBox install to use the newer jar, or have any other insight that might help?

If you want to run the test suite yourself, the code is on the develop branch:

Thanks!
Julian (aka CFSimplicity)

2

Hmm, there may not be a good answer with Adobe CF since they do not use OSGI. Depending on the classloader that is used to load the new POI jars, it may work in some cases but it will work more on accident than anything. Since nearly all of the core Lucee jars are in OSGI bundles these days, you can get away with dropping a lot of stuff in the system classpath without conflicts. And of course, even if you have conflict with the servlet jars, you can turn your project into using OSGI. When it comes to Adobe CF, it really matters what classloader is being asked to create the java classes and what classloader actually loaded the jars. Class loaders have a child/parent hierarchy so if the child classloader doesn’t find the class, it will ask its parent. That’s basically how OSGI works.

3

I'm no java expert, Brad, but I'm not sure it's a class loading issue.

I'm aware of the "accidental" nature of loading when you have a class
clash in a non OSGi environment, but I'm using JavaLoader here and in
my experience that avoids clashes since it has its own network class
loader. That's why the current version of my library works 100%
reliably even with the older POI jars present in the ACF and Lucee 4.5
engine lib paths.

Also the same thing is happening with the embedded Lucee 5 server
which is OSGi, so there shouldn't be any clashes.

As I say, this does appear to be specific to WildFly, but I don't
really know where to look to check it, or change the Xerces
dependency.

Cheers
Julian.

4

Interesting. It would sound then like the POI lib is using a jar that is part of the servlet jars or just the core servlet stuff. I’m not familiar with the POI libs specifically as it’s been a few years since I used them directly. If the Xerces classes are specifically the ones that are picking up the wrong version then it would seem that adding that jar into the same class loader as your POI jars would make sense.

I also noticed that one of those pages specifically said 2.11 is incompatible and 2.12 is needed. CommandBox uses Undertow which is a subset of Wildfly and I’m not sure that Undertow has Xerces, however Lucee does include as an OSGI bundle Xerces 2.11 but I’m unclear on whether that bundle is at play.

I’m cloning your repo now to try the tests, but here’s a couple random observations

  • Your Git repo is way too big in size
  • Stop storing Jars in there and instead use dependencies in your box.json for CommandBox to pull in the jars at install time. This will make your repo super small
  • Javaloader also shouldn’t be in your repo. Use box.json and have CommandBox install it as a dependency. You can probably install the module version of javaloader and just directly call its classes the same way you do now.
5

And one more:

  • You’re using TestBox for your tests, but it’s not listed as a dependency in your box.json! Save it as a development dep so when I clone the repo, I can just do “box install”. I looked and saw there were no dependencies in the box.json so I skipped the install step, but then the tests errored.
6

I have cloned the repo and ran the tests on Lucee 5, ACF 2016, and ACF 2018 and I cannot reproduce the error you showed. I do get a couple errors in Adobe but they’re unrelated to Java classes and have to do with 1990 being an invalid date and attempting to reference a string as a struct.

7

Hi Brad

Interesting. It would sound then like the POI lib is using a jar that is part of the servlet jars or just the core servlet stuff. I'm not familiar with the POI libs specifically as it's been a few years since I used them directly. If the Xerces classes are specifically the ones that are picking up the wrong version then it would seem that adding that jar into the same class loader as your POI jars would make sense.

Yes that's what I would expect too. But as I say putting the newer
XercesImpl.jar in my JavaLoader lib path makes no difference.

I also noticed that one of those pages specifically said 2.11 is incompatible and 2.12 is needed. CommandBox uses Undertow which is a subset of Wildfly and I'm not sure that Undertow has Xerces, however Lucee does include as an OSGI bundle Xerces 2.11 but I'm unclear on whether that bundle is at play.

I don't think so because I can run the POI4.0 version with Lucee 5
running in Tomcat with no problem. But not with CommandBox (which as
you say is using WildFly).

I'm cloning your repo now to try the tests, but here's a couple random observations

Your Git repo is way too big in size
Stop storing Jars in there and instead use dependencies in your box.json for CommandBox to pull in the jars at install time. This will make your repo super small
Javaloader also shouldn't be in your repo. Use box.json and have CommandBox install it as a dependency. You can probably install the module version of javaloader and just directly call its classes the same way you do now.

optout.

But that would mean forcing folks to use CommandBox to set up the
library. The box files are there because someone asked for that as an
option and offered them as a contribution.

Cheers,
Julian.

8

Sorry if it made things harder, Brad, but as I say I didn't write the
box files. I'll see if I can get them improved for those who do want
to use CommandBox for setup.

9

That's excellent news, Brad. Thanks so much for taking a look at this.

Now I just need to work out why my CommandBox set up doesn't appear to
work the way yours does. I'm using the latest 4.x version.

Julian.

10

Can I just double check though that you cloned the *develop* branch,
which contains the new POI 4.0 version (and not the master)?

11

Ahh crap I was on the default branch of master. I changed to dev and still no error on Lucee 5 but Adobe CF 2016 did error. This SO post indicates another jar that might be missing.

https://stackoverflow.com/questions/13284565/java-lang-noclassdeffounderror-could-not-initialize-class-org-apache-poi-poixml

Check out that xmlbeans jar and see if you get any mileage out of it. I’d check it right now but I need to hit the hay before jetlag murders me tomorrow now that I’m in Munich for CFCamp.

Regarding the packaging of the project-- my take on that is pretty opinionated. I say it’s 2018 and no other language on the planet coddles people who are scared of a package manager. Want to Ruby? You’ll need to use Gems. Want to Java? You’ll need to use Maven. Want to PHP? You’ll need to use Composer. Want to Phython? You’ll need to use PiP. Want to Node? You’ll need to use npm. ColdFusion is the only community where we have yet to embrace modern ways of packaging and distributing packages. Even if you don’t want it to be a box module per se, it can at least be a proper package. That’s my take anyway :slight_smile:

12

OK, that's a shame but again I appreciate your input, Brad.

And one positive: you're right that it's working fine with Lucee 5 on
CommandBox, not just in Tomcat. Not sure why I saw that as failing.
Lucee 4.5 still fails as well as ACF2016, but at least that indicates
CommandBox/Wildfly per se isn't the breaking factor as I'd thought.

That means I could press ahead with a Lucee 5 only release in the hope
a solution would emerge at some point to allow ACF support.

The xmlBeans jar is already there in the JavaLoader lib path and in
fact I've updated it to the latest 3.0.2 version which supposedly
contains a patch for the WildFly XML parsing issue. See XMLBEANS-519
on the release notes:

http://svn.apache.org/viewvc/xmlbeans/trunk/CHANGES.txt?view=markup&pathrev=1844288

I was hoping this would solve the issue but sadly it doesn't seem to.

I understand your take on packaging, but don't agree. The advantages
are clear, but it also adds an extra layer of complexity which can be
confusing. You often see articles bemoaning the countless hoops you
have to jump through these days just to add a tiny JS library to your
web page. While most of the Github projects I've used do have NPM etc
installation as the recommended route, there's almost always a
straightforward zip download option. Options are good. Forcing people
to jump through unnecessary hoops isn't in my view.

Hope the jet lag isn't too severe, Brad, and that you have a great
time here in Europe!

Cheers
Julian.

13

Hi Brad

Just to follow up on this with some great news: a 4.0.1 patch of POI
is about to be released and the RC seems to make this problem go away
completely. All tests passing in Lucee 4/5 and ACF2016/2018 within
CommandBox.

issues were internal to POI.

Apologies for my misdiagnosis.

Julian.

14

Glad to hear the issue seems to have gone away. Hopefully it stays away :slight_smile: