Issue Using Commander to Test CommandBox CLI

Ok, this one might be a bit of a stretch.

I wanted to try to use Commander (GitHub - commander-cli/commander: Test your command line interfaces on windows, linux and osx and nodes viá ssh and docker) to test my CommandBox custom commands. It looks like a fairly robust and configurable tool for CLI testing.

However, in my initial testing, it has a hard time running the CommandBox binary on my machine. CommandBox works fine on its own. Commander works fine on its own. So the issue appears to have to do with how Commander is calling Commandbox.

When I set it up to run a basic test with the command box version, it results in the following output to Stderr:

usage: dirname path
Error: Could not find or load main class cliloader.LoaderCLIMain

Just looking for any leads/tips/insights on what might be causing this.

I’m using CommandBox 5.2.1+00295.

Thanks!

That’s odd. What OS are you running this on? That changes how it’s bootstrapped by the OS, but in either case, that class in the error is the “main” class in the jar and bundled at the top level of the executable. I’ve never seen that error before.

@mjclemente I was able to get Commander working. Though it is a little odd since it seems to run the test with no environment which means I had to provide a full path to the binary or it would say it was not found. I created this yaml file

tests:
  D:\commandboxInstall\box version:
    exit-code: 0
    stdout: 
      contains: 
              - CommandBox

And than ran it like so:

commander test commander.yaml
Starting test file commander.yaml...

✓ [local] D:\commandboxInstall\box version

Duration: 9.692s
Count: 1, Failed: 0, Skipped: 0

Seems to work ok. This is on Windows with the AMD 64 download.

Also, let me remind you that you can test your CommandBox modules using CommandBox Tasks as well. We have done this for the docbox commands and will be following this approach in all our commands.

I am working on a TestBox approach to this as well.

1 Like

@bdw429s Thanks so much for taking the time to look into this! Really appreciate it :grinning:

The absolute path trick didn’t work on my Mac, but I did managee to figure it out! There’s an option in the config, inherit-env, which allows you to inherit all environment variables from your active shell. When set to true, it works with CommandBox as expected!

I’m guessing that it probably needed to inherit the envs in order to find my JAVA_HOME - could be a consequence of how my shell is set up. Regardless, very happy it’s working.

@lmajano Thanks for the tip about using Tasks! I hadn’t thought about that approach, and it certainly has its benefits (speed, being written in CFML). What I like about Commander is that I can test the commands as they would be run from the CLI, and write real tests/assertions regarding the output, whether stdout, stderr, or file content. Very happy that I can use it with CommandBox now!

No problem. Remember that the task test approach is the a real/integration test. You are not mocking anything. So in reality, it’s as true as the commander approach.

@mjclemente Yes, the environment is needed to find java, however I’m confused since if I run box in a linux enviornment with no java installed, I get this error:

exec: line 87: java: not found

so I’m not sure why you got the error you received. Perhaps you could do a test for me. The shell script which is used to start CommandBox on Unix OSs is found here:

It’s pretty basic logic. It looks in this order

  • a JAVA_HOME env var
  • a JRE env var
  • whatever which java returns based on the PATH

You can basically toss that file onto your machine, chmod it so it’s executable and then edit the last line to be echo instead of exec and you can see the exact command that’s running.

For example, when I do that in a foundeo/minibox container, I get this output:

java -client cliloader.LoaderCLIMain

And since the executable jar file is concatenated with the shell script, the LoaderCLIMain class is right there in the file.

What’s actually more likely is the commander util is somehow stripping the $0 env var which is used to set the CLASSPATH env var inside the shell script. That would explain its inability to know where to load the class from. You can test this out pretty easily by just adding this to the bottom of the test shell script from above

echo "This is the class path: $CLASSPATH"

In my tests, it points to the name of the script itself (which would be the box binary normally). I have a feeling its empty when commander strips all the normal bash env vars out.

You’r right about the output - when I run the box.sh script with the inherit-env flag set to true, I get the following:

/Users/matthew/.jenv/versions/openjdk64-11.0.6/bin/java -client -Dcom.apple.mrj.application.apple.menu.about.name=CommandBox -Dcom.apple.mrj.application.growbox.intrudes=false -Dapple.laf.useScreenMenuBar=true -Xdock:name=CommandBox -Dfile.encoding=UTF-8 -Djava.awt.headless=true cliloader.LoaderCLIMain
This is the class path: ./box.sh

However, when I run it without inherit-env, it’s empty:

java -client -Dcom.apple.mrj.application.apple.menu.about.name=CommandBox -Dcom.apple.mrj.application.growbox.intrudes=false -Dapple.laf.useScreenMenuBar=true -Xdock:name=CommandBox -Dfile.encoding=UTF-8 -Djava.awt.headless=true cliloader.LoaderCLIMain
This is the class path:

It seems that when it runs without the inherit-env option, the shell context in which it runs has no environment variables whatsoever. I guess this is intended to give you more control over what is actually being executed… but honestly I’m not sure - certainly not helpful for my use case here. But glad I can override it to behave as I expected.

Thanks for helping test that. It might be worth putting some additional checks into that shell script just in case someone hits this again. I didn’t know it was possible to run a process in bash and NOT have those special env vars present, but without them, CommandBox clearly can’t bootstrap.