Testbox with Gitlab-ci status


we are trying to run our tests with Gitlab-ci with:

      - box testbox run verbose=false directory=tests.specs.unit.services outputFile='testbox-junit.xml' reporter='junit' > /dev/null
      - testbox-junit.xml
    junit: testbox-junit.xml

The tests runs but Gitlab always says “Job success” and put the status as Green in the job:

Captura de pantalla de 2021-04-30 13-06-16

But there are errors in the tests:

I think that is becacuse the command "box testbox run … " always success although there are some test that fails.

How can be the status of the job failed instead?


Could it be because you are redirecting the output to /dev/null? I’m not sure how CommandBox will interpret that line.

Redirecting the output to dev/null shouldn’t make it pass, though I could be wrong. Why are you redirecting the output nowhere? That would come in very handy in debugging this issue.


it seems that the problem is when the reporter is junit:

Outputfile option:

Outputfile with the junit reporter:


Hi David!

We had a similar issue with unit test results in the pipeline as well.

Our problem comes from the testbox-junit.xml content which had an extra line on the top.

Could you check if this is your case?. If so you can run this command after your box testbox run command

sed -i '1d' testbox-junit.xml

I hope this work on your end

Ei, Hi Florindo!

Thanks for your response.

No, our file not have an extra line on the top…

I think it could be the exit code of the command:

  • without junit reporter, a failed “box textbox…” the exit code is 1

  • with junit reporter, the exit code is 0

  • for instance, with json reporter, the exit code is 1 as well.

So, could be the junit reporter that not set the correct exit code ?

We have a solution using the antjunit reporter instead of the junit reporter:

In the pipeline set the correct status “failed”:

Captura de pantalla de 2021-05-06 11-38-56