Hi,
we are trying to run our tests with Gitlab-ci with:
script:
- box testbox run verbose=false directory=tests.specs.unit.services outputFile='testbox-junit.xml' reporter='junit' > /dev/null
artifacts:
paths:
- testbox-junit.xml
reports:
junit: testbox-junit.xml
The tests runs but Gitlab always says “Job success” and put the status as Green in the job:
But there are errors in the tests:
I think that is becacuse the command "box testbox run … " always success although there are some test that fails.
How can be the status of the job failed instead?
Thanks!
elpete
April 30, 2021, 5:18pm
2
Could it be because you are redirecting the output to /dev/null? I’m not sure how CommandBox will interpret that line.
Redirecting the output to dev/null shouldn’t make it pass, though I could be wrong. Why are you redirecting the output nowhere? That would come in very handy in debugging this issue.
Well,
it seems that the problem is when the reporter is junit:
Outputfile option:
Outputfile with the junit reporter:
[…]
Hi David!
We had a similar issue with unit test results in the pipeline as well.
Our problem comes from the testbox-junit.xml content which had an extra line on the top.
Could you check if this is your case?. If so you can run this command after your box testbox run command
sed -i '1d' testbox-junit.xml
I hope this work on your end
Ei, Hi Florindo!
Thanks for your response.
No, our file not have an extra line on the top…
I think it could be the exit code of the command:
without junit reporter, a failed “box textbox…” the exit code is 1
with junit reporter, the exit code is 0
for instance, with json reporter, the exit code is 1 as well.
So, could be the junit reporter that not set the correct exit code ?
We have a solution using the antjunit reporter instead of the junit reporter:
In the pipeline set the correct status “failed”:
