Hi, brand new to TestBox, just started looking at it, so please forgive the possibly-newb questions.
I’d like to run tests in my own specified order, ideally at each level, suite on down, but most importantly at the individual test method level.
I have a solution working here, but it involves modifying TestBox.cfc, makes me pretty sure I’m not on the right track.
I see there is support for custom runners, but it wasn’t clear to me how to override test order, which seems to be just structKeyArray of the tested component’s methods.
Is there built-in support for controlling test order?
Is there a reason why test order matters? Usually I write tests to be completely stand alone so any test can be ran by itself. If I need shared fixtures to setup and tear down data, I seperate those into a function and run then in beforeAll() afterAll() for beforeEach() and afterEach().
I’m not sure what the answer is to your question but I think it’s no. I know I’ve had tests fail before where I had accidentally assumed certain variables would exist, but later on the order of the tests would change just based on how they were stored in memory. If you really really want to do it this way (even though it’s not really a recommended approach to testing) you could put in a ticket and see if there’s an easy way Luis could support it. Building your own runner would also probably allow you to do this but it sounds like a lot of work.
But both MX Unit and Visual Studio Unit have ways to order tests. The need, or lack thereof, is dependent upon the individual needs of the user. If they need a test order, they need a test order.
I honestly would like to see the results in alphabetical order Just so you can find what test you want to work on easier.
As far as the run order I don’t think it should matter and tests should be stand alone, but the results should be sortable in some way.
I understand, and I’m aware that completely standalone tests are common practice.
However, this is a very data-dependent app, with many processes that the existence of prior data. For example, create an account, then a user for that account, then Thing A, Dependent Thing A.1, Dependent Thing A.1.1, etc… In prior lifetimes (not TestBox, testing a different app), we did a full db teardown and restore before every test, and it was a significant PITA. It made tests run much slower, and there were reliability issues as well. Just testing account creation, then Thing A, Dependent Thing A.1, etc., in order, effectively accomplishes a similar thing, but much more efficiently. It also resembles real life, where you’d do these operations in a specified order.
To accomplish that at the individual tests level, we need to override the method that gets the tests to run for each component. It looked to me like that method was part of TestBox.cfc, not the runner, so a custom runner wouldn’t help.
Will testBox have ordered tests (or has it been added and I missed it)? Remembering that every unit test framework I have looked at has test ordering.
I am in the same situation as Dave Merrill, we need to create first, then view, update, etc. The ‘View’ feature should not have to create data.
