Hello,
We are trying to run our TestBox unit and integration tests using the HTMLRunner through ANT and are hitting a roadblock. We have so many tests that the request to run the entire directory of tests will never finish and always times out. Did you guys anticipate this in your design of TestBox and if so, is there are way around it? I apologize but I am very ignorant on this piece of the framework. I dug into the code a bit and noticed the virtual application is created and torn down for each BaseTestCase. Is this necessary? It seems like only having 1 framework creation when the runner executes a directory or group of bundles would be more efficient.
Thanks in advance,
Micah
There are several choices.
Partition your tests in sections
Use the async Annotation within Describe block so they run in parallel
Increase the timeouts.
Luis Majano
CEO
Ortus Solutions, Corp
www.ortussolutions.com
P/F: 1-888-557-8057
Direct: (909) 248-3408
ColdBox Platform: http://www.coldbox.org
ContentBox Platform: http://www.gocontentbox.org
Linked In: http://www.linkedin.com/pub/3/731/483
Social: twitter.com/ortussolutions | twitter.com/coldbox | twitter.com/lmajano | twitter.com/gocontentbox
Thanks Luis. I’ll give those options a try.
-Micah
Luis,
What did you think about my suggestion to use a single virtual application for all base test cases in a single runner execution? Would that be feasible? I tinkered with this a bit but ran into problems where Wirebox could not inject certain properties of certain objects. I stopped there and decided to post here on the forums and get some advice.
Thanks!
Micah
Hi Micah,
DO you mean for ColdBox integration?
Luis Majano
CEO
Ortus Solutions, Corp
www.ortussolutions.com
P/F: 1-888-557-8057
Direct: (909) 248-3408
ColdBox Platform: http://www.coldbox.org
ContentBox Platform: http://www.gocontentbox.org
Linked In: http://www.linkedin.com/pub/3/731/483
Social: twitter.com/ortussolutions | twitter.com/coldbox | twitter.com/lmajano | twitter.com/gocontentbox
Yes. I dug into the code a bit and noticed the virtual Coldbox application is created and torn down for each bundle that extends BaseTestCase. This takes about 1 second for each of my test bundles so you can imagine that running a directory of 50 bundles would not happen very fast. Could all bundles share 1 virtual Coldbox application?
Hi Micah,
Yes that indeed is very interesting proposal.
Let’s do two things.
- Create a ticket for creating a new annotation to basically keep the application running in the background. The only way it will disappear is that you manually remove it or it times out in the Application.cfc of the test harness: [COLDBOX-449] Create a new testing annotation to control if application should be teardown in integration tests - Welcome
- You can now do this by overriing the afterTests() function so it does not unload it. Basically an empty function for now:
function afterTests(){}
Then just add an application timeout to the test harness to clean itself out every 15 minutes or so.
Luis Majano
CEO
Ortus Solutions, Corp
www.ortussolutions.com
P/F: 1-888-557-8057
Direct: (909) 248-3408
ColdBox Platform: http://www.coldbox.org
ContentBox Platform: http://www.gocontentbox.org
Linked In: http://www.linkedin.com/pub/3/731/483
Social: twitter.com/ortussolutions | twitter.com/coldbox | twitter.com/lmajano | twitter.com/gocontentbox
Thanks Luis. I will do those 2 things and let you know how the change affects testing performance.
Luis,
I have had mixed results in getting my tests to execute properly using a persistent virtual Coldbox application that is NOT rebuilt for each test case. In some tests it works just fine but in others I get errors where the Injector cannot find objects which definitely exist. There may be a little more to the solution than we originally thought.
Thanks,
Micah