Hmm no, just integration testing.
My understanding was that you write unit tests, which test individual methods, mocking everything external.
You then have integration tests that test 2 or more units playing together, without mocking.
I'm very new to this, so perhaps I'm getting myself confused 
Robert
Kk, never done it that way before and I am looking at the definition of integration testing now.
I will admit that I have no idea about this, but I would assume based on what I am reading, is that the integrated test is made up of smaller unit tests. Which to me sounds like Test Suites in mxUnit. I might be wrong there, so if anyone else knows please chime in.
I have only ever worried about the unit and automated user testing.
Regards,
Andrew Scott
Can I ask when are you planning on running your integration tests?
Regards,
Andrew Scott
Yeah I don't quite understand just yet either.
I know what you mean about the test suites, certainly sounds like that from the Wikipedia entry, but I'm not convinced.
This description seems much better: http://www.testinggeek.com/index.php/testing-types/life-cycle/54-integration-testing
The two core sentences for me are:
"Objective of Integration testing is to make sure that the interaction of two or more components "
and
"each module is tested in increasing detail, replacing more and more levels of detail with actual code rather than stubs."
Thanks,
Robert
Yeah that kind of makes sense.
But I look at it this way if you have a login event you have tests for the event, and possible the service, and possibly the model and Domain Model (Depending on what framework you are using for your DTO’s) with that logic you could be assumed to be already doing integration testing. This is what I am reading.
So I am learning here too!
Regards,
Andrew Scott
Sure thing,
I think the point is, that unit tests are only _so_ effective, because it's being tested against your mocked assumptions to the behaviour of dependant components. If your mocks are wrong, then when put into production with the real thing, it'll break
Integration testing is about replacing the mocks with real objects, to ensure that your mocks/assumptions were correct, and that they play together in the real world.
I think in my login example, it would be fair for me to still mock the data interaction layer. I am then successfully ensuring that the handler talks to the security service, the security service talks to the member service. The only [x] factor then left is how the member service talks to the database.
If I did this, I would know that firstly, all the component parts work on their own (units), but also that they integrate with one another.
Robert
I use unit tests to test additional functionality that i have added to my model objects.
So if if i add a calculateAge() method on my user object, I will write a unit test to ensure that the method works as I would like. (I would actually write the test first before the method, as I use test driven development). With my unit tests there is no interaction with the database, so no objects are fetched or saved to the database.
Now my integration tests check the flow between objects, mirroring the actions a user might take on the site. So I may write an integration test which saves a user to the database, links that user to an event, an ensures that everything plays well together. Here there is interaction with the database, and the tests are generally longer and more complex than my unit tests.
Lastly I have my functional tests, which test the front end. Here I use sellenium-rc, which will run through and check on the browser. So for example, I may have a test that ensures that when someone tries to login with incorrect data, the correct error message appears on the screen.
Hi John,
Thanks for your input! That was my understanding too!
I’d be interested to get your thoughts on my particular integration test use case, where the data in the database will influence the behaviour. Do you just populate your database with test/sample data and hardcode values and id’s in your tests? Or do you mock that?
Robert
Im not sure I would do this in my integration tests, because you are testing event logic, I would have this in my functional tests, and use sellenium.
What i do is set up each scenaro for each test. So for scenario 1) Attempting with valid credentials takes me to the welcome page. I would have a test like this
test_valid_credentials_takes_me_to_the_welcome_page(){
// Create a new user in database
newUser = userservice.get();
newUser.setUserName(‘john’);
newUser.setPassword(‘1234’);
newUser.save();
// Perform test
Go to login page - enter correct details - check you get redirected.
}
Then in my teardown method i would revert the database so the newUser gets removed and the database is clean for the next test. I would probably move the create new user bit into a new method.
Cheers
}
See I don’t.
I always adhere to 100% code coverage, so if an event is called doLogin() then I write a test for doLogin() and check that a successful and non-successful are catered for in the one test, if the login event calls a service then the service us mocked.
My next test would be to write a test for the service, and cater for all inputs and outputs that are to be expected. The call to the Database is mocked.
My next test would be to unit test the database, this uses real database calls and rolls back the data when finished.
This gives me 100% code coverage in my development in this example.
Regards,
Andrew Scott
Yeah I just populate the database with test/sample data and hardcode the tests.
I have looked into mocking, but I havent used it yet, but im not sure you would use mocking in this instance. I think mocking is for mocking dependencies, so if the object you are testing depended on the twitter api, you could mock the twitter api dependancy, so you wouldnt have to call it directly in your tests all the time.
In ruby on rails, you have fixtures, which can populate your database for you prior to unit tests, am looking to find a way to do the same in coldfusion/coldbox etc.
http://clarkware.com/blog/2005/10/24/faster-testing-with-rails-1-0#Rails10FastTesting
Thanks John,
I think if you have the benefits of using ORM there are parameters for a sql script which could be used to populate the database, I don’t think it’s anywhere near as flexible as fixtures but might be a start point.
I had a discussion with John Which about it. I believe he has a database backup with all the data populated, he imports the backup at the start of the tests so gets a clean dataset every time.
Robert
The Issue with the sqlscript is that you have to do an ormreload before each unit test, which slows things down massively. This is why I use transactions instead.
http://forevr.tumblr.com/post/1726359751/speedy-unit-tests-using-transactions#disqus_thread
I see John, that’s a really nice way of handling things!
I’ll have a play with this later and see how I get on!
Thanks.
Robert
But do it the way I suggest Robert, the database level is where you don’t mock. But you rollback the data that you are testing with, for everything prior to the databse calls you mock the database calls. So if your service layer is loading the data or saving the data then mock it.
I think I have said that a few times now.
But when are you going to be doing you integration tests? From what I read these are best at the CI level.
Regards,
Andrew Scott
Ok, i see. Thats an nice way of doing it…I guess its a seperation of concerns when testing.
So lets say in your unit tests for an artist, you where testing that an non-valid artist returned the correct error messages.
function test_should_not_be_valid_without_name(){
artist = artistService.getArtist();
artist.setName(’’);
artist.setStyle(‘dubstep’);
result = validationService.validate(artist);
assertEquals(‘noName’,result);
}
In this case, you wouldnt really need this test, because you would already have a unit test for the validation service? Which has tested all possible scenarios?
Yeah Andrew, That's what John is saying too, he uses transactions to rollback the changes. I'm sorry if it felt like I wasn't listening, there is just so much to take in I have taken on board everything you've said.
Yeah, that's exactly my plans for integration, it won't be until I have full coverage of successfully unit tests. There is no point testing how things integrate when they don't yet work on their own.
Robert
Yes but I don’t write a test for every single condition, I write a test to cover every single condition and that is a big difference.
For example
Unit test - > event doLogin()
Function doLoginTest() {
//First let’s check that the correct user can sign in
Execute(‘doLogin’);
assertTrue()
//Now let’s check that a user login fails
Execute(‘doLogin’);
assertFalse()
}
This is missing the information to setup what is going to be passed for the needed condition to fail or pass, it also doesn’t show that the doLogin() dependencies are also mocked.
But as the doLogin() only has 3 exit points (in my case anyway) that is success, failure and an exception is could be thrown. But I check all my conditions in one place like this. I find this much better and easier as I described in another post, becuae if these are two different test methods, then I am forced to run both to make sure that my test actually works. Whereas this way I am running one test, and not two during my SDLC.
I love productivity and I hate creating extra work for myself, and as far as I am concerned a separate test method for each test condition is a waste of productivity.
Regards,
Andrew Scott
Then if I read right, then this would not be the place to mock anything. So your thoughts are right there, at least that is my interpretation.
Regards,
Andrew Scott
Not sure I agree with grouping all conditions into one test.
I think tests are clearer if they are testing for one thing, much like functions are clearer if they are doing one thing. I also think having one assertion per test is the ideal.
If a test that has grouped conditions fails it may be a number of things that have caused this, whereas if a test called test_login_with_invalid_date fails, you know exactly whats going on.