Software testing approach by project sizeby Ascendro Aug 2, 2013
Which testing method its worth using, manual or automated? Let's just say it is entirely dependent on the project profie and size.
We are constantly looking to improve the outcome of our Quality Assurance department and because we have experience with both manual and automated testing we have put together some general ideas regarding the testing approach depending on the project profile.
First some short definitions about these 2 types of testing and main pros and cons.
Is a software testing technique using tools that provide recording and playback or/and programming using software to run the test cases from a test suite.
Is a type of software testing in which testers play the role of the end user, and manually execute a test plan to ensure correct behavior of an application, by validating results against expected results for every test case in the given test plan.
Main Benefits and downsides we consider:
|Dynamic form of handing||Not stimulating for tester|
|Less costly on short term||Full testing regressions are difficult to make|
|Ran by human tester|
|Fast to run||Costly to implement|
|Minimize the error risk||Debugging the test scripts|
|Ran by human tester|
We generally made the recommendations for a 6-10 people company considering the agile way and the man days approach for the estimates.
Now, lets see which testing type fits best for what project size.
Small size project (Websites, Blogs, Presentation/Sales Sites)
- Under 3 months -> man days
- Simple solution
- Mainly Configuration and Setup
- 1-2 developers
- 1 tester
- Approximate under 5000 lines of code
For small size projects, manual testing has the advantage that is flexible, it has a dynamic form of handling, to put this way, if something you would like to change in your test case is quicker, it’s also less costly on short term, and if done with right attitude it’s more likely to find real, logic based bugs and UI issues.
The high amount of time needed to implement and maintain automated testing makes it not suitable for small size projects. Manual testing is the base of any project that requires testing, because we, as humans, are after all those who build the machines and have the capability of judging plus plenty of imagination.
Medium size project (Platforms and Online Communities)
- Between 3 and 12 months
- 2 to 4 developers
- Module extensions and custom programming
- Average complexity
- Cache solutions
- 1 tester
- Approximate under 30000 lines of code
For medium size projects automated testing can be used for main features and main workflows and can cover up to 30 % of the test cases. The automated testing has to be done for non-changing initial requirements and the stable design of a platform/application. Consider costs of the tools used and qualified personal are some of the cons for automated testing in this case.
Large size project (Complex web applications)
- Above 12 months
- Multiple end-user profiles
- Mainly custom programming
- Complex reports section
- 4 or more developers
- High performance required
- Approximate over 70000 lines of code
For larger project, manual testing is a huge investment of human resources and time consuming. Having few people working for a project, proportionally leaves room for more human errors. Also on the other hand if it’s only one person working on the project, his job will not be stimulating, repeating the same tests over again, which again, leaves room for errors.
Automated testing has the benefit of being a fast form of testing. It can be run on multiple browsers in the same time, few times a day. This definitely reduces the number of testers for completing the job and the test cycles and the clear outcome is a faster time on the market.
In case of unit testing, the problems are found early in the development cycle, the automated tests can be shared better and they can also run remotely.
Also adding new features will not be a problem because automation allows you to accumulate your test cases over the life of the application so that both existing and new features can always be tested efficiently. It is also important to mention that the failure risk is significantly minimized. (see graphic below).
Best practice as far as the testing in concerned is to find what suits your project by correctly estimating your project size and by applying the right testing strategy in order to fulfill your goals. The automated testing means progress but no matter what testing approach you use the most important thing is to provide high quality software and respect your commitments to the customer.