Ascendro Blog

Displaying 1-1 of 1 result.

Software testing approach by project size

Which testing method its worth using, manual or automated? Let's just say it is entirely dependent on the project profie and size.

We are constantly looking to improve the outcome of our Quality Assurance department and because we have experience with both manual and automated testing we have put together some general ideas regarding the testing approach depending on the project profile.

 

First some short definitions about these 2 types of testing and main pros and cons.

Automated testing:

Is a software testing technique using tools that provide recording and playback or/and programming using software to run the test cases from a test suite.

Manual testing:

Is a type of software testing in which testers play the role of the end user, and manually execute a test plan to ensure correct behavior of an application, by validating results against expected results for every test case in the given test plan.

Main Benefits and downsides we consider:

Manual testing

Benefits Downsides
Flexible Time consuming
Dynamic form of handing Not stimulating for tester
Less costly on short term Full testing regressions are difficult to make
Ran by  human tester  

Automated testing

Benefits Downsides
Fast to run Costly to implement
Ran remotely Maintenance
Minimize the error risk Debugging the test scripts
Ran by human tester  

We generally made the recommendations for a 6-10 people company considering the agile way and the man days approach for the estimates.

Please see the following interesting resources considering project estimates: Hofstadter’s Law and Projects using COCOMO

Now, lets see which testing type fits best for what project size.

Small size project (Websites, Blogs, Presentation/Sales Sites)

  • Under 3 months -> man days
  • Simple solution
  • Mainly Configuration and Setup
  • 1-2 developers
  • 1 tester
  • Approximate  under 5000 lines of code

For small size projects, manual testing has the advantage that is flexible, it has a dynamic form of handling, to put this way, if something you would like to change in your test case is quicker, it’s also less costly on short term, and if done with right attitude it’s more likely to find real, logic based bugs and UI issues.

The high amount of time needed to implement and maintain automated testing makes it not suitable for small size projects. Manual testing is the base of any project that requires testing, because we, as humans, are after all those who build the machines and have the capability of judging plus plenty of imagination.

Medium size project (Platforms and Online Communities)

  • Between  3 and 12 months
  • 2 to 4 developers
  • Module extensions and custom programming
  • Average complexity
  • Cache solutions
  • 1 tester
  • Approximate under 30000 lines of code

For medium size projects automated testing can be used for main features and main workflows and can cover up to 30 % of the test cases. The automated testing has to be done for non-changing initial requirements and the stable design of a platform/application.  Consider costs of the tools used and qualified personal are some of the cons for automated testing in this case.

Large size project (Complex web applications)

  • Above 12 months
  • Multiple end-user profiles
  • Mainly custom programming
  • Complex reports section
  • 4 or more developers
  • High performance required
  • Approximate  over 70000 lines of code

For larger project, manual testing is a huge investment of human resources and time consuming. Having few people working for a project, proportionally leaves room for more human errors. Also on the other hand if it’s only one person working on the project, his job will not be stimulating, repeating the same tests over again, which again, leaves room for errors.

Automated testing has the benefit of being a fast form of testing. It can be run on multiple browsers in the same time, few times a day. This definitely reduces the number of testers for completing the job and the test cycles and the clear outcome is a faster time on the market.

In case of unit testing, the problems are found early in the development cycle, the automated tests can be shared better and they can also run remotely.

Also adding new features will not be a problem because automation allows you to accumulate your test cases over the life of the application so that both existing and new features can always be tested efficiently. It is also important to mention that the failure risk is significantly minimized. (see graphic below).

graph

Conclusions:

Best practice as far as the testing in concerned is to find what suits your project by correctly estimating your project size and by applying the right testing strategy in order to fulfill your goals. The automated testing means progress but no matter what testing approach you use the most important thing is to provide high quality software and respect your commitments to the customer.

LEAVE A COMMENT

No results found.