Software Testing Principles

What is Software Testing Principles

Software Testing Principles can be called as Software Testing Rules as well. We can not do Software Testing randomly. We must follow some rules, so that the Software Testing Process should be effective and We can test our application in an efficient way by putting less efforts. We can do maximum coverage with minimum efforts if we use Software Testing Principles.

Software testing principles refer to a set of guidelines and concepts that must be followed while performing software testing, so that the testing process is effective, efficient, and delivers reliable results

Different Software Testing Principles

There are 7 Different Software Testing Principles.

• Exhaustive(Complete) testing is not possible
• Testing shows presence of defects
• Absence of errors fallacy
• Early testing
• Defect clustering
• Pesticide paradox
• Testing is context dependent

Exhaustive (Complete) Testing is not possible

Exhaustive Testing or Complete Testing is not possible, that means if we start testing a particular feature of a Software for all the input values, then it will take a lot’s of time, but we always have a limited amount of time, when we do software testing, so we need to find the optimum way to do testing, so that our efforts and time should be reduced but we can do maximum coverage.

Example of Exhaustive (Complete) Testing is not possible

In Order to understand this principle, Let’s take an example, where we have to test a field on a Website

1. which accepts 4 digits at the most, and 
2. only positive numbers are allowed, and 
3. we have to test this functionality in 10 minutes, 

This means that the field can accept any number between 0 to 9999 but any number less than 0 and greater than 9999 is not allowed.

So if suppose, we will try to test this field by entering all the numbers between 0 to 9999, then it is not possible to test this functionality in 10 minutes by entering all the numbers.

If we try to enter all the numbers less than 0 in the field and all the numbers greater than 9999, then the testing will never end and Testing will never be completed.

Why are we saying that we should test it in 10 minutes, because it refers to a timeline and every time there will be a timeline given to Software Testers to complete their Testing, and that’s why we have assigned some timeline to this testing as well.

How to overcome this problem

So, We can clearly see with this example, that Exhaustive (Complete) Testing is not possible.

So, We need to complete this testing using some other alternative approach and those approaches are a part of Software Testing Technique, which we will study later.

Testing shows presence of defects

When we do testing, then it is for sure, we will find defect. It is impossible that an application is free of defect. Testing always talks about presence of defects not absence of defects.

Example of Testing shows presence of defects

Let’s understand this with the previous example where we were testing a field

1. which accepts 4 digits at the most, and 
2. only positive numbers are allowed.

So, this means, that we can enter numbers from 0 to 9999 and no other number is allowed, but if suppose, we type a number (-1) (negative number) or 10000 (number with more than 4 digits) then this filed should not allow this but if it allows this, then it is a defect.

And there is a high chance for this type of defect to occur. This type of Testing is called Negative Testing.

Sometime what happens is that the testers will perform Positive testing and will say that the Product is defect free. So that’s why we should be doing Negative Testing, because there is a high chance to get a defect during Negative Testing.

So, We have seen, when we do more and more testing, we tend to find more and more defects and that’s why this principle says that “Testing shows presence of defects”.

Absence of errors fallacy

fallacy means a false belief. So When we have a false belief that there is no defect in our application, and we say that the application is defect free and the application is launched into production environment so that end users should use but If the end user says that the application is not at all useful as it is not working as expected, which means the Application is not usable by the end user. This is called Absence of errors fallacy.

So, If there is a Software Application, which is tested properly and there is no defect in the Application but still the end user is not able to use it or it is not giving the required output. This is called Absence of error fallacy.

Example of Absence of errors fallacy

We can understand this with the previous example where the field is accepting only numbers between 0 to 9999 and if it is tested only for positive test cases but not for negative test cases, and we say that the application is working fine. then we will have a false believe that the application is working fine and there is no defect because we didn’t test it for negative test cases and end user might get a defect while doing negative testing. So, This is one example of Absence of Error Fallacy.

Now, Let’s see one more example.

Absence of Error Fallacy might also occur if the Requirement was something else but the developer developed the Application for some other requirement, and tester tested as per the instructions from developer.

So, Testing is not just about testing the test cases, finding defects and fixing those defects, Testing is all about Building the right product which can be used by the end user.

So, a Tester should know about functional requirement and business requirement very well, So even if the developed application is developed for wrong requirement, then the tester should be able to find it and should be able to get it fixed and make it usable for the end user.

Early Testing

Early testing is a software testing principle that emphasizes the importance of starting the testing process as early as possible in the software development life cycle.

Early testing involves testing the software at every stage of the development process, from requirements gathering to design, coding, and integration. This can include unit testing, integration testing, system testing, and acceptance testing, among others.

Example of Early Testing

Let’s take an example of a situation, where we have to test an application within 4 weeks of time, but because of Delayed Deployment from Developers, the Application was not available to test in the first week of time and Testers were left with only 3 weeks of testing.

In the 2nd Week, There were some major issues in the Application, and the developers couldn’t resolve the issue for the entire week and Testers were left with only 2 weeks of testing.

In 3rd week, Testers could start full fledge testing and they started logging defects for the issues they faced and Passed few test cases where the functionality worked fine. Developers worked on the defects fixed them and the testers could test them in the 4th week.

In 4th Week, 

• Testers tested the remaining functionality, Few of the test cases were passed and few were failed, so more defects were logged for those.
• Retested the defects which were fixed by the developers. Few defects were fixed and closed but few didn’t work fine and were reopened.

So, Now the Problem is, there are still some defects which need to be fixed but the timeline is over and this is a big problem.

In order to avoid this Situation, We must start testing early, which is the Principle, Early Testing.

Defect clustering

Defect clustering is a phenomenon in software testing where a small number of modules or components in a software application contain a high proportion of defects.

Defect clustering is also called Defect Grouping. In this, defects are grouped to their respective functionality. 

There is a very famous rule or principle, Pareto Principle. It says that in most of the cases, “roughly 80% of consequences come from 20% of causes“. This is also called 80/20 principle.

So when we apply Pareto Principle to Software Testing, then we can say that “mostly 80% defects come from 20% of the functionalities“.

Example of Defect clustering

Let’s understand this with an example of Internet Banking Software Application.

What are the functionality in Internet Banking.

• Launching Net Banking Application
• Login to Net Banking
• Payment Transfer NEFT
• Payment Transfer IMPS
• Payment Transfer UPI
• Check Bank Statements Online
• Logout from Net Banking

So, we can see, How many functionality are there in Net Banking. 

Now, Let’s assume that we found some defects while doing testing these functionalities. If we apply the concept of Defect Clustering, then we will understand that how many defects are related to which functionality. 

Let’s assume that the total no of defects are 10.

Let’s see the Percentage division of Defects.

The Data is not exactly correct but this is just to show you that most of the times, few functionality generates a large no of defects.

Pesticide paradox

Farmers use pesticide in their farms while farming, So that insects will eradicate. When this process is repeated over a long period of time, then these insects develop resistance to the pesticide and these pesticides become ineffective on the insects. This effect is called Pesticide Paradox.

The same applies to Software Testing as well. 

The pesticide paradox in software testing is a phenomenon that refers to the idea that the repeated execution of the same test cases over and over again can reduce the effectiveness of testing. 

The pesticide paradox occurs because, the software is tested repeatedly using the same test cases, it becomes less likely to find new defects. It looks like that the Software is working fine because the defects are not coming, but actually the software will have defects and we are not testing those.

Example of Pesticide paradox

Let’s understand this with an example of Net Banking Application. Let’s assume, we are testing below functionality in our Net Banking Application Testing.

• Launching Net Banking Application
• Login to Net Banking
• Payment Transfer NEFT
• Payment Transfer IMPS
• Payment Transfer UPI
• Check Bank Statements Online
• Logout from Net Banking

When we test these functionality for the first time, We find some defects in all the functionality almost.

Once Developer fixes these defects, the Testers test these functionality again, We find that the number of defects are reduced now.

then in the next iteration, when the remaining defects are fixed by developers, again the number of defects gets reduced and this cycle goes on.

At some point of time, We find that the number of defects become 0 if we test those functionalities using the same test cases. So that means our test cases are not able to find new defects. This effect in Software Testing is called Pesticide Paradox. This is one of the very important Software Testing Principles.

Iterations

So, We can see the number of defects are getting iteration by iteration, and thus it makes no sense to test the functionality with same test cases where we are not getting any defects.

How to overcome the problem of Pesticide paradox

Now, If we want to overcome the problem of Pesticide paradox, then we should write test cases for each functionality thoroughly and we should update the test cases in each iteration so that the test cases remain relevant and effective. This way, we will be able to find new defects and test our software properly and will be able to deliver a quality product to our customer.

Testing is context dependent

Testing is context dependent. Now the question comes what does it mean by context dependent. So, Context dependent is made up of 2 words.

Context + dependent. We have to understand the meaning of Context. The meaning of Context is

1. Type of the Software Application – Testing approach will change when the type of application changes. We have different types of Applications such as Web Application, Mobile Application, Desktop Application. Web Application can be accessed using Browser, Desktop Application need to be installed on the Desktop and Mobile Application must be installed on the Mobile.
2. Domain of the Application – The Functionality or Feature of an application depends on the domain of the Application. The Application we are testing is a Banking Application, an eCommerce Application, or a POS Application at Merchant. You need to have Domain Knowledge to test the application. Requirement is going to be different for each domain. Security is the primary aspect of a Banking Application. Performance is the primary aspect for an eCommerce Application. So this way, we see that each of the Application is different from each other because of it’s domain and functionality.
3. System dependency of an Application – Desktop Application should be compatible with Windows and Mac OS. Web Application should be compatible with any Browser such as Chrome, Mozilla, Safari. Mobile Application should be compatible with iOS and Android System

So, when we say that Testing is Context dependent means what type of application is being tested, What is the domain of the Software Application, and many other factors, So If the Software Application is different then the Testing Approach will be different as well and it will be depend on the nature of the Software Application.

Example of Testing is context dependent

eCommerce Website has its own needs, features, and functionality, where as Net Banking Application has its own needs, features, and functionality. So, As the need, features and functionality changes, then the Testing Approach also changes.

Security is going to be the most important feature for a Net Banking Application, as the bank is responsible to keep customer’s money safe and secure.

Performance is going to be the most most important feature for an eCommerce Application, as the eCommerce Website shouldn’t go down if many users are trying to do shopping during Sale and Offers Season.

Appearance and User experience is also an important feature for an eCommerce Website, So the Customer should be attracted towards eCommerce Website.

Importance or Advantages of Software Testing Principles

Software Testing Principles are very important so that we can do testing with full coverage with minimum efforts.

1. Exhaustive Testing is not possible tells us that we can’t test a feature with all the input values so we have to use some techniques to minimize our efforts but we need to make sure that we have tested the feature with full coverage.
2. Absence of Error Fallacy tells us that we should be testing the product for correct requirements so that the product should be useful for the end user.
3. Early Testing helps in identifying the defect as early as possible so that the development team gets enough time to fix them.
4. Defect Clustering helps in identifying that most of the defects come from which functionality or feature, so which feature needs more testing.
5. Pesticide Paradox tells us that we must keep changing our test cases, We should remove the test cases where defects are not coming and keep adding new test cases for new features and functionalities where we think, more defects might come.

Summary

We have covered below topics in this session.

1. Software Testing Principles
2. What is Software Testing Principles
3. Different Software Testing Principles
4. Exhaustive(Complete) testing is not possible
5. Example of Exhaustive(Complete) testing is not possible
6. Testing shows presence of defects
7. Example of Testing shows presence of defects
8. Absence of errors fallacy
9. Example of Absence of errors fallacy
10. Early testing
11. Example of Early testing
12. Defect clustering
13. Example of Defect clustering
14. Pesticide paradox
15. Example of Pesticide paradox
16. Testing is context dependent
17. Example of Testing is context dependent

Question

1. Give an example of each Software Testing Principles.
2. What is the importance or advantages of Software Testing Principles.