![\"\"](\"https:\/\/blog.planview.com\/wp-content\/uploads\/2020\/02\/LK_blogPhoto_QApuzzle-1024x614.jpg\")
<\/p>\n<\/p>\n
If software QA began with a specific number of bugs, it might be easier to find all the issues. For example, many puzzle books show you a drawing and ask you to find an exact number of hidden objects. Or, they show you a pair of drawings and ask you to spot a certain number of differences between them.<\/p>\n
In software development, the target number of differences \u2014 or bugs \u2014 isn\u2019t always so specific. How much easier would software QA be if someone could whisper in our ear how many bugs there are to find? In reality, we never know. We only know a minimum: if we\u2019ve found 37 bugs, we know there are at least 37 bugs. Maybe there\u2019s one more to find, or maybe there are hundreds \u2014 we can\u2019t be sure.<\/p>\n
It would be helpful to have a rough idea of how many bugs remain, so we could best utilize our testing resources. If we always assume there are hundreds of bugs left to find, we may be over-testing one product at the expense of others. If we have reason to suspect we\u2019ve found all the bugs, or nearly all of them, we can redistribute our resources to make the best use of their time and efforts.<\/p>\n
Nothing can tell you how many bugs are left during software QA, but a little math can give you an estimate. Suppose you have a tester who has found some number of bugs. If you knew not only how efficient she was at finding bugs, but also the probability of her finding each bug during a period of time, you could estimate how many there are to find. But you can\u2019t know what proportion of bugs she\u2019s found without knowing how many there are to find.<\/p>\n
Now suppose you have another tester, and he\u2019s also found some number of bugs. You don\u2019t know what proportion of bugs he\u2019s found, either. But by combining the bug counts from the two testers, you can estimate how many bugs there are.<\/p>\n
This may seem too good to be true, but it\u2019s not \u2014 the key is to consider how many bugs both testers found. Suppose the first tester found 20 bugs and the second found 30, but there was only one bug that both found. You might suspect that there are a lot of bugs to find, since both could find so many with hardly any overlap. On the other hand, if 18 of the bugs on the first list are on the second list, you might feel like your testers have probably found most of them.<\/p>\n
You can quantify an\u00a0estimate with a tool known as the \u201cLincoln Index.\u201d If the first tester found A bugs, the second found B, and C bugs were in common between them, the estimated total number of bugs would be AB\/C. In the first example above, A = 20, B = 30, and C = 1. In this case there would be an estimated 600 total bugs to find. But in the second example above, A = 20, B = 30, and C = 18. In this case there would be an estimated 33 1\/3 bugs.<\/p>\n
How do you find 1\/3 of a bug? You might find 1\/3 of a bug in your soup, but you won\u2019t find 1\/3 of a software error. The Lincoln Index is a simple mathematical model, so it can\u2019t tell you exactly \u00a0how many bugs there are. It makes a couple simplifying assumptions, namely that testers find bugs at random and independently. That won\u2019t be exactly true in practice, which means we should be skeptical of the number it generates. Still, the Lincoln Index gives us an estimate to run with \u2014 much better than saying, \u201cI have no idea how many bugs there are.\u201d<\/p>\n
Here\u2019s the math behind the Lincoln Index: Suppose there are N bugs total, and the first tester has a probability p of finding each bug. Then she would find around A = Np bugs. If the second tester has a probability q of finding each bug, he would find around B = Nq. And if they find bugs independently, they\u2019d find around C = Npq bugs in common. Then AB\/C should be around (Np)(Nq)\/(Npq) = N, the number of bugs. The probabilities of each tester finding a bug cancel each other out.<\/p>\n
While the software QA process will never be as simple as a \u201cspot the differences\u201d puzzle, for certain types of software development, there are simple tools to help estimate that elusive number of total bugs. Using the Lincoln Index, we can determine the level of overlap between our testing resources, which usually speaks to the proportion of bugs that are left to uncover. With this rough estimate, we can make data-driven decisions about how to best allocate our resources.<\/p>\n","protected":false},"excerpt":{"rendered":"
If software QA began with a specific number of bugs, it might be easier to find all the issues. For example, many puzzle books show you a drawing and ask you to find an exact number of hidden objects. Or, they show you a pair of drawings and ask you to spot a certain number…<\/p>\n","protected":false},"author":157,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_editorskit_title_hidden":false,"_editorskit_reading_time":0,"_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}","footnotes":""},"categories":[38],"tags":[511],"class_list":["post-12217","post","type-post","status-publish","format-standard","hentry","category-work-collaboration","tag-leankit"],"acf":[],"yoast_head":"\n