Re:No Serious Comittment to Testing/QA by Major OS
Posted by: Anonymous Coward
on March 15, 2007 02:16 AM
I think you are missing a key point I made. I was not discussing the overall bugginess of commerical code vs OSS. The example issue I was making against OSS is there are serious bugs that have been about for years in most of the major cornerstone OSS programs >80% of those that use OSS use.
By the very nature of those serious bugs, it can be well argued 'the most "successful" proprietary product' is in fact less buggy and especially considering the excellent examples of at least one major very "successful" OSS project. It is by no means the only one OSS project. A very "successful" OSS project took over 7 years to attempt to fix issues that were identified at least 3-4 years ago, actually turned far worse than better for almost a year with releases over that time when the sucessful major OSS project stated they were aware of and working on same said issues. Then once the major issue clearly became much worse the same major very "successful" OSS project managed to deal with that dip to only get back to point of before the far deeper dip in the bugs. I suspect this was because of its high visiability to the aveage user, but still it took a few months to have the issue address that in esence was like a DoS attack to a user's system for a common staple function most users use. This clearly demonstrates how code was released that never identified the issue in the "casual" testing/beta cycle of hoping to identify issues. To date the problem still exists. The fixes listed in the release notes keep repeating over several releases of the security fixes made for the same issue and yet the bug continues, not to mention the repeating bug statement exactly the same over those several releases. One has to wonder why the same bug has been fixed over several releases and still is not fixed.
To put in perspective how serious this very "succesfull" major OSS software bug is I will give you two examples. At the big dip down the "successful" application would increase memory use from about 50MB to over 700MB just doing one simple common routine action 10 times or less. This happened about 2 yeas ago. A simple function that could in about 30 minutes, due to he bug causing the action to take longer each time as well as burn up more memory. Not only would this very "successful" OSS program grow from 50 MB to 700+ MB in progressively larger increments, it would would also loop such that one keystroke would take about 40 minutes or more to register in the application once the application memory approached available RAM. The very "sucessful" OSS program would bring the entire system to its knees and send the paging subsystem into thrashing simply because the application was looping through its memory leak big time. This then would cause any mouse or keyboard action for the entire system to take 40 minutes as well to effect. What is even worse is this same very "successful" OSS program would cause the exact same problem when run in a "proprietary" OS as well. This clearly proved with no extra effort required that this very "successful" OSS program was the sole cause of the bug.
The second example for this very same very "successful" OSS project is the ongoing memory leaking that climbs to well over 700 MB. If one closes the same OSS program and start it again exactlay in the "state" it was it will use about 50-125 MB depending on what the state was when the OSS program climbed to over 700 MB. That clearly shows a serious memory leak. If one closes some of the states open the memory is no released further proving the serious nature of this OSS bug. As a consequence this also stresses any OS this very "successful" OSS program does run on or brings the OS to its knees. It is very easy to demonstrated that this ongoing bug of this very "successful" OSS program is a risk for DoS for a user using the very "successful" OSS project, let alone hackers that once they find the trigers for this can have a field day. This is in part why I will not name the OSS project/program.
I can assure you almost no proprietary product would ever be released with such a serious and grossly major issue. Not even the most "successful" proprietary product has such issues. I have only seen one instance of common used proprietary product that had a memory issue. It was a poduct, not an OS, and it would suddendly spike memory in less than a second by at least 300 MB and not release that memory until the application was closed, if then. Needless to say I had the program removed from the systems. It was a progam many consumers purchase and use to protect their systems. Another program from different vendor I discovered was creating files in order of 500 - 800MB silently and never deleting them at all. This could lead to system crashes and file system problems with this issue going unchecked. Again the pogram was removed as soon as it was determined there was no user setting or configuration that would control this behaviour. The point here is OSS has same issues, just more of such issues that have gone unaddressed for several years in many cases. I also know of OSS software that still to this day, after reporting the issues over 7 years ago, that can damage a user's hardware. The very hardware the OSS community preaches can be used for OSS.
If you take a look at the OSS and the number of security fixes OSS has, it has as many as those proprietary class of pograms collectively. Yes, OSS is more responsive and open to the secuity and bug fixes made to OSS than many proprietary programs and products. It has been incorrectly stated that OSS is more secure and bug free due to the open source nature of the OSS source code. The fact is this is incorrect and I believe even RS has stated a couple years back that just because the souce is available it is does not result in higher quality of code. The article and link I do not have handy.
As much as I am a supporter and user of OSS, I can tell you that OSS, as much as it has lots to offer, has some very serious downfalls in terms of overall software quality and has a geneal distain for testing software. I personally have lost many weeks due to the ongoing bugs nobody will either admit to or take the time and effort to fix. I can tell you to actually test software takes more time than to develop software. I can tell you from first had experience that developers are the same in both the proprietary and OSS world. The difference is the OSS model generally has nobody that actually can ensure OSS developers will concur with a bug and spend their energy looking at the bug if it requires any long time and effort to invesigate. I can say this with confidance as we all know that this is the case with proprietary software and and we have the very public and open OSS model that in fact proves my point for OSS. The difference is in OSS it is not cost/benefit that enters into the decision, but if a OSS developer wants to, has the skills, is the developer familar with the code in question making the fix, and/or has the time to spend on the issues/bugs.
In closing OSS has a great opportunity to shine and show that they can write better code and programs than proprietary companies. The question is will the OSS community actually dig their heels in and commit to software quality or just bury the major problems that only formal testing can uncover or be harder to uncover as the problems are buried deeper. Leaving testing to chance with a few average users knowing there is a problem and not knowing how to duplicate and quantify hard to duplicate or very time consuming issues and bugs is not how one aspires to creating stable reliable code. As a qualified QA peson I find some issues a challenge to duplicate or quantify, but I have created many scripts, a few programs and fact gathering tools to help when needed. Still reporting the facts falls on mostly deaf ears of the OSS developers or their soap opera behaviour that I have no interest to persue. The average user would not have the support of the conclusive facts and would just be written off as not knowing what they are talking about or unable to provide the requested information, assuing that information would be taken seriously.
I have purposely avoided the whole useability issue to date which goes with software quality. The reason is simple, if the bugs cannot be addressed with their facts and objective nature, getting into the subjective side of the discussion of software quality will not be productive.
Learn about Linux
Download Linux
Get Linux help
Re:No Serious Comittment to Testing/QA by Major OS
Posted by: Anonymous Coward on March 15, 2007 02:16 AMBy the very nature of those serious bugs, it can be well argued 'the most "successful" proprietary product' is in fact less buggy and especially considering the excellent examples of at least one major very "successful" OSS project. It is by no means the only one OSS project. A very "successful" OSS project took over 7 years to attempt to fix issues that were identified at least 3-4 years ago, actually turned far worse than better for almost a year with releases over that time when the sucessful major OSS project stated they were aware of and working on same said issues. Then once the major issue clearly became much worse the same major very "successful" OSS project managed to deal with that dip to only get back to point of before the far deeper dip in the bugs. I suspect this was because of its high visiability to the aveage user, but still it took a few months to have the issue address that in esence was like a DoS attack to a user's system for a common staple function most users use. This clearly demonstrates how code was released that never identified the issue in the "casual" testing/beta cycle of hoping to identify issues. To date the problem still exists. The fixes listed in the release notes keep repeating over several releases of the security fixes made for the same issue and yet the bug continues, not to mention the repeating bug statement exactly the same over those several releases. One has to wonder why the same bug has been fixed over several releases and still is not fixed.
To put in perspective how serious this very "succesfull" major OSS software bug is I will give you two examples. At the big dip down the "successful" application would increase memory use from about 50MB to over 700MB just doing one simple common routine action 10 times or less. This happened about 2 yeas ago. A simple function that could in about 30 minutes, due to he bug causing the action to take longer each time as well as burn up more memory. Not only would this very "successful" OSS program grow from 50 MB to 700+ MB in progressively larger increments, it would would also loop such that one keystroke would take about 40 minutes or more to register in the application once the application memory approached available RAM. The very "sucessful" OSS program would bring the entire system to its knees and send the paging subsystem into thrashing simply because the application was looping through its memory leak big time. This then would cause any mouse or keyboard action for the entire system to take 40 minutes as well to effect. What is even worse is this same very "successful" OSS program would cause the exact same problem when run in a "proprietary" OS as well. This clearly proved with no extra effort required that this very "successful" OSS program was the sole cause of the bug.
The second example for this very same very "successful" OSS project is the ongoing memory leaking that climbs to well over 700 MB. If one closes the same OSS program and start it again exactlay in the "state" it was it will use about 50-125 MB depending on what the state was when the OSS program climbed to over 700 MB. That clearly shows a serious memory leak. If one closes some of the states open the memory is no released further proving the serious nature of this OSS bug. As a consequence this also stresses any OS this very "successful" OSS program does run on or brings the OS to its knees. It is very easy to demonstrated that this ongoing bug of this very "successful" OSS program is a risk for DoS for a user using the very "successful" OSS project, let alone hackers that once they find the trigers for this can have a field day. This is in part why I will not name the OSS project/program.
I can assure you almost no proprietary product would ever be released with such a serious and grossly major issue. Not even the most "successful" proprietary product has such issues. I have only seen one instance of common used proprietary product that had a memory issue. It was a poduct, not an OS, and it would suddendly spike memory in less than a second by at least 300 MB and not release that memory until the application was closed, if then. Needless to say I had the program removed from the systems. It was a progam many consumers purchase and use to protect their systems. Another program from different vendor I discovered was creating files in order of 500 - 800MB silently and never deleting them at all. This could lead to system crashes and file system problems with this issue going unchecked. Again the pogram was removed as soon as it was determined there was no user setting or configuration that would control this behaviour. The point here is OSS has same issues, just more of such issues that have gone unaddressed for several years in many cases. I also know of OSS software that still to this day, after reporting the issues over 7 years ago, that can damage a user's hardware. The very hardware the OSS community preaches can be used for OSS.
If you take a look at the OSS and the number of security fixes OSS has, it has as many as those proprietary class of pograms collectively. Yes, OSS is more responsive and open to the secuity and bug fixes made to OSS than many proprietary programs and products. It has been incorrectly stated that OSS is more secure and bug free due to the open source nature of the OSS source code. The fact is this is incorrect and I believe even RS has stated a couple years back that just because the souce is available it is does not result in higher quality of code. The article and link I do not have handy.
As much as I am a supporter and user of OSS, I can tell you that OSS, as much as it has lots to offer, has some very serious downfalls in terms of overall software quality and has a geneal distain for testing software. I personally have lost many weeks due to the ongoing bugs nobody will either admit to or take the time and effort to fix. I can tell you to actually test software takes more time than to develop software. I can tell you from first had experience that developers are the same in both the proprietary and OSS world. The difference is the OSS model generally has nobody that actually can ensure OSS developers will concur with a bug and spend their energy looking at the bug if it requires any long time and effort to invesigate. I can say this with confidance as we all know that this is the case with proprietary software and and we have the very public and open OSS model that in fact proves my point for OSS. The difference is in OSS it is not cost/benefit that enters into the decision, but if a OSS developer wants to, has the skills, is the developer familar with the code in question making the fix, and/or has the time to spend on the issues/bugs.
In closing OSS has a great opportunity to shine and show that they can write better code and programs than proprietary companies. The question is will the OSS community actually dig their heels in and commit to software quality or just bury the major problems that only formal testing can uncover or be harder to uncover as the problems are buried deeper. Leaving testing to chance with a few average users knowing there is a problem and not knowing how to duplicate and quantify hard to duplicate or very time consuming issues and bugs is not how one aspires to creating stable reliable code. As a qualified QA peson I find some issues a challenge to duplicate or quantify, but I have created many scripts, a few programs and fact gathering tools to help when needed. Still reporting the facts falls on mostly deaf ears of the OSS developers or their soap opera behaviour that I have no interest to persue. The average user would not have the support of the conclusive facts and would just be written off as not knowing what they are talking about or unable to provide the requested information, assuing that information would be taken seriously.
I have purposely avoided the whole useability issue to date which goes with software quality. The reason is simple, if the bugs cannot be addressed with their facts and objective nature, getting into the subjective side of the discussion of software quality will not be productive.
#