Linux.com

Once, I downloaded a language file, and ran it through a spellchecker and fixed the typos and spelling mistakes and sent it back, and it got included in the next version of the software.<nobr> <wbr></nobr>:)

Took me only a couple of minutes, and no special skills was needed.

#

Thanks, that's a great tip for people looking for ways to contribute!

#

One time I translated a software from English to Swedish. It was easy, no programming skills needed.
I hope to do more translation work in the future.

#

I have been party to various bugs and the processes various OOS projects have to report bugs. I find that OSS has no displine for the basics of Software Quality Assuarance and in fact there is no displine in this regard. This has been the subject of so many discussions in articles like these and/or the actual mailing lists of many major projects.

Case in point the issue of memory problems of one cornerstone large OSS project were reported over 3, perhaps 4 years ago for issues that had been ongoing for over 2 years before that. Serious memory leaks that also put the same key software at serious security risk not to mention the major strain it places on the Operatings Systems. The response to the bug reports was they were aware and working on it. The problems progressed to very severe instead.

Now over the past year memory problems are listed fixed as security related issues are cited as reason for the security fix, yet for a few months the memory problems still progressed much worse. Then the serious worse issues appeared to be fixed, but the core issues that have been about for over 5 years remain.

To really see how uncommitted many of the key OSS projects are to bug fixes just look a the bug reports, how long the bug reports stay open, the soap opera the bug reports take on and the attitude of those reviewing the bug reports. While I appreciate there are those bugs that are not well definied or supported with facts, there are many that are. Those that well defined and reported drag on and on and on.

Personally as a QA Specialist with many years of experience I gave up reporting OSS bugs/issues. Those I did report I abandoned as the attitude, soap opera, lack of maturity, inability to discuss facts was just a waste of my time. The icing on the cake was I was told by the buglist moderators I would have to put up with the gross lack of maturity. That for me was the proof that I was wasting much of my time that could be more productive on other tasks I had and/or me taking on code when I could to sort out some of the issues myself.

Some may ask me I should contribute the fixes back. I tried that and gave up on that as again the patch was riddled with opinions if was coded correctly, if meets the personal coding standards of the lead project developer, and generally complain about the code is not fixing anything. So I do not provide any of the code fixes I spent weeks or months figuring out when a project developer could effect the code cahnge in a hour or two. Remember I am not a developer so I cannot figure out the more complicated deep in code across many lines of code problems. My code is good as I am used to coding from my assembler coding experience.

I know there will be many that disagree with my opinion. That is your right to do so. Am I the last authority about QA Testing? No. I will leave my experience and knowledge at that.

I will say this in closing. I have used OSS and Linux for over 7 years. I am not a Windows fan. That said I can say that sadly there are far less commerical programs with the severe problems that plague many of the key OSS projects. I am very tired of the "attitude" of these people that know that there is no problem. Even worse when the facts are conclusive they will blame some other software. For some problems I have managed to find in the code where the problem is and fix it. I am not a developer so it takes me much more time to get to know and understand the code to find out where the issue is. It is amazing how basic some of these bugs are and their root cause. What may take me weeks or months to figure out I am sure a project developer would be able to find and fix in hour or less for most of the bugs I have fixed.

I have had some good experiences with a rare handful of OSS persons and/or projects. In my experience when one has such rare people involved in the bug screening, reporting and validation process the project gains. The problem is that most OSS projects do not have such people and they often are only part of the project for a short time. It is such a shame as the whole experience is enjoyable, and in end either results in a fix or understanding why this may not be a bug, future feature to be worked on, or limitation. Even if it takes much time in terms of testing, doing what is asked to try to add information for the QA Gatekeeper to evaluate the issue it is ok. Sometimes not convient, but that is ok, at least the activity makes sense, is objective and enables all involved to be productive.

#

C'mon, admit it, you're talking about Firefox/Seamonkey, aren't you?
All they'd have to do is fix how they handle bookmarks and history, and most of the memory problems would go away (and it wouldn't take 15 minutes to exit (on Debian, up to an hour on MS-Windows 95)).

#

Sadly there are a number of cornerstone programs with the general quality issues I noted. No one program has all of these issues, but most have many, if not most, of the issues.

It is very interesting you make a conclusion that I am pointing to seamonkey/firefox. This is suggestive that people are aware that the issues I noted exist.

With regards to your comment about seamonkey/firefox being the example source of issues, if that is the case then there should be effort made to address the issues. If people know about these issues then I think you have actually proved my major points on OSS code quality and the lack of focus, in part lack of quality focus, but the overall lack of focus that has quality as a casulty of the lack of focus.

As to your comments about no more than 15 minutes to exit on Debian, of no more than hour on Windows you are confirming how people make asumptions and conclusions that nobody could possibility have a very consistant different experience to another's experience or assumption to cause of issue. Basically what you may be implying is issues reported by others could not possibiliy happen. I know for a fact that typically with OSS projects there assumption is if they cannot experience the problem it cannot possibility happen. Actually this is true of even commerical based software development as well.

Do you experience most of issues the many bugs fixed in the programs you use in Debian? I would suspect not, but I am sure you, like most people, apply the security and bug fixes so you will not. Therefore you are not experiencing most of the issues the many bug fixes fix, but wish to do your best to not have any of the possible issues by applying the updates.

This means in general most people do not experience directly about 99% of the bugs that are fixed. These bug and security fixes are by and large valid fixes for valid issues. No question about that. My point is most people generally have not actually experienced the issue, ergo one cannot state that one's experience in using a program is the one and only way the program will behave (or misbehave) and one cannot conclude if one has not experienced the issue it cannot exist.

QA Anonymous

#

I agree.

There is a lot of denial in the OSS world especially regarding bugs in desktop software.

Yes, the progress is fast but the quality lacks. Also, the answer one gets is almost always "upgrade to latest version" so one has to upgrade to the latest distro version.

This process is not going to work in the long run. The OSS world must realise this.

#

I agree that there is not enough testing of free software. Part of the problem is that users are not encouraged enough to do this.

Linux distros make public releases of beta and release candidates in the same way that they make final releases.

What they should instead do is make these testing versions *really* available for testing-- and what that would entail is having testers register to obtain their beta for testing and have as part of those releases a simple gui app for people to use to daily write out any bugs that they encountered in a way that can be easily tracked and uploaded to bugzilla.

As it is betas and RCs are mostly used by people who just can't wait for the next release, instead of being used for it's real purpose-- to test the software for bugs.

#

Using the most "successful" proprietary product as a guide, you have to admit that QA fails miserably. It's nothing but mouthwash.

Proprietary shops have learned that it is cheaper to release buggy software than it is to release better software.

#

I think you are missing a key point I made. I was not discussing the overall bugginess of commerical code vs OSS. The example issue I was making against OSS is there are serious bugs that have been about for years in most of the major cornerstone OSS programs >80% of those that use OSS use.

By the very nature of those serious bugs, it can be well argued 'the most "successful" proprietary product' is in fact less buggy and especially considering the excellent examples of at least one major very "successful" OSS project. It is by no means the only one OSS project. A very "successful" OSS project took over 7 years to attempt to fix issues that were identified at least 3-4 years ago, actually turned far worse than better for almost a year with releases over that time when the sucessful major OSS project stated they were aware of and working on same said issues. Then once the major issue clearly became much worse the same major very "successful" OSS project managed to deal with that dip to only get back to point of before the far deeper dip in the bugs. I suspect this was because of its high visiability to the aveage user, but still it took a few months to have the issue address that in esence was like a DoS attack to a user's system for a common staple function most users use. This clearly demonstrates how code was released that never identified the issue in the "casual" testing/beta cycle of hoping to identify issues. To date the problem still exists. The fixes listed in the release notes keep repeating over several releases of the security fixes made for the same issue and yet the bug continues, not to mention the repeating bug statement exactly the same over those several releases. One has to wonder why the same bug has been fixed over several releases and still is not fixed.

To put in perspective how serious this very "succesfull" major OSS software bug is I will give you two examples. At the big dip down the "successful" application would increase memory use from about 50MB to over 700MB just doing one simple common routine action 10 times or less. This happened about 2 yeas ago. A simple function that could in about 30 minutes, due to he bug causing the action to take longer each time as well as burn up more memory. Not only would this very "successful" OSS program grow from 50 MB to 700+ MB in progressively larger increments, it would would also loop such that one keystroke would take about 40 minutes or more to register in the application once the application memory approached available RAM. The very "sucessful" OSS program would bring the entire system to its knees and send the paging subsystem into thrashing simply because the application was looping through its memory leak big time. This then would cause any mouse or keyboard action for the entire system to take 40 minutes as well to effect. What is even worse is this same very "successful" OSS program would cause the exact same problem when run in a "proprietary" OS as well. This clearly proved with no extra effort required that this very "successful" OSS program was the sole cause of the bug.

The second example for this very same very "successful" OSS project is the ongoing memory leaking that climbs to well over 700 MB. If one closes the same OSS program and start it again exactlay in the "state" it was it will use about 50-125 MB depending on what the state was when the OSS program climbed to over 700 MB. That clearly shows a serious memory leak. If one closes some of the states open the memory is no released further proving the serious nature of this OSS bug. As a consequence this also stresses any OS this very "successful" OSS program does run on or brings the OS to its knees. It is very easy to demonstrated that this ongoing bug of this very "successful" OSS program is a risk for DoS for a user using the very "successful" OSS project, let alone hackers that once they find the trigers for this can have a field day. This is in part why I will not name the OSS project/program.

I can assure you almost no proprietary product would ever be released with such a serious and grossly major issue. Not even the most "successful" proprietary product has such issues. I have only seen one instance of common used proprietary product that had a memory issue. It was a poduct, not an OS, and it would suddendly spike memory in less than a second by at least 300 MB and not release that memory until the application was closed, if then. Needless to say I had the program removed from the systems. It was a progam many consumers purchase and use to protect their systems. Another program from different vendor I discovered was creating files in order of 500 - 800MB silently and never deleting them at all. This could lead to system crashes and file system problems with this issue going unchecked. Again the pogram was removed as soon as it was determined there was no user setting or configuration that would control this behaviour. The point here is OSS has same issues, just more of such issues that have gone unaddressed for several years in many cases. I also know of OSS software that still to this day, after reporting the issues over 7 years ago, that can damage a user's hardware. The very hardware the OSS community preaches can be used for OSS.

If you take a look at the OSS and the number of security fixes OSS has, it has as many as those proprietary class of pograms collectively. Yes, OSS is more responsive and open to the secuity and bug fixes made to OSS than many proprietary programs and products. It has been incorrectly stated that OSS is more secure and bug free due to the open source nature of the OSS source code. The fact is this is incorrect and I believe even RS has stated a couple years back that just because the souce is available it is does not result in higher quality of code. The article and link I do not have handy.

As much as I am a supporter and user of OSS, I can tell you that OSS, as much as it has lots to offer, has some very serious downfalls in terms of overall software quality and has a geneal distain for testing software. I personally have lost many weeks due to the ongoing bugs nobody will either admit to or take the time and effort to fix. I can tell you to actually test software takes more time than to develop software. I can tell you from first had experience that developers are the same in both the proprietary and OSS world. The difference is the OSS model generally has nobody that actually can ensure OSS developers will concur with a bug and spend their energy looking at the bug if it requires any long time and effort to invesigate. I can say this with confidance as we all know that this is the case with proprietary software and and we have the very public and open OSS model that in fact proves my point for OSS. The difference is in OSS it is not cost/benefit that enters into the decision, but if a OSS developer wants to, has the skills, is the developer familar with the code in question making the fix, and/or has the time to spend on the issues/bugs.

In closing OSS has a great opportunity to shine and show that they can write better code and programs than proprietary companies. The question is will the OSS community actually dig their heels in and commit to software quality or just bury the major problems that only formal testing can uncover or be harder to uncover as the problems are buried deeper. Leaving testing to chance with a few average users knowing there is a problem and not knowing how to duplicate and quantify hard to duplicate or very time consuming issues and bugs is not how one aspires to creating stable reliable code. As a qualified QA peson I find some issues a challenge to duplicate or quantify, but I have created many scripts, a few programs and fact gathering tools to help when needed. Still reporting the facts falls on mostly deaf ears of the OSS developers or their soap opera behaviour that I have no interest to persue. The average user would not have the support of the conclusive facts and would just be written off as not knowing what they are talking about or unable to provide the requested information, assuing that information would be taken seriously.

I have purposely avoided the whole useability issue to date which goes with software quality. The reason is simple, if the bugs cannot be addressed with their facts and objective nature, getting into the subjective side of the discussion of software quality will not be productive.

#

While I suspect you are correct about the unaddressed, and serious, issues in many widely used programs, I think you make a fundamental error in comparing FOSS(Free and Open Source Software) to proprietary software on a quality basis. The benefits of FOSS do not primarily lie in their greater security or quality, but in the lack of lock-in and greater verifiability.

With a proprietary program, if the vendor chooses not to fix something, you as a user have no recourse; if the program's data format is proprietary, you may not even be able to use an entirely different program without re-entering all your data. This is never the case with FOSS, no matter what happens. This is a CRITICAL benefit, and one that is unaffected by the quality or lack thereof of the software.

When it is necessary to be sure of what a program does, in detail, FOSS is a big winner; with proprietary software, you can never be sure what it does, or what it might do later. With FOSS, you can hire independent reviewers to examine what you will be running, and ensure it does only what is intended.

Finally, you seem to confuse a common organizational model used to produce much FOSS code with the licensing system itself; they are not the same. FOSS can, and often has, been developed by a small, paid team, including dedicated testers.

(On a side note, (maybe another commenter can expand on this) you don't seem to have considered that the bugs you are reporting, as important as they are, may also be very difficult to solve, and may simply be beyond the skills, or available time, of the developers who have read your reports. The solution to this, of course, is to find developers who do have the skills/time to implement patches for your reports. How to do this? Pay someone, or wait.)

--DifferentAnonymousReader

#

Hello DifferentAnonymousReader,

Thanks for your posted comments and thinking process behind them. I like to respond to your points so we are on the same page.

Thanks for confirming my general point "... suspect you are correct about the unaddressed, and serious, issues in many widely used programs<nobr> <wbr></nobr>...". I am happy to see I am not the only one to share this opinion.

I was not trying to suggest that OSS/FOSS has better security or quality due to the open nature of the code. My comparison to proprietary software was simply to point out a few basic points of differences or common themes that exist in software that transcend FOSS/OSS and proprietary software. Those points being:


    1) Developers generally have the same attitude in both venues, proprietary software and FOSS/OSS, in that they often will say the problem cannot exist because they do not experience the issue, they know better what feature and how to implement the feature, or are unwilling to make the effort to validate/consider the issue.


    2) The serious issues that I stated exist in FOSS/OSS as rule never happen with proprietary software as too much is at state. Of course there are proprietary software examples where clearly functionality was not thought out or changed even if not buggy in addition to some classic buggy releases or bug fixes that messed up more than fixed issues. Generally the serious nature of system crippling software behaviour, let alone going on over several releases and years of a proprietary software program do not happen. This was the basic point I was making.


    3) FOSS/OSS really has no formal QA or testing plans. To highlight this more there are a number of key FOSS/OSS projects that are interrelated and there is no effort to QA these as a collective and their interactions/assumptions let alone the number with many intermediate releases. What happens often is alot of fingerpointing that problem is some other FOSS/OSS project by the FOSS/OSS project the issue is beign reported to.


    4) The FOSS/OSS issues/security fixes are more open book in what they are about. The code fixed by implication means the exact nature of a fix is known. Security fixes are geneerally more responsive in time for FOSS/OSS than proprietary software. Proprietary software often only fixes the issue if the issue is publicly known or will be made made public.


    5) One reason for these major FOSS/OSS issues is because it depends on developers having the free time to work on the issue and the extent of time it may take to work on the issue. This is nowithstanding a comment you make I will address a bit later in this reply.

You are 100% correct for one reason, at least for me, why I like FOSS/OSS. Open data formats, no data lock in, not being forced to re-enter data to use a different program, the code does only what you expect and can be inspected to confirm, et al. Whereas with proprietary software one can never be sure and as you know there have been many privacy issues uncovered with proprietary software and likely proprietary software will likely continue to make attempts regarding privacy or other unaccceptable behaviour on hopes it will nobody will discover, have time to discover or the proprietary software vendor may try to use the DMCA or copyright to prevent people discovering their unacceptable practices.

I tend to believe the FOSS/OSS code generally has better security as the code can be verified. Some of these FOSS/OSS programs are very secure as they are the basis of the key servers and software infrastructure that much traffic and dependance is on for the internet. That said, these are not the core pieces like average or us tend to use.

I do not believe I am confused about the FOSS/OSS model. Actually for the smaller projects their software quality and issue reporting process is generally more effective and attentive. Again, generally based on my experience. The reason of course is they are smaller projects. They have less people involved adn less people in any project is better timezone and geographical distances aside. I can say I have had excellent experiences with smaller projects and developers communication, interest, giving explanations where I need to be educated or why time will not permit. In most cases when a bug issue is found smaller projects do and will spend the time to invesigate and generally will find and fix the issue. My concern is with the major FOSS/OSS projects that are the basic building blocks most of us use and the basic model looks ok until one takes a closer look at the model vs project size and project importance. Yes the large FOSS/OSS projects have testers, but the issue is there is really no formal test plan and frankly speaking the test plan they may have is far from even reasonable scope of the functionality/complexityof the software. These projects depend it seems heavily on a Beta/RC level test phase to flush out issues. The problem with this approach is most users will stay away from Beta/RC software and the relative short times some of these phases in fact are. Those that do try to test the Beta/RC release may not have enough time, let alone skills to actually test the software. Often many use the Beta/RC more to have the extra functionality more than any interest in reporting any issues, let alone actually take time to do conscious testing. So what happens is we have the 80/20 rule where 20% of the code is used 80% of time and in general this translates to about 30% of the code tested 80% of the time. That is not a math error that I made. Many will no use the Beta/RC code until it is released as "stable". Then once the software "passes" beta or RC phase it is released as "stable" code. Users then think the code is stable and use it under that assumption. In this model this is not a reasonable assumption to make, but most assume this unaware of the shortfalls of this model. Then I would say about 40% of the functionality is used about 80% of time overall once released as "stable". Again no math error here. The 10%-20% delta of the code not really used in Beta/RC is now used and serious issues can arise, and have, just in that 10%-20% delta area, let alone the 60% of code rarely tested at all before the code was released. Therein lies the problem, but as you indicated FOSS/OSS is not chosen for it better quality.

Here is the core issue. These large major FOSS/OSS projects that are basically the building blocks of FOSS/OSS will make or break accpetance of the average user and businesses of FOSS/OSS solutions and hopefully as OS of choice. Let's skip the fact of using the "latest" vs most stable releases of code, where they exist. In some cases "stable" code has not existed for years or months at time depending on the FOSS/OSS project.

With respect to you last paragraph about time, skill level et al vs the serious nature of the issues/bugs. I believe I noted a couple factors why this was a problem - time, maturity, attitude, how familar with the code base. I believe in some cases the skill exists, but it is seems to be a matter of patience level and/or not as cool to work on such issues as it is to make new or cool new features. As I noted the adding on of features while ignoring such major issues can often bury the serious issue or futher compound it. Case in point was one project knew about the problems for few years, then when reported said were aware and addresing, but not for 5 years and security issues of no surprise are then flagged publically that the project really started to work on the issues and yet the core issue remains. During a about 6+ months the issue turned extremely worse, then that one matter seemed to be resolved, but the did not improve on the ongoing worsening issues prior, not for some time afterwards. This is likely a case where skill, familarity with the code base, changing developers often, attitudes clashing, lack of skilled QA resources (who tire of the ego, inmaturity and soap opera affairs of process) who could assist delivering the details and how to duplicate the issues so developers can then see and therefore focus on fixing he issue/bug. So while you do mention skill and time of developers, there is a big lack of skilled of QA resources that can prove and deliver the formula to duplicate issues. If the developer cannot see the issue they cannot fix it or may have to guess how to fix. That is true no matter if proprietary software or FOSS/OSS. That process of software development life cycle is the same for proprietary software and FOSS/OSS.

One other elements that may be not be obvious is with FOSS/OSS the development model for many projects, and certainly the major ones, is development colabaration via distance. For sure this has many positive qualities, but at same time the endearing quality of the model is a big hinderance development, and very much so to QA. There are communications gaps caused beyond he normal dynamics due to geographical/time zone differences that hinder any abilility to show the problem and for people to have the same hardware elements or spare test system to perform the test on. I know there are paid developers via verious sponsors or distributions and non-profit arangements of key project team members, teams and/or hardware for testing. The fact is the distance colabaration is also a hinderance as it takes greater skill for those reporting such problems to know how to package or report issues so developers can see the issue at hand.

Rememeber the issues I am discussing here are the major issues, not the simplier ones. It is the major issues that take alot more time, effort, skills and dedication that make all the other code and functionality less useful. Think of it as a car that has a mystry fuel line leak nobody can find, but the car will stall or run out of gas often at the most important times of need or use, even at times be danger if happens at the wrong time or place. So while I am aware code is sadly never perfect as much as I like it to be, if ther code has serious issues those issues can and do mask out all the wonderful productive features of the software or dependant software.

I hope my reply has clarified my posted comments thus far for you.

QA Anonymous

P.S. It is interesting to note someone about 6 months or more ago has a thesis about the various serious problems with OSS Quality and lack of process and conviction for. It was the subject of mainstream posting I will have to try to see if I can find the link(s) for.

#