Testing as part of a quality system

The problem of software quality is becoming an increasingly acute, especially with increased use of information technologies and the growth of the software. High quality products not only gives developers a competitive advantage and credibility of clients, but also facilitates the maintenance and development of software. This article presents the views of staff units of software development corporation “Galactica” on the problem of creating quality software.

The developers, who stood at the origins of “Galaxy”, have gained experience in software development in the structures of the military-industrial complex. Naturally, the demands on the reliability of automated systems for command and control were very high – failure in the war meant the emergence of deadly danger for the hundreds of thousands of people. On the basis of measures designed to ensure reliability then managed to create a system in operation which did not appear any error in the software. Past experience was the basis of the corporate culture of software development, an essential aspect of which – his constant attention to quality issues.

In the field of economics and business administration right to exist has only definitely good software and not “sturgeon second freshness, so the concept of “good enough software”, which is today making a reality of many leading software vendors, it looks at least strange. You can certainly argue that the system software of human life and the corporate information system differ in terms of critical errors. But, on the other hand, the client may expect that a software product does not jeopardize the “health” and the existence of his enterprise.

Purpose and targets

Most appropriate benchmark for corporations is the experience of IBM – one of the leading developers of software for the U.S. defense projects. It is well known, for example, that three million lines of code on-board software “shuttles” are less than one error in ten thousand lines of. We will actively introduce into their practice organizational and technological expertise IBM.

Another landmark became the generally accepted standards of quality ISO 9000. According to the formulation of ISO 8402, under the Quality means the totality of characteristics of software related to its ability to meet established and projected needs of clients. The main quality parameters are considered: the functional completeness, compliance with the legislation of the CIS, information security, ease of use, requiring no special knowledge in the field of information technology, ergonomic user interface, minimizing operating costs, development and modernization.

Under the reliability usually refers to the ability of the system to perform specified functions, while maintaining the basic characteristics under certain operating conditions. With regard to the software is primarily a trouble-free operation, no errors that prevent the normal functioning of the enterprise.

Quality and reliability together ensure high consumer properties. In the process of creating a software product they are simultaneously and continuously monitored and improved. However, how realistic is to ensure quality and reliability of complex multifunctional system with limited terms of development? For illustration, we can bring the results of the survey more than a thousand large companies conducted by the Ministry of Trade and Industry. It was found that the average failure rate of information systems are: 1 failure per year – 40% of companies, a failure in a month – 29%, a failure in a week – 15% of companies, a failure on the day – 7% and 5% of companies have seen a more than one failure per day. The share of failures and software faults in the list of reasons not work (downtime) of information systems was 24% .

Depending on the kit, the system of “Galactica” may include more than three thousand interrelated business functions, the results of which are controlled by more than 300 settings. It is obvious that to achieve the required quality and reliability can only denoting them as a priority goal and continually move to it in the following areas:

• Organization of industrial production of software, with a clear specialization, the optimal distribution of functions, powers and responsibilities of staff;
• Introduction of a complex of the most modern and efficient technologies, including both technology development and maintenance of software and control technology development (project);

quality systems development based on the recommendations of ISO 9000-3.

Steps leading to quality

One of the key elements of quality assurance – is tested. Many well-known developers are testing their products in several stages, which are different types of work performed and the involved resources. Corporation “Galaxy” in this sense is no exception.

In fact, the testing begins in the process of coding the next version. In teams of specialists working on a particular part of the system have so-called “local” testers. Their mission – operational testing of newly developed or modified system functions. This “conveyor belt” organization of work can save you time and effort, since much of the errors identified and eliminated virtually the moment of occurrence. Work testers at this stage as it is localized in the part of the system developed by this group, so we’re talking about a “local” test.

It is known that when a person is a long time working on one problem, He had the certain stereotypes that are often disturbed to see their own mistakes. To avoid this, a certain degree of readiness of the system we start the cross-testing. Developers do not just “fresh eyes” check each other’s work, but at the same time share their experiences.

Both local and cross-testing is accompanied by a verification code. If the tester’s work with the system – a search for bugs in their manifestation in the course of the program, then work with the source code allows you to “catch” mistakes in routine testing does not show up immediately.

During the encoding functions are checked separately taken or blocks within a single module system. Then start testing the system as a whole (integrated testing) on a set of business processes, which are used to implement the functions of some modules. This stage of the development cycle includes several stages.

First, the work includes only division of the Department of Software Development (Department of the integral test, etc.) – this phase of the development cycle is called internal testing. Verified the functional completeness of systems, compliance with project documents, the correctness of the design decisions.

In the next stage of work involves resources, external to the Department of software development: subdivision corporations, marketing and technical support, customers – customers new system functions, other interested organizations.

The concept of “external testing” – wider than the traditional “beta testing”, in which all current and potential consumers. At the stage of external testing is the concentration of the efforts of hundreds of experienced experts, using different methodologies and different approaches to work with the system. All experts are united in a common information network system “Problems and Solutions.” Almost all units corporations involved in the external testing, and integration with the corporation “Sail” created an opportunity for the exchange of software products for cross-integral testing.

And on the domestic and external testing is constantly carried out a statistical analysis of errors found and repaired, on the basis of which the decision to move to the next stage.

Read More:

Top 7 Open Source Statistical Analysis Software

Top 10 Best Statistical Analysis Software

Final testing of the integrated department conducts tests of the Department of software development. Its mission – to re-check the implementation of the maximum number of business processes and make sure that the correction of errors at previous stages did not cause new errors. In fact, this “run” system, which is given 10 working days. For comparison – during a reception systems for military purposes to the same procedure was allocated a maximum of 4 days. We attach to it more time and resources to ensure high reliability due to complete coverage of typical business processes.

Next version of the system is transferred into trial operation at the company. This is also a responsible step, because even the most comprehensive testing is not always possible to identify all the nuances that are found in the real operation. (Typically, an interested customer acts as picky and meticulous tester.) If within one month of serious comments were received, the version passed the rest of the customers and for commercial sale.

As a result, the version on the way from developer to customer is six levels of testing, each of which is provided to minimize errors and the target set in the early development of the values of quality and reliability.

Foundation of quality and its components

All works to improve the quality of software is certainly require institutional, technical and methodological support.

Following the experience of IBM and the recommendations of ISO 9000-3, in the staffing structure of the Department of software development was the post of specialist in quality, which is functionally subordinate local testers groups and the Department of the integral test. The main task of the expert – to ensure the necessary level of quality and reliability of the software (version, release).

With regard to logistics, there is, first of all, we should note an automated testing AQA, allowing us to solve a number of issues.

• Saving resources and improving the quality of testing. Automated testing for a given scenario does not require human intervention – the system will test the software in all the right conditions, nothing is missing. Human intervention is only required for updating the script library.
• Stabilization of reliability. When the system to perform any changes, the most elusive bugs – those that occur in the already tested components. Restarting the test script after making changes can detect errors in situations where the tester is very likely would skip them. Thus, the reliability is already debugged and tested components of the system is under constant review and may not be affected by changes in other components.
• Parallel testing of the system on different platforms. Well-established test scripts can be run on any of the currently supported platforms (Btrieve, Oracle, MS SQL).

All of this relates to the testing of new versions of the full cycle of development which occupies almost half a year. In the period between releases, usually once a month out so-called “releases”, the issue of which is the need to track changes in legislation and quickly solve the problem of strategic clients of the corporation. Deadlines require special testing techniques. On the one hand, the bulk of the work shifted to a system of AQA – no team testers can not for 2 days to go through all types of business processes and ensure that changes are not reflected in the “old”, repeatedly tested functions. On the other hand, new features may require manual testing, both worked through technology verification and create a script that will then be used for automated testing with the help of AQA.

Automated testing system theoretically allows one hundred percent quality guarantee system, you need only make a comprehensive library of scripts. Traditionally it is considered that the quality of applications is a function of the number of tests. But for a complex multi-functional software, such as “Galaxy”, the creation of such a library – an extremely complex task that requires enormous resources. Therefore, we adhere to a different approach: the majority of errors are identified and eliminated in the early stages of development, and during the integrated testing of the priority role for integration tests, validating the implementation of business processes as a whole, as well as the interaction of various system modules. Development of such scenarios do testers have extensive experience in automation of large enterprises in various industries and forms of ownership.

Another source of development of quality test is the interaction with the units working directly with clients, in particular, with the services of consulting and commissioning. Description of business processes implemented with the introduction of the system at a particular company – a desirable food for the testers. And established on the basis of this description of scripts for automated testing – a guarantee of safe operation of our software in the enterprise.

Automated System Problems and Solutions “(PIR) is a means of operational control of quality and reliability, which is heavily used during the testing for registration and statistical processing of information about detected and corrected errors. At the same time PIR – a system of rapid feedback from users. The rate of income is actually determined by the speed of information transmission lines, while once known the person responsible for problem solving, and controlled as possible.

Methodological support for testing include: technology, as set out in regulations and instructions, libraries, business processes and automated test scripts, as well as an analysis of the causes of errors.

So, before you get a commercial version of the system is six levels of testing, each of which revealed a number of errors. Used at different levels of methodological base has its own characteristics and should help to reduce the number of errors in the transition from level to level. In particular, the conditions of the tests and test metrics correspond to the specification development phase. Post-project analysis, each subject to error, found out the cause and identify gaps in the methodology is not possible to detect an error at the previous levels. Thus achieving the main goal – finding the maximum number of critical errors even at the lower levels of testing and their removal to its final phase.

Technology testing depends, inter alia, the amount of information stored in the test database (DB). An indispensable element of testing is to check the efficiency of the system on an empty database, which is actually a model of a new client: the system must be configured to fill the main directory, enter the initial data. For a detailed examination of complex business processes that require adjustment to the regional or industry-specific, use the base with the volume up to 1 GB). At all levels of the integral test runs a series of standard and specific tests for a set of databases. Thus, the graph  adds another dimension (DB). As a result, testing becomes a “three-dimensional.”
This gives an additional effect: verify completeness and consistency of tuning parameters – setting specifies the algorithms to perform many functions. Excluded are also problems when upgrading, since testing new versions of the “old” databases ensures high reliability of the conversion.

Prevention advantageous treatment

Thorough testing of the software – the most obvious way to ensure its reliability. Indeed, testing – it’s diagnosis of the disease, analysis of symptoms, identifying the source and determine the best methods of treatment. But it is equally important to take preventive measures.

Warning system “disease” includes a number of arrangements, which essentially boils down to ensuring the reliability and quality at all stages of development, from designing. At present, the ratio of time spent on design, coding and testing is 40%, 20% and 40% respectively. Design is divided into several stages: the development of technical specifications, its analysis, the creation of the layout of the system. The results of each phase are subject expertise, and synergies between the cross-validation. Availability of detailed project documentation greatly reduces the likelihood of errors and an additional guarantee of reliability of the product.

It would seem, what is the interest coverage test question – one of the most important aspects of software development – is to our customers? They are interested in the final result: the system should provide a reflection of specific business processes, be easy to learn and dynamically respond to changing realities of life. And it is not so important, what means all this will be achieved. However, be sure to pay attention on efforts to improve the quality of software. And for that, there are at least two reasons:

• A high level of methodological, technical and organizational support at all stages of testing determines the quality of the product, ensure that once correct the error appears again, so – strengthens user confidence in the product;
• User engagement, continuous feedback facilitates the establishment of adequate verification schemes design decisions and also serves as a common goal – the creation of quality and reliable software.