Software Quality Assurance
ISO definition (from ISO 9000-3) lists four components necessary to assure the quality of the software development process :
✓ computer programs (code)
✓ procedures
✓ documentation
✓ data necessary for operating the software system.
Basic Definitions
Software Error – made by programmer
✓ Syntax (grammatical) error
✓ Logic error (in place of multiply, addition
operator is applied in between two operands) Software Fault –
✓ All software errors may not cause software faults
✓ That part of the software may not be executed (An error is present but not encountered….)
3. Software Failures
✓ A software fault becomes a software failure when/if it is activated.
✓ Faults may be found in the software due to the
way the software is executed or other constraints on the software’s execution, such as execution options.
• Some runs result in failures; some not.
Software development process
software fault software failure software error
The Nine Causes of Software Errors
1. Faulty requirements definition
• Usually considered the root cause of software errors
• Incorrect requirement definitions
• Simply stated, ‘wrong’ definitions (formulas, etc.)
• Incomplete definitions
• Unclear or implied requirements
• Missing requirements
• Just flat-out ‘missing.’ (e.g. Program Element Code)
• Inclusion of unneeded requirements
• (many projects fail due to including too many requirements that will never be used.
2. Client-developer communication failures
✓ Misunderstanding of instructions in requirements documentation.
✓ Misunderstanding of written changes during development.
✓ Misunderstanding of oral changes during development.
The Nine Causes of Software Errors
✓ Lack of attention
•to client messages by developers dealing with requirement changes and
•to client responses by clients to developer questions
✓ Very often, these very talented individuals come from different planets, it seems.
• Clients represent the users; developers represent a different mindset entirely some times!
3. Deliberate deviations from software requirements
✓ Developer reuses previous / similar work to save time.
✓ Often reused code needs modification, as a result it
may contains unneeded / unusable/ unnecessary code.
✓ Developer may add unapproved enhancements.
✓ developer(s) may blatantly omit functionality due to time / budget pressures.
The Nine Causes of Software Errors
4. Logical design errors
✓ Definitions that represent software requirements by means of incorrect algorithms.
•Wrong formulas; Wrong Decision Logic Tables;
incorrect text; wrong operators / operands…
✓ Process definitions: procedures specified by systems
analyst not accurate reflection of the business process specified.
The Nine Causes of Software Errors
5.Coding errors
✓ Too many to try to list.
•Syntax errors (grammatical errors)
•Logic errors (program runs; results wrong)
•Run-time errors (crash during execution)
The Nine Causes of Software Errors
6. Non-compliance with documentation & coding instructions
•Non-compliance with published templates (structure)
•Non-compliance with coding standards (attribute names…)
•Size of program;
•Other programs must be able to run in environment.
The Nine Causes of Software Errors
7. Shortcomings of the Testing Process
✓ Likely the part of the development process cut short most frequently.
✓ Incomplete test plans
•Parts of application not tested or tested thoroughly
✓ Failure to document, report detected errors and faults
•So many levels of testing.
✓ Failure to quickly correct detected faults due to unclear indications that there ‘was’ a fault.
✓ Failure to fix the errors due to time constraints.
8. User interface and procedure errors 9. Documentation errors
• Errors in the design documents
• Trouble for subsequent redesign and reuse
• Errors in the documentation within the software for the User Manuals
• Errors in on-line help, if available.
• Listing of non-existing software functions
• Planned early but dropped; remain in documentation.
• Many error messages are totally meaningless.
The nine causes of software errors are:
1.Faulty requirements definition
2.Client-developer communication failures
3.Deliberate deviations from software requirements 4.Logical design errors
5.Coding errors
6.Non-compliance with documentation and coding instructions
7.Shortcomings of the testing process 8.User interface and procedure errors 9.Documentation errors
Software quality is:
(1)The degree to which a system, component, or process meets specified requirements.
by Philip Crosby
(2)The degree to which a system, component, or process meets customer or user needs or expectations.
by Joseph M. Juran
Pressman believes that Software quality is :
• Conformance to explicitly stated functional and performance requirements, (meets the specs)
• Explicitly documented development standards, and
✓ seems to imply a documented development process
• Implicit characteristics that are expected of all professionally developed software.
✓ further seems to imply quality as found in reliability, maintainability, scalability, usability, and more
Roger Pressman’s Definition of Software Quality
18
Modern view of quality
These are quality factors with a software product :
1. Correctness
2. Reliability
3. Efficiency (includes efficiency of resource utilization)
4. Portability
5. Usability
6. Reusability
7. Maintainability
2. Reliability: Does the software deliver all features and capability without failure? Is it available when it is needed? Does it deliver functionality that is error free?
1. Correctness: A software product is correct,
if different requirements as specified in the SRS document have been correctly implemented.
Accuracy of results.
20
4. Portability
A software product is said to be portable,
if it can be easily made to work in different operating systems, in different machines, with other software products, etc.
3. Efficiency
It deals with the hardware resources needed to perform the different functions of the software system. It includes processing capabilities (given in MHz), its storage capacity (given in MB or GB) and the data communication capability (given in MBPS or GBPS).
21
5. Reusability
A software product has good reusability,
if different modules of the product can easily be reused to develop new products.
6. Usability
A software product has good usability,
if different categories of users (i.e. both expert and novice users) can easily invoke the functions of the product.
22
7. Maintainability
A software product is maintainable,
if errors can be easily corrected as and when they show up,
new functions can be easily added to the product,
functionalities of the product can be easily modified, etc.
Software quality assurance is:
A systematic, planned set of actions necessary to provide adequate confidence that the software development process or the maintenance process of a software system product conforms to established functional technical requirements as well as with the managerial requirements of keeping the schedule and operating within the budgetary confines.
✓ Says to plan and implement systematically!
✓ Refers to a software development process
✓ Refers to the specification of technical requirements
SQA incorporates all software development processes starting from defining requirements to coding until release. Its prime goal is to ensure quality.
SQA Team: testing not only execution software but coding standards; manuals, messages displayed; resources needed;
resources named (file names, program
names,…)
Quality Dimensions
David Garvin [Gar87]:
1. Performance Quality. Does the software deliver all content, functions, and features that are specified as part of the requirements model in a way that
provides value to the end-user?
2. Feature quality. Does the software provide features that surprise and delight first-time end- users?
25
26
3. Reliability. Does the software deliver all features and capability without failure? Is it available when it is needed? Does it deliver functionality that is error free?
4. Conformance. Does the software conform to local and external software standards that are relevant to the application? Does it conform to de facto design and coding conventions? For example, does the user interface conform to accepted design rules for menu selection or data input?
Quality Dimensions
5. Durability. Can the software be maintained (changed) or corrected (debugged) without the inadvertent generation of unintended side effects? Will changes cause the error rate or reliability to degrade with time?
6. Serviceability. Can the software be maintained (changed) or corrected (debugged) in an acceptably short time period.
Can support staff acquire all information they need to make changes or correct defects?
27
Cost of Quality
Prevention costs include
quality planning
formal technical reviews
test equipment
Training
Internal failure costs include
rework
repair
failure mode analysis
28
29
•External failure costs are
•complaint resolution
•product return and replacement
•help line support
•warranty work
Achieving Software Quality
Critical success factors:
Software Engineering Methods
Project Management Techniques
Quality Control
Quality Assurance
30
31
Role of the SQA Group-I
Prepares an SQA plan for a project.
The plan identifies
evaluations to be performed
audits and reviews to be performed
standards that are applicable to the project
procedures for error reporting and tracking
documents to be produced by the SQA group
amount of feedback provided to the software project team
Participates in the development of the project’s software process description.
The SQA group reviews the process description for compliance with organizational policy, internal software standards, externally imposed standards (e.g., ISO-9001), and other parts of the software project plan.
Reviews software engineering activities to verify compliance with the defined software process.
identifies, documents, and tracks deviations from the process and verifies that corrections have been made.
Audits designated software work products to verify compliance with those defined as part of the software process.
reviews selected work products; identifies, documents, and tracks deviations; verifies that corrections have been made
periodically reports the results of its work to the project manager.
Ensures that deviations in software work and work products are documented and handled according to a documented procedure.
Records any noncompliance and reports to senior management.
Noncompliance items are tracked until they are resolved.
Role of the SQA Group-I
33
SQA Goals
Requirements quality. The correctness,
completeness, and consistency of the requirements model will have a strong influence on the quality of all work products that follow.
Design quality. Every element of the design model
should be assessed by the software team to ensure that it exhibits high quality and that the design itself conforms to requirements.
Code quality. Source code and related work products
(e.g., other descriptive information) must conform to local coding standards and exhibit characteristics that will facilitate maintainability.
Quality control effectiveness. A software team
should apply limited resources in a way that has the highest likelihood of achieving a high quality result.
SQA Goals
Software Quality Assurance vs. Software Quality Control
different objectives.
Quality Control is designed to evaluate the quality
of a set of activities developed or manufactured product
We have QC inspections during development and before deployment
QC activities are only a part of the total range of QA activities.
Quality Assurance’s objective is to minimize the cost of guaranteeing quality by a variety of activities performed throughout the development / manufacturing processes / stages.
✓ Activities prevent causes of errors; detect and correct them early in the development process.
✓ QA substantially reduces the rate of products that
do not qualify for shipment and/at the same time, reduce the costs of guaranteeing quality in most cases.
(1) Assuring an acceptable level of confidence that the software will conform to functional technical requirements.
(2) Assuring an acceptable level of confidence that the software will conform to managerial scheduling and budgetary requirements.
(3) Initiation and management of activities for the improvement and greater efficiency of software development and SQA activities.
(1) Assuring an acceptable level of confidence that the software maintenance activities will conform to the functional technical requirements.
(2) Assuring an acceptable level of confidence that the software maintenance activities will conform to managerial scheduling and budgetary requirements.
(3) Initiate and manage activities to improve and increase the efficiency of software maintenance and SQA activities.
