Lectures Software Engineering - Chapter 19: Verification and Validation

42 trang hoanguyen 3860

Download

Bạn đang xem 20 trang mẫu của tài liệu "Lectures Software Engineering - Chapter 19: Verification and Validation", để tải tài liệu gốc về máy bạn click vào nút DOWNLOAD ở trên

Tài liệu đính kèm:

lectures_software_engineering_chapter_19_verification_and_va.ppt

Nội dung text: Lectures Software Engineering - Chapter 19: Verification and Validation

Verification and Validation l Assuring that a software system meets a user's needs ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 1
Objectives l To introduce software verification and validation and to discuss the distinction between them l To describe the program inspection process and its role in V & V l To explain static analysis as a verification technique l To describe the Cleanroom software development process ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 2
Topics covered l Verification and validation planning l Software inspections l Automated static analysis l Cleanroom software development ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 3
Verification vs validation l Verification: "Are we building the product right" l The software should conform to its specification l Validation: "Are we building the right product" l The software should do what the user really requires ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 4
The V & V process l Is a whole life-cycle process - V & V must be applied at each stage in the software process. l Has two principal objectives • The discovery of defects in a system • The assessment of whether or not the system is usable in an operational situation. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 5
Static and dynamic verification l Software inspections Concerned with analysis of the static system representation to discover problems (static verification) • May be supplement by tool-based document and code analysis l Software testing Concerned with exercising and observing product behaviour (dynamic verification) • The system is executed with test data and its operational behaviour is observed ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 6
Program testing l Can reveal the presence of errors NOT their absence l A successful test is a test which discovers one or more errors l The only validation technique for non-functional requirements l Should be used in conjunction with static verification to provide full V&V coverage ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 8
Types of testing l Defect testing • Tests designed to discover system defects. • A successful defect test is one which reveals the presence of defects in a system. • Covered in Chapter 20 l Statistical testing • tests designed to reflect the frequence of user inputs. Used for reliability estimation. • Covered in Chapter 21 ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 9
V& V goals l Verification and validation should establish confidence that the software is fit for purpose l This does NOT mean completely free of defects l Rather, it must be good enough for its intended use and the type of use will determine the degree of confidence that is needed ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 10
V & V confidence l Depends on system’s purpose, user expectations and marketing environment • Software function » The level of confidence depends on how critical the software is to an organisation • User expectations » Users may have low expectations of certain kinds of software • Marketing environment » Getting a product to market early may be more important than finding defects in the program ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 11
Testing and debugging l Defect testing and debugging are distinct processes l Verification and validation is concerned with establishing the existence of defects in a program l Debugging is concerned with locating and repairing these errors l Debugging involves formulating a hypothesis about program behaviour then testing these hypotheses to find the system error ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 12
V & V planning l Careful planning is required to get the most out of testing and inspection processes l Planning should start early in the development process l The plan should identify the balance between static verification and testing l Test planning is about defining standards for the testing process rather than describing product tests ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 14
The structure of a software test plan l The testing process l Requirements traceability l Tested items l Testing schedule l Test recording procedures l Hardware and software requirements l Constraints ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 16
Software inspections l Involve people examining the source representation with the aim of discovering anomalies and defects l Do not require execution of a system so may be used before implementation l May be applied to any representation of the system (requirements, design, test data, etc.) l Very effective technique for discovering errors ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 17
Inspection success l Many diffreent defects may be discovered in a single inspection. In testing, one defect ,may mask another so several executions are required l The reuse domain and programming knowledge so reviewers are likely to have seen the types of error that commonly arise ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 18
Inspections and testing l Inspections and testing are complementary and not opposing verification techniques l Both should be used during the V & V process l Inspections can check conformance with a specification but not conformance with the customer’s real requirements l Inspections cannot check non-functional characteristics such as performance, usability, etc. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 19
Program inspections l Formalised approach to document reviews l Intended explicitly for defect DETECTION (not correction) l Defects may be logical errors, anomalies in the code that might indicate an erroneous condition (e.g. an uninitialised variable) or non-compliance with standards ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 20
Inspection pre-conditions l A precise specification must be available l Team members must be familiar with the organisation standards l Syntactically correct code must be available l An error checklist should be prepared l Management must accept that inspection will increase costs early in the software process l Management must not use inspections for staff appraisal ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 21
Inspection procedure l System overview presented to inspection team l Code and associated documents are distributed to inspection team in advance l Inspection takes place and discovered errors are noted l Modifications are made to repair discovered errors l Re-inspection may or may not be required ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 23
Inspection teams l Made up of at least 4 members l Author of the code being inspected l Inspector who finds errors, omissions and inconsistencies l Reader who reads the code to the team l Moderator who chairs the meeting and notes discovered errors l Other roles are Scribe and Chief moderator ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 24
Inspection checklists l Checklist of common errors should be used to drive the inspection l Error checklist is programming language dependent l The 'weaker' the type checking, the larger the checklist l Examples: Initialisation, Constant naming, loop termination, array bounds, etc. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 25
Inspection checks
Inspection rate l 500 statements/hour during overview l 125 source statement/hour during individual preparation l 90-125 statements/hour can be inspected l Inspection is therefore an expensive process l Inspecting 500 lines costs about 40 man/hours effort = £2800 ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 27
Automated static analysis l Static analysers are software tools for source text processing l They parse the program text and try to discover potentially erroneous conditions and bring these to the attention of the V & V team l Very effective as an aid to inspections. A supplement to but not a replacement for inspections ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 28
Stages of static analysis l Control flow analysis. Checks for loops with multiple exit or entry points, finds unreachable code, etc. l Data use analysis. Detects uninitialised variables, variables written twice without an intervening assignment, variables which are declared but never used, etc. l Interface analysis. Checks the consistency of routine and procedure declarations and their use ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 30
Stages of static analysis l Information flow analysis. Identifies the dependencies of output variables. Does not detect anomalies itself but highlights information for code inspection or review l Path analysis. Identifies paths through the program and sets out the statements executed in that path. Again, potentially useful in the review process l Both these stages generate vast amounts of information. Must be used with care. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 31
138% more lint_ex.c #include printarray (Anarray) int Anarray; { printf(“%d”,Anarray); } main () { int Anarray[5]; int i; char c; printarray (Anarray, i, c); printarray (Anarray) ; } 139% cc lint_ex.c 140% lint lint_ex.c lint_ex.c(10): warning: c may be used before set lint_ex.c(10): warning: i may be used before set printarray: variable # of args. lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(10) printarray, arg. 1 used inconsistently lint_ex.c(4) :: lint_ex.c(11) LINT static analysis printf returns value which is always ignored
Use of static analysis l Particularly valuable when a language such as C is used which has weak typing and hence many errors are undetected by the compiler l Less cost-effective for languages like Java that have strong type checking and can therefore detect many errors during compilation ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 33
Cleanroom software development l The name is derived from the 'Cleanroom' process in semiconductor fabrication. The philosophy is defect avoidance rather than defect removal l Software development process based on: • Incremental development • Formal specification. • Static verification using correctness arguments • Statistical testing to determine program reliability. ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 34
Cleanroom process characteristics l Formal specification using a state transition model l Incremental development l Structured programming - limited control and abstraction constructs are used l Static verification using rigorous inspections l Statistical testing of the system (covered in Ch. 21). ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 36
Formal specification and inspections l The state based model is a system specification and the inspection process checks the program against this model l Programming approach is defined so that the correspondence between the model and the system is clear l Mathematical arguments (not proofs) are used to increase confidence in the inspection process ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 38
Cleanroom process teams l Specification team. Responsible for developing and maintaining the system specification l Development team. Responsible for developing and verifying the software. The software is NOT executed or even compiled during this process l Certification team. Responsible for developing a set of statistical tests to exercise the software after development. Reliability growth models used to determine when reliability is acceptable ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 39
Cleanroom process evaluation l Results in IBM have been very impressive with few discovered faults in delivered systems l Independent assessment shows that the process is no more expensive than other approaches l Fewer errors than in a 'traditional' development process l Not clear how this approach can be transferred to an environment with less skilled or less highly motivated engineers ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 40
Key points l Verification and validation are not the same thing. Verification shows conformance with specification; validation shows that the program meets the customer’s needs l Test plans should be drawn up to guide the testing process. l Static verification techniques involve examination and analysis of the program for error detection ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 41
Key points l Program inspections are very effective in discovering errors l Program code in inspections is checked by a small team to locate software faults l Static analysis tools can discover program anomalies which may be an indication of faults in the code l The Cleanroom development process depends on incremental development, static verification and statistical testing ©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 19 Slide 42