Available online at www.sciencedirect.com

ScienceDirect

Procedia Computer Science 20 (2013) 256 - 261

Complex Adaptive Systems, Publication 3 Cihan H. Dagli, Editor in Chief Conference Organized by Missouri University of Science and Technology

2013- Baltimore, MD

Basis Path Analysis for Testing Complex System of Systems

Francisco Zapataa*, Aditya Akundia, Ricardo Pinedaa, and Eric Smitha

aResearch Institute for Manufacturing and Engineering Systems, Universtiy of Texas at El Paso, 500W. University Ave., El Paso, TX 79968, USA

Abstract

In Systems of Systems (SoS), a major challenge is to determine how to design a test suite that will check that the complete SoS mission and objectives are achieved. Combinatorial strategies that test for every interface in the SoS are not always optimal due to their exponential nature, and given the mission of the SoS, not all the constituent systems might have to actually interact with each other. To overcome these problems, this paper proposes the use of Basis Path Testing, which is a white-box software testing technique that makes use of Graph Theory to analyse the complexity of a structured system by creating a control flow graph from each of the program's functions to design an optimal test suite. This test suite is a set of paths that traverse through the functions, which are assumed linearly independent and that can be used to create a test strategy that will exercise all of the program's functions at least once to verify and validate their functionality. By applying Basis Path Testing analysis to the constituent systems in a SoS, the tester can develop an optimal test suite that will guarantee that all possible independent paths, all possible logical decisions, and all their interfaces are executed at least once. This paper presents a SoS sample architecture and show how to generate a test suite using Basis Path Testing analysis. © 2013 The Authors. Published by Elsevier B.V.

Selection and peer-review under responsibility of Missouri University of Science and Technology Keywords: Basis path testing; complex SoS; systems of systems; test suite

1. Introduction

A System of Systems (SoS) can be defined as a Meta system composed of many independent constituent systems, which play a key important role in the systems behaviour [3]. These independent systems behave based on their own mission and the multiple possible interactions between them comprise the dynamic behaviour ascertained in any SoS. According to the principles of Complexity Science, the inherent complexity of any system can be well understood by characterizing the relations, interactions and the behaviour of the constituent systems [8].

Industries such as Aerospace and Defense, driven by new technology developments and innovations are emerging with unparalleled technological advances; resulting in emerging complex SoS with dynamic behaviour

* Corresponding author. Tel.: 915-412-6259; fax: 915-747-6013. E-mail address: fazapatagonzalez@utep.edu.

1877-0509 © 2013 The Authors. Published by Elsevier B.V.

Selection and peer-review under responsibility of Missouri University of Science and Technology doi: 10.1016/j.procs.2013.09.270

and knowledge emergence. It is important to understand that the integration of such independently developed systems is essential for mission critical scenarios in Defense and Aerospace industries. Testing complex systems for the individual logic of their constituent systems, and the behaviour of the SoS as a whole, goes hand in hand for successful deployment under different mission scenarios.

When a SoS has the requirement of been highly reliable, combinatorial strategies for testing can help detect many defects by testing every possible interface between the constituent systems. However, this strategy has the problem of not been very efficient because given the mission of the SoS, not all the constituent systems might have to actually interact with each other. Even further, the combinatorial nature of the strategy makes the test suite to grow at an exponential rate in relation with the number of constituent systems and their corresponding interfaces causing phenomena known as combinatorial explosion.

To overcome the problem of combinatorial explosion, this paper proposes a white-box software testing technique known as basis path testing. Basis path testing makes use of Graph Theory to analyse the complexity of a structured system by creating a control flow graph from each of the program's functions to design an optimal test suite. In this paper, we explain the underlying principles of basis path testing and show how it can be used to generate a test suite for Systems of Systems using a sample architecture based on "A war zone threat response system" (WTRS) developed along the principles of the Department of Defense Architectural Frameworks.

2. Basis path Testing

Basis Path Testing is a white-box testing technique originally proposed by Thomas McCabe [10], that helps to understand and define a logical complexity measure of a procedural design. With this logical complexity measure the tester can design a basis set of execution paths that are guaranteed to execute every statement in the program at least one time during the testing stage.

2.1. Flow Graph Notation

To analyze a SoS, the following notation is defined: constituent systems are modeled as circles, which in the graph are nodes. A node that contains a conditional capability is defined as a predicate node. Each decision point in a conditional expression containing one or more Boolean operators (and, or, etc.) should be modeled with an individual predicate node. A predicate node should have two edges coming out of it corresponding to the True and False outcomes of the evaluation. Nodes should be connected with edges or links represented with arrows that show the flow of control in a given direction. Every edge should begin and finish at a node, and they should not intersect with or go across another edge. The areas that are bounded by a set of edges and nodes are called regions. In the process of determining the total amount of regions, the tester needs to also include the area outside the graph as a region.

Fig. 1. Control flow graph of a structured SoS

2.1.1. Independent System Paths

An independent system path is defined as a path that can go from the initial node of the graph and all the way to the final node following a unique combination of nodes and edges. The basis set of independent systems paths for the sample graph above would be the following:

Path 1: 0- 1 -11

Path 2: 0- 1 -2-3-4-5-10-1-11

Path 3 0- 1 -2-3-6-8-9-10-1-11

Path 4: 0- 1 -2-3-6-7-9-10-1-11

3. Cyclomatic Complexity

Cyclomatic complexity or conditional complexity is a software metric that provides a quantitative measure of the logical complexity of a functional unit and it's defined by the number of independent system paths in the basis set [2]. Cyclomatic complexity provides an upper bound for the total number of tests that need to be executed to make sure that all possible functions have been triggered at least once.

The cyclomatic complexity of a structured SoS (representable by a control flow graph), is defined as:

CC = E - N + 2P

Where, E is the number of edges, N is the number of nodes, and P is the number of connected components (exit nodes of the SoS).

If the number of connected components (every connected SoS is a connected component) is just 1 and the exit node is connected back to the start node, then the cyclomatic complexity of the whole SoS corresponds to the cyclomatic complexity of its own control flow graph and is defined as CC = E - N + P, where P is 1 for a single SoS system.

For example in the previous example graph, the Cyclomatic complexity would be computed as follows:

CC = 14 edges - 12 nodes + 2 = 4

3.1. Deriving the Basis Set and Test Cases

Once the Cyclomatic complexity of a modeled SoS has been computed, the total number of independent system paths that will comprise the test suite can be determined. The general methodology would be as follows:

a) Using the SoS design, draw the corresponding flow graph.

b) Compute the Cyclomatic complexity of the flow graph.

c) Compute a basis set of independent system paths.

d) Compute the set of tests cases that will traverse each individual path in the basis set.

4. Application of Basis path testing to an SoS architecture

The Department of Defense Architecture Framework (DoDAF) is a common approach providing guidelines to describe system specifications, design process, system parameters, system activities and personnel involved [1],[5]. The architectural view point developed for this paper is based on operational elements, activities, tasks and information exchanges required in a system in order to successfully accomplish a war zone threat response System mission.

The illustrations below represent a traditional example of DoDAF operational views OV-1 and OV-5 developed based on a mission critical scenario of unmanned air assistance for a war zone threat response system. OV-1 illustrates a graphical overview of this complex system explaining an assistance operation, which is based on a distress signal transmitted from a surveillance drone in a war zone. This involves a sequence involving a monitoring unit that receives the signal and passes it along to the command center along with the war zone coordinates. The command center coordinates an assistance mission with an air command center and unmanned air vehicle using secure communication protocols. OV-5, a complementary view of OV-1, illustrates the operations that are conducted along with the description of activities, workflows, requirement's capture, operational planning and logistics support towards achieving a mission goal.

Fig. 2. (a) OV-1 of WTRS; (b) WTRS OV-5 control graph

Based on the operational view (OV-5) developed, we start the analysis by contemplating each constituent system to be a node in a flow graph and each decision point to be a conditional expression modelled using a predicate node. The control flow graph notation for OV-5 is modelled as shown below.

Node Description

1 Surveillance Drone

2 Determines if assistance is required based on the threat condition

3 Monitoring unit

4 Command Center

5 Secondary evaluation for threat response mission

6 Air Command Center

7 Unmanned Air Vehicle

8 Check for threat mitigation

9 Threat mitigated

Fig. 3. Control flow graph for the Basis Path analysis based on WTRS OV-5 (double circled nodes are predicate nodes)

Let's now compute the cyclomatic complexity or conditional complexity of the OV-5. The number of edges E is 12, the number of nodes N is 9, and the number of connected components P is 2, since the exit node 9 is not connected back to the start node 1.

CC = 12 — 9 + 2 = 5

Once we know the cyclomatic complexity of the graph, we can start deriving the set of independent basis paths for the test suite:

- Path 1: 1-9

- Path 2: 1-2-1-9

- Path 3: 1-2-3-4-5-1-9

- Path 4: 1-2-3-4-5-6-7-8-4-5-6-7-8-9

- Path 5: 1-2-3-4-5-6-7-8-9

5. Conclusion

Designing optimal test suites for SoS is a very challenging task because the constituent systems are heterogeneous, in different development life cycle stages, geographically distributed, have managerial and operational independence, show an evolutionary development nature, and present emergent knowledge and behaviour. Most combinatorial testing strategies aim at testing all possible interactions between constituent systems, making it not practically feasible to implement due to its exponential nature. By applying Basis Path Testing analysis to a SoS, the tester can develop an optimal test suite that will guarantee that all possible independent paths, all possible logical decisions, and all their interfaces are executed at least once. In this paper, we presented a methodology to apply basis path testing analysis to Systems of Systems, and developed a sample architecture that shows how to generate a test suite using Basis Path Testing analysis.

References

1. A. Tang, H. Jun, and C. Pin., "A Comparative Analysis of Architecture Frameworks.", Swinburn School of Technology, School of Information Technology Centre for Component Software and Enterprise Systems : Technical Report SUTIT-TR2004.01 (25 August 2004).

2. A. Watson and T. McCabe, "Structured Testing: A Testing Methodology Using the Cyclomatic Complexity Metric", National Institute of Standards and Technology (NIST), Special Publication 500-235, September, 1996.

3. C. Haskins (ed.), Systems Engineering Handbook: A Guide for Systems Life Cycle Processes and Activities, Version 3.2.2, International Council on Systems Engineering (INCOSE) Document INC0SE-TP-2003-002- 03.2.2, San Diego, CA, 2011.

4. C. Keating, "Emergence in System of Systems", in: M. Jamshidi (ed.), System of Systems Engineering: Innovations for the 21st Century, John Wiley and Sons, 2009, pp. 169-190.

5. C. Kobryn and C. Sibbald, "Modeling DoDAF Compliant Architectures The Telelogic Approach for Complying with the DoD Architectural Framework.", A Telelogic White Paper, October 25, 2004.

6. F. Zapata, R. Pineda, and M. Ceberio, "How to Generate Worst-Case Scenarios When Testing Already Deployed Systems Against Unexpected Situations", Proceedings of the North American Fuzzy Information Processing Society, Annual Conference NAFIPS 2013, Edmonton, Canada, June 24-28, 2013.

7. R. Brooks and A. Sage, "System of systems integration and test", Information Knowledge Systems Management, 2005/2006, Vol. 5, pp. 261-280.

8. R. Pineda, "Understanding Engineered Complex Systems of Systems (CSoS), Proceedings of The Fourth General Assembly of the Cartagena Network of Engineering CNE'2010, Metz, France, September 21- 24, 2010.

9. R. Pressman, and I. Darrel, Software engineering: a practitioner's approach. Vol. 5. New York: McGraw-hill, 1992.

10. T. McCabe, "A Complexity Metrics", IEEE Trans. Software Engineering, Vol. 2, No. 4, 1976, pp. 308-320.