CC BY-NC-ND
0Wang Chao, Qiu Jing, Liu Guanjun, Zhang Yong
To appear in:
Accepted Manuscript
Testability Evaluation Using Prior Information of Multiple Sources
PII: DOI:
Reference:
S1000-9361(14)00061-2 http://dx.doi.org/10.1016/j.cja.2014.03.029 CJA 271
Received Date: 16 July 2013
Revised Date: 1 December 2013
Accepted Date: 28 December 2013
Please cite this article as: W. Chao, Q. Jing, L. Guanjun, Z. Yong, Testability Evaluation Using Prior Information of Multiple Sources, (2014), doi: http://dx.doi.org/10.1016/j.cja.2014.03.029
This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Testability Evaluation Using Prior Information of Multiple
Sources
Wang Chaoa,b, Qiu Jinga,b, Liu Guanjuna,b*, Zhang Yonga,b
aScience and Technology on Integrated Logistics Support Laboratory, National University of Defense Technology, Changsha 410073,
Changsha 410073, China Received 16 July 2013; revised 1 December 2013; accepted 28 December 2013
Abstract
Testability plays an important role in improving the readiness and decreasing the life-cycle cost of equipment. Testability demonstration and evaluation is of significance in measuring such testability indexes as fault detection rate (FDR) and fault isolation rate (FIR), which is useful to the producer in mastering the testability level and improving the testability design, and helpful to the consumer in making purchase decisions. Aiming at the problems with a small sample of testability demonstration test data (TDTD) such as low evaluation confidence and inaccurate result, a testability evaluation method is proposed based on the prior information of multiple sources and Bayes theory. Firstly, the types of prior information are analyzed. The maximum entropy method is applied to the prior information with the mean and interval estimate forms on the testability index to obtain the parameters of prior probability density function (pdf), and the empirical Bayesian method is used to get the parameters for the prior information with a success-fail form. Then, a parametrical data consistency check method is used to check the compatibility between all the sources of prior information and TDTD. For the prior information to pass the check, the prior credibility is calculated. A mixed prior distribution is formed based on the prior pdfs and the corresponding credibility. The Bayesian posterior distribution model is acquired with the mixed prior distribution and TDTD, based on which the point and interval estimates are calculated. Finally, examples of a flying control system are used to verify the proposed method. The results show that the proposed method is feasible and effective.
Keywords: Bayes theory; Consistency check; Prior credibility; Prior information of multiple sources; Testability evaluation
College of Mechatronic Engineering and Automation, National University of Defense Technology
1. Introduction1
Testability plays an important role in improving the operational readiness and decreasing the life-cycle cost of equipment.1,2 Design for testability (DFT) focuses on the fault detection and isolation capability of equipment which are respectively measured by indexes such as the fault detection rate (FDR) and fault isolation rate (FIR). Testability demonstration and evaluation are carried out to check whether the testability indexes meet the requirements in the contract, which is useful to the producer in mastering the testability level and improving the testability design, and helpful to the consumer in making purchase decisions.3 Testability demonstration is implemented by injecting a certain amount of artificial faults into the equipment and detecting/isolating the faults with predesigned methods. The demonstration result is accepting or rejecting the testability level, which is determined by the total injected fault number and the failed detection/isolation number. Unlike testability demonstration, testability evaluation uses the testability demonstration test data (TDTD) to get the point and interval estimates of the testability indexes.
As the classical testability demonstration method suffers from such problems as a large sample, high risk and a long test period, some testability demonstration planning methods based on small-samples are studied and applied.4,5 However, the testability evaluation result is not accurate with the traditional evaluation method because of the meager amount of test data. The Bayes method provides theoretical and methodological support for the small sample test as it can take advantage of the prior information. So it is used in the testability index evaluation with small sample test data. Li et al.6,7 proposes a testability evaluation method based on the testability growing test data and field test data, in which the Dirichlet distribution is used as prior probability density function (pdf). The prior data with a growing tendency are used to obtain the parameters of prior pdf, while the field data are used to get the posterior pdf and Markov Chain Monte Carlo (MCMC) method is used to make the evaluation. Chang et al.8 propose a Bayes testability evaluation model based on prior test data and field test data, in which a hybrid prior pdf is used.
The prior testability information includes the testability prediction information (TPI), the testability expert information (TEI) and testability virtual test data (TVTD). The TPI is acquired by analyzing the testability model with testability modeling and analyzing softwares. The TEI is obtained from the opinions of relevant experts. The TVTD is from the testability virtual prototype (TVP), which is of lower risk and cost, and larger amount of test data.9-11 However, the prior information listed above cannot be directly used in the testability evaluation without pretreatment for two
Corresponding author. Tel.: +86-0731-84573397. E-mail address: gjliu342@qq.com
reasons. First, the forms of the prior information are different, which include the success-fail form (TVTD), the prior mean form and the prior interval form under a certain confidence level (TPI and TEI). Therefore, the prior information needs to be transformed to the complete prior pdf. Secondly, the premise of the usage of prior information is data consistency, which means that each source of prior information should be compatible with the TDTD.
As the prior information comes from different sources, a consistency check is needed between each prior information and TDTD.12 When the consistency check is passed, the similarity between the prior information and TDTD is quantitatively analyzed, That is, the prior credibility of the prior information is obtained, which can be used to balance the weights of the prior information in the evaluation.12,13
In the relevant literature on prior information transformation, Li7 gives a method to convert the TPI and TEI to the success-fail data used in the testability demonstration. Savchuk and Martz14 proposes a reliability evaluation method based on multiple sources of TEI, in which the maximum entropy method is used to transform the prior information of point mean and interval form to beta pdf. The maximum entropy method is also used in [15-18]. For the prior information with success-fail form, the empirical Bayesian method is usually applied to get the parameters of the beta prior pdf.19
The data consistency check methods include the parametrical and the non-parametrical methods. Zhang12 introduces the procedures of a non-parametrical method under both large and small sample conditions. Tang20 proposes a parametrical consistency check method of normal distribution under a small sample condition. Liu and Guo21 use both the parametrical and the non-parametrical methods in the consistency check for success-fail data. On the calculation of prior credibility, based on the method of [12], Liu and Wu22,23 give the analytical form of type II error, which makes the calculation of credibility more convenient. Duan et al.24 proposes a credibility calculation method based on information divergence. Deng et al.25 defines the credibility as the enclosed area of the prior and posterior pdfs.
From the analysis above, it can be seen that there is no relevant reference on the testability evaluation using prior information of multiple sources. The commonly used evaluation methods using the prior data of multiple sources in other fields tend to neglect data consistency between prior information and field test data. Furthermore, there are few methods of high operability on data consistency check and prior credibility calculation. So a Bayesian testability index evaluation method based on the prior information of multiple sources and TDTD is proposed, in which data consistency and credibility are fully considered.
The rest of the paper is organized as follows. The overall flow of testability evaluation is introduced in section 2. In section 3 the prior pdf parameter calculation methods for different forms are given. In section 4 the consistency-check method and the prior credibility calculation method are presented. In section 5 a mixed prior pdf and posterior pdf model is given, based on which the testability index point evaluation and interval evaluation are shown. Section 6 gives examples to verify the proposed method. Section 7 gives the conclusions.
2. Flow of testability evaluation
The flow of testability evaluati
ior information of multiple sources is shown in Fig. 1.
Fig. 1 Flow of testability evaluation
Firstly, the prior pdf parameters of prior information and TDTD are calculated. Secondly, consistency checks are made between prior information and TDTD with a parametrical method. For each source of prior information to pass the consistency check, a credibility of the prior information is calculated. Thirdly, a mixed prior pdf of testability index
is obtained with every prior pdf and its corresponding credibility. Then the testability index posterior pdf is obtained based on the mixed prior pdf and TDTD. Finally, testability index evaluation is carried out with the posterior pdf.
3. Parameter calculation of prior pdf
3.1. Parameter calculation for prior information with mean and interval estimate forms
In the testability prior information, the TPI and TEI are given with the following forms:
1) The point mean p0 of the testability index.
2) The interval [ p1, p2 ] under a confidence level j.
For the prior information with form 1, a point mean p0 is specified for the testability index. Defin the prior pdf of index p. So in terms of Bayes analysis, we have:
J0 p n(p)dp = p0
For the prior distribution selection with given prior forms, Dyer and Chiou26 have pr distribution over other distributions. Furthermore, the test result of testability demonstratio: So beta distribution or the conjugate distribution of binomial distribution is selected, that is:
n( p) = beta (p; a, b)
where a and b are the parameters of beta distribution.
The maximum entropy method is used to get the beta distribution parameter Shannon-Jaynes entropy for the prior pdf n(p) is27:
H (n)=—J0
So the determination of parameters becomes a process to find a comb: H (n), that is:
e that n( p) is (1)
the advantages of beta is of a binomial model.
H (n) = H (beta( p; a, b)) = — Jibeta( p; a, b)ln(beta( p; a, b))dp
,,,,, ,,, dB(a, b) dB(a, b) = ln( B(a, b)) — a1---b1 •
p))dp (3)
of a and b which maximize the entropy
ia1 = (a — 1)/ B(a, b) [b1 = (b — 1)/ B(a, b)
B(a, b) = J0 pa—1(1 — p)b—1 dp
Taking Eq. (2) into Eq. (1), we have:
p0 = a /(a + b)
Define that a* and b* are the optimal solutions to a and b respectively, and the determination process becomes:
H(beta(p;a ,b )) = max(H(beta(p; a,b)))
under the restrictions:
ia > 0, b > 0 |bpo — a(1 — po) = 0
In fact, as there exists a proportional relationship between a and b, the determination process can be tranformed to a single-parameter optimization problem.
For the prior information with form 2, the index interval under confidence level ¡x is[p1, p2], so the prior pdf satisfies:
JPl n(p)dp = x
The determination of the optimal a and b solutions becomes:
H(beta(p;a ,b )) = max(H(beta(p; a,b)))
under the restrictions:
ia > 0, b > 0
Jp2 pa—1(1 — p)b—1 dp —¡B(a,b) = 0 Jp,
(10) (11)
According to [27], the entropy of a uniform distribution is 0 and all other distributions have smaller entropy. For the parameter calculation under the restrictions in Eq. (9) or Eq. (12), the entropy of beta (1,1) is 0, and for all the other a and b combinations, the entropy is minus. So there is an optimal combination which can both maximize the entropy and satisfy the corresponding restriction.
The solution to Eq. (11) with the restriction of Eq. (12) is difficult. Usually, a gradient method with continuous penalty functions is adopted for its solution. Detailed procedures are shown in references [15] and [16].
3.2. Parameter calculation for prior information with success-fail form
The prior information with a success-fail form, such as the testability trial test data and TVTD, is given as (n, c), where n is the total fault injection times, and c is the failed detection/isolation times in n tests.
Assume that there are m groups of prior data from the same prior success-fail information source, which are defined as (ni, q), i = 1,2,...m -1, m . The point mean of group i is:
Empirical Bayes method19 is used to determine the parameters of the prior pdf as follows 1) If m is relatively large, we have
2d P-I P
m(mI P2 - kI p ) - (m - k)(I p )2
■ = m - n(I p )/i
where k = I m 1.
2) If n derived from Eq. (14) is minus, modify the expression of n as:
- m -1 n = (—)
IP-II P
IP2 -II P
3) If m=1, Eq. (14) and (15) are not available. We have:
The parameters of the testability index prior pdf are presented as:
a = n - c b = c
So the beta prior pdf is:
n( p) = beta( p; a, b) =
B(a, b)
pa-1(1 - P)b-1
4. Consistency check and credibility calculation
4.1. Consistency check
To ensure the accuracy of the testability evaluation under prior information of multiple sources, we need to make sure that the prior information is of high consistency with TDTD. So data consistency checks are used to assure the compatibilities among all prior information sources and TDTD. However, the prior information and TDTD may not be from the same population. So the consistency check is actually a process to test the comparability of two populations under a certain confidence level.
In section 3, we have transformed the prior information into complete prior beta pdfs, so a parametrical consistency check method is adopted. For the prior pdf of each prior information n(p), the interval C = [pL, pU ] with significant level a is determined by:
[Fl n(p)dp = a/2
0 (19)
Ï n(p)dp = a/2
The empirical Bayesian method is used to get the prior pdf n0(p) of TDTD. With a square-error loss function, we can get the point estimate of testability index p. According to [21], if p satisfies that pL < p < pU , the prior information is compatible with TDTD under significant level a . If p < pL or p > pU , the prior information is not adopted.
4.2. Credibility calculation
For the prior information to pass the consistency check, the prior credibility is calculated in this section. Assume that X* is the prior information, X is the TDTD. To check the consistency of the two data sources, hypotheses are constructed as:
H0 : X* and X are from the same population. H1 : X* and X are from different populations.
Define that A is the event that accepts H0, A is the event that rejects H0. Definition: The credibility of prior information is the probability that the hypothesis H0 is true when accepting H0. The analytical definition equation of prior credibility is expressed as:
R = P(H01 A) (20)
With the Bayes theorem, we have:
R = P(H0 I A)
= P(AIH0)P(H0) (21)
P(A I H0)P(H0) + P(AI Hi)P(Hi) where P(H0) is the probability of H0 and P(H1) = 1 -P(H0). From the consistency check process, we have:
P(AI H0) = 1 -a (22)
where a is the significant level of consistency check. Define that
P(AI H1) = ß (23)
where ß is the probability of accepting H0 when H1 is true. So the prior credibility can be transformed to:
R = P(H0 I A) = (1-a)P(H0) (24)
0 (1 -a) P( H0) + ßP( H1)
When there is no prior probability for H0 and H1, we consider that the prior probabilities of the two hypotheses are the same, that is, P(H0) = P(H1 ) = 0.5 . For the TVTD, P(H0) can be considered as the VV&A result of TVP.
From Eq. (24) we can see that ß is needed to get the prior credibility. So a parametrical calculation method for ß is illustrated as the following:
The prior pdf of the prior information is n(p), the prior pdf of TDTD is n0 (p). Hypotheses are set as: H0 : n(p) and n0 (p) are the same pdf. H1 : n(p) and n0 (p) are different pdfs.
In the consistency check, the interval under confidence level 1 -a is C = [pL,pU] . When H0 is true, Eq. (22) is still satisfied. So ß can be expressed as:
ß = P(AI H1) = Jn(p)dp (25)
Taking ß into Eq. (24), we can get the prior credibility of prior source X. 5. Testability evaluation model using prior information of multiple sources Assumptions:
1) There is prior information from N sources with the forms mentioned above. According to the forms of the prior
ior probab
data, we can get the prior parameters (at, bt), i = 1,2,..., N for every source.
2) Consistency checks are implemented between each of the sources and TDTD. Assume that the prior information of M (M < N) sources pass the check and the prior credibility is Rj , j = 1,2,..., M respectively.
A mixed prior pdf of the testability index is established as:
where Wj is the weight of source j and Wj satisfies that:
n( p; a, b) = I Wj ■ beta( p; a., b- )
According to the Bayes theorem, the mixed posterior pdf with TDTD (n, c) is expressed as:
n(p l(n, c)) = (1/C)
Za, +n-c-1 ,, -.b, + c-1 , y,,
Wj ■ pj (1 - p) j / B(a-
C = I Wj ■ B(aj + n -
With a square-error loss function L(p, p) = (p - p)2 , the posterior point esti
p = (1/C) •
stimate an
)/B(aj, bj )
nd variance are:
^Wj • B(a. + n-c-1,bj + c-1)/B(aj,bj)
Var( p) = a2p = (1/C) •
£wj • B(aj + n-c + 2,bj + c)/B(aj,bj)-p2
For the posterior interval estimate under confidence level 1 -y, the limits of the interval can be obtained from:
control :
^ n(p l(n, c))dp = yH
f1 n(p l (n, c))dp = y!2
6. Examples
The FDR of a flying control system is estimated to verify the proposed method. The system is composed of a power supply, a 1553B bus, an inertial measurement unit, a central computer, an integrated controller and a rudder system. The structure is shown in Fig. 2.
J 1553B Bus
Inertial Measurement
_IMU_Signal
Rudder Angle
Central Computer
Integrated Controller
Control Signal
Rudder System
Power Supply
Fig. 2. Structure of a flying control system
In the prior information, the TPI can be derived from the testability modeling and analyzing software. Here we resort to the testability analysis and design system (TADS) to establish the testability model of the flying control system and make prediction on the FDR.
The TEI is given by the relevant experts who give the mean or interval estimates of FDR according to their design experience and similar testability designs. 6
The TVTD is derived from the TVP established in [10]. Virtual fault injection and detection tests are carried out on the TVP and the test data are collected.
6.1 Example 1
The prior information of FDR includes:
1) Prediction value of FDR: p0 = 0.95 .
2) TEI 1: with a confidence level 0.95, the interval of FDR is [0.95, 0.99].
3) TEI 2: with a confidence level 0.90, the interval of FDR is [0.90, 0.95].
4) TVTD: four groups of data with the success-fail form, they are (64, 3), (100, 6), (80, 3) and (124, 8). result of the TVP is 0.90.
The TDTD is (43, 2). According to the classical evaluation method, the point estimate and standard den of FDR are: pFDR = 0.9535 , <FDR = 0.0321.
The two-sided interval estimate under 0.90 confidence level is [0.8643, 0.9917] with an intervangth of 0.1274. The two-sided interval estimate under 0.95 confidence level is [0.8419, 0.9943] with an intervangth of 0.1524. According to the methods in section 2, the parameters of prior pdfs are:
1) TPI: (a1,b1) = (18.278,0.962) ;
2) TEI 1: (a2,b2) = (296.100,9.706);
3) TEI 2: (a3,b3) = (305.000,22.540);
4) TVTD: (a4,b4) = (307.020,16.920). The significant level of the consistency check is 0.1, the intervals under 0.'confidence level of each source are:
1) [pL, pU ]1 = [0.8521,0.9976];
2) [pL, pU ]2 = [0.9502,0.9828];
3) [pL, pU ]3 = [0.9068,0.9526];
4) [pl , PU ]4 = [0.9260,0.9664]. With a square-error loss function, the point estimate of the TDTD is p = 0.9535 . We can see that the data of group 3
fail in the consistency check, so the data will not be considered.
For the prior information to pass the check, prior credibility is calculated. For TPI and TEI, the corresponding P(H0) is set as 0.5. The prior parameters and credibility are shown in Table 1.
Table 1 Prior pdf parameters and credibility of the prior information
Data source ► a b Prior credibility R
TPI 18.278 0.962 0.4774
TEI 296.100 9.706 0.6606
TVTD 307.020 16.620 0.9515
According to Eq. (26) and (28), the prior and posterior pdfs are obtained with the data in Table 1 and the TDTD.
The Bayes point estimate of FDR and posterior standard deviation are: pFDR = 0.9548 , <FDR = 0.0166 .
The two-sided posterior interval estimate of FDR under 0.90 confidence level is [0.9277, 0.9788] with an interval length of 0.0511.
The two-sided posterior interval estimate of FDR under 0.95 confidence level is [0.9199, 0.9822] with an interval length of 0.0623.
Compared with the results of classical evaluation method, the standard deviation of point estimate decreases. The interval lengths under 0.95 and 0.90 confidence levels are smaller than the interval lengths of classical method respectively. The introduction of prior information improves the accuracy of the point and interval evaluation.
6.2 Example 2
The TPI is obtained by analyzing a testability model without considering the uncertainty in fault detection and isolation and the TEI is usually subjective. The TVTD is from a prototype which is established based on a function-fault-behavior-test-environment model with a VV&A process, so it is more similar to the physical test data.
Based on the analysis above, we take TVTD as a major source of prior information. In the following examples, we will discuss the influences of certain TVTD factors on the evaluation result.
In this example, the TPI and TEI remain the same. Two groups of TVTD are added: (92, 5) and (150, 8) with the same VV&A result of 0.90.
According to the method in section 2, the prior parameters of TVTD are (a4,b4) = (512.544,28.549) with the added data. The interval estimate under 0.90 confidence level is [0.9306, 0.9620], so TVTD still passes the consistency check and a new credibility can be obtained. The prior parameters and credibility of the prior information are shown in Table 7
Table 2 Prior pdf parameters and prior credibility of prior information
Data source a b Prior credibility R
TPI 18.278 0.962 0.4774
TEI 296.100 9.706 0.6606
TVTD 512.544 28.549 0.9618
According to Eq. (26) and (28), the prior and posterior pdfs are obtained with the data in Table 2 and the TDTD. The Bayes point estimate of FDR and posterior standard deviation are: pFDR = 0.9543 , uFDR = 0.0159. The two-sided posterior interval estimate of FDR under 0.90 confidence level is [0.9306, 0.9787] with ai
nterval
interval
th an inte
length of 0.0481.
The two-sided posterior interval estimate of FDR under 0.95 confidence level is [0.9233, 0.9821] with an inte length of 0.0588.
We can see that compared with the estimate results of example 1, with the same TPI, TEI and VV&A results of TVP, the standard deviation of point estimate decreases and the interval lengths of interval estimate get smaller after increasing the groups of TVTD. The TVTD amount can influence the accuracy of testability evaluation.
To illustrate the differences of the prior and posterior pdfs between example 1 and example 2, the prior and posterior pdfs of FDR are shown in Fig. 3.
A: Prior pdfs B: Posterior pdfs
Fig. 3 Comparison of prior and posterior pdfs used in example 1 and example 2
As we can see in Fig. 3A, compared with the prior pdf of example 1, the prior pdf of example 2 has a larger peak value and a narrower shape, which means that the probability density is more concentrated. With the same TDTD, the posterior pdfs are shown in Fig. 3B. Compared with the posterior pdf of example 1, the posterior pdf of example 2 also has a larger peak value and a narrower shape and the probability density is more concentrated. The predominance of the prior and posterior pdfs of example 2 over the pdfs of example 1 explains the reason why the increase of TVTD amount can improve point and interval estimate accuracy.
6.3 Example 3
In this example, the TPI and the TEI remain unchanged, the amount of TVTD is the same as with example 2 except that the VV&A result increases to 0.95.
The prior parameters and credibility of the prior information are shown in Table 3.
Table 3 Prior pdf parameters and prior credibility of the prior information
t the VV&A The prior pa
Data source
Prior credibility R
TPI 18.278 0.962 0.4774
TEI 296.100 9.706 0.6606
TVTD 512.544 28.549 0.9815
The prior and posterior pdfs are given with the data in Table 3 and the TDTD. The beta pdfs of FDR are shown in Fig.
u FDR w
»07, 0.978
.0158. 86] with an interval
Fig. 4 Prior and posterior pdfs of FDR
The Bayes point estimate of FDR and posterior standard deviation are: pFDR = 0.9542 , The two-sided posterior interval estimate of FDR under 0.90 confidence level is length of 0.0479.
The two-sided posterior interval estimate of FDR under 0.95 confidence level is [0.9234, 0.9821] with an interval length of 0.0587.
Compared with example 2, with the same TPI, TEI and data amount of TVTD, the standard deviation of point estimate decreases and the interval lengths of interval evaluation get smaller after increasing the VV&A result of TVP. The increase in the VV&A result of TVP can modify the accuracy of testability index evaluation.
6.4 Example 4
In example 3, we find that the VV&A result of TVP will influence the accuracy of evaluation. In this example, we will further discuss the influential trend of the VV&A result.
In this example, the TPI and the TEI remain unchanged. The amount of TVTD is the same as with example 2, but the VV&A result of TVP changes from 0.60 to 0.99, and the corresponding point estimate, interval estimates under 0.90 and 0.95 confidence levels are listed in Table 4.
Table 4 FDR evaluation under different VV&A results of TVP
Point estimate
Interval estimate under 0.90 confidence level
Interval estimate under 0.95 confidence level
result Point mean Standard deviation Interval Interval length Interval Interval length
0.60 0.9549 0.0162 [0.9305,0.9791] 0.0486 [0.9224,0.9826] 0.0602
0.65 0.9548 0.0162 [0.9305,0.9790] 0.0485 [0.9226,0.9825] 0.0599
0.70 0.9547 0.0161 [0.9305,0.9790] 0.0485 [0.9228,0.9824] 0.0596
0.75 0.9545 0.0160 [0.9306,0.9789 0.0483 [0.9229,0.9823] 0.0594
0.80 0.9545 0.0160 [0.9306,0.9788] 0.0482 [0.9231,0.9823] 0.0592
0.85 0.9544 0.0159 [0.9306,0.9787] 0.0481 [0.9232,0.9822] 0.0590
0.90 0.9543 0.0159 [0.9306,0.9787] 0.0481 [0.9233,0.9821] 0.0588
0.95 0.9542 0.0158 [0.9307,0.9786] 0.0479 [0.9234,0.9821] 0.0587
0.98 0.9542 0.0158 [0.9307,0.9786] 0.0479 [0.9234,0.9821] 0.0587
0.99 0.9542 0.0158 [0.9307,0.9786] 0.0479 [0.9234,0.9820] 0.0586
With the increase of the VV&A result of TVP, the point estimate of FDR increases and the standard deviation decreases. However, when the VV&A reaches a certain level, both the point estimate and standard deviation remain unchanged. For the interval estimate, the FDR intervals under the two confidence levels become more concentrated with decreasing interval lengths. The varying tendency of interval lengths under the two confidence levels are shown in Fig. 5.
Fig. 5
0.7 0.75 0.8 0.85 W&A result ofTVP Varying tendency of interval lengths under the two confidence levels
As shown in Fig. 5, the interval lengths become smaller with the increase of t can see that the slopes of the two curves become smaller, which means that the in decreases with the increase of the VV&A result.
fV&A result of TVP. However, we uential ratio on the evaluation result
7. Conclusions
(1) A testability evaluation method using prior information of multiple sources is proposed. In the method, a consistency check is used to eliminate the incompatible prior information with the TDTD and the credibility is used to determine the weight of prior information in the mixed prior and posterior pdfs, which makes the evaluation more reasonable. Moreover, the introduction of prior credibility can balance the impact of prior information in the evaluation to prevent prior information of a certain source from overwhelming the TDTD.
(2) The introduction of prior information makes the testability evaluation results much more close to the actual testability level and can modify the accuracy of both point and interval estimates.
(3) The TVTD amount can influence the testability evaluation results. With the increase of the TVTD amount, the probability density of the prior and posterior pdfs become more concentrated, which eventually improves both the point and interval estimate accuracy.
(4) The VV&A result of TVP can also affect the testability evaluation results. Both the point and interval estimate accuracy are improved with the increase of the VV&A result of TVP. However, the influential ratio becomes smaller with the increase of the VV&A result of TVP.
Acknowledgement
This study was co-supported by the National Natural Science Foundation of China (Grant No. 51105369) and Shanghai Aerospace Science and Technology Foundation (Grant No. SAST201313).
References
1. Tian Z, Shi JY. System testability design, analysis and demonstration. Beijing: Beihang University Press; 2003 [Chinese].
2. Yang SM, Qiu J, Liu GJ. Sensor optimization selection model based on testability constraint. Chin J Aeronaut 2012;25(2):262-8.
3. Shi JY. Testability design analysis and verification. Beijing: National Defense Industry Press; 2010 [Chinese].
4. Li TM, Qiu J, Liu GJ, New methodology for determining testability integrated test scheme with test data in the development stages. Journal of Mechanical Engineering 2009,45(8):52-7 [Chinese].
5. Wang C, Qiu J, Liu GJ, Zhang Y. Testability demonstration test planning based on sequential posterior odds test method. Proc IMechE, Part O: J Risk and Reliability. Epub 2013 Nov 11.
6. Li TM, Qiu J, Liu GJ. Research on testability field statistical verification based on Bayes inference theory of dynamic population. Acta Aeronaut Astronaut Sin 2010;31(2):335-41 [Chinese].
7. Li TM. Research on optimization design and integrated evaluation of testability verification test for equipments 10
[dissertation]. Changsha: National University of Defense Technology; 2010 [Chinese].
8. Chang CH, Cao PJ, Yang JP, Hu L. Testability evaluation of complex equipment based on test data in development stages. China Mechanical Engineering 2011;23(13):1577-81 [Chinese].
9. Zhang Y, Qiu J, Liu GJ, Yang P. Integrated function-faultbehavior-test-environment model for virtual testability verification. Acta Aeronaut Astronaut Sin 2012;33(2):273-86 [Chinese].
10. Zhang Y, Qiu J, Liu GJ. A Fault Sample Simulation Approach for Virtual Testability Demonstration Test. Chin J Aeronaut 2012;25(4):598-604.
11. Zhang Y. Research on key technology of virtual testability demonstration test for equipments [dissertation]. Changsha: National University of Defense Technology; 2012 [Chinese].
12. Zhang JH. Bayes test analysis technology. Changsha: Press of National University of Defense Technology; 2010 [Chinese].
13. Huang HY, Duan XJ, Wang ZM. A novel posterior-weighted Bayesian estimation method considering the credibility of the prior information. Acta Aeronaut Astronaut Sin 2008;29(5):1245-51 [Chinese].
14. Savchuk VP, Martz HF. Bayes reliability estimation using multiple source of prior information: binomial sampling. IEEE Trans Reliab 1994;43(1):138-44.
15. Ma ZB, Zhu JS, Xu NX. Reliability assessment using multiple information sources. Chinese Journal of Computational Physics 2003;20(5):391-8 [Chinese].
16. Ma ZB, Zhu JS, Xu NX. Reliability assessment based on subjective inferences. Chinese Journal of Nuclear Science and Engineering 2003;23(2):127-31 [Chinese].
17. Gao F, Zheng XY, Liu C. A reliability verification test model based on hybrid Bayesian prior distribution. In: Proceedings of 5th International Joint Conference on Computational Science and Optimization; 2012. p. 770-4.
18. Rufo MJ, Perez CJ, Martin J. Merging experts' opinions: A Bayesian hierarchical model with mixture of prior distributions. European Journal of Operational Research 2010;207:284-9.
19. Martz HF, Waller RA (Los Alamos Scientific Laboratory, NM). The basics of Bayesian reliability estimation from attribute test data. Final report. Energy Research and Development Adminstration; 1975 Oct. Report No.: LA-6126. Contract No.: W-7045-ENG-36.
20. Tang XM. Consistency test methods in small sample situation. Syst Eng Electron 2001;23(10):66-8 [Chinese].
21. Liu H, Guo B. Methods of compatibility test of Bayes assessment for product reliability under the circumstance of small sample. Machinery Design and Manufacture 2007;(5):165-7 [Chinese].
22. Liu Q, Wu XY. Influence analysis of prior reliability in Bayes method application. Aeronautical Computing Technique 2010;40(6):1-4 [Chinese].
23. Liu Q, Wu XY. Principles of consistency and reliability for prior distributions selection in Bayesian method application. Syst Eng Electron 2010;32(11):2356-9 [Chinese].
24. Duan XJ, Huang HY. Weighted Bayesian fusion evaluation based on information divergence of prior sample. Acta Armamentar 2007;28(10):1276-80 [Chinese].
25. Deng HJ, Zha YB. Research on applying simulation credibility into weapon system appraisal. Journal of System Simulation 2005;17(7):1566-8 [Chinese].
26. Dyer D, Chiou P. An information-theoretic approach to in corporating prior information in binomial sampling. Communication in Statistics-Theoretical Methods 1984;13(17):2051-83.
27. Berger JO. Statistical decision theory and Bayesian analysis. 2nd ed. New York: Springer-Verlag; 1985.
Wang Chao received his B.S. degree in 2007 from Zhengzhou University, Zhengzhou, China, and M.S. degree from National University of Defense Technology (NUDT), Changsha, China, in 2009. He is a Ph.D. candidate at NUDT. His research interests include testability demonstration and evaluation, Bayes theory and application. E-mail: zhongwangchao@163.com
Qiu Jing is a professor and Ph.D. supervisor at the College of Mechatronic Engineering and Automation, NUDT. He received his B.S. degree from BeihangUniversity, Beijing, China, in 1985, M.S and Ph.D. degrees from NUDT, in 1988 and 1998, respectively. He has published over 100 papers, primarily on condition monitoring, fault diagnosis techniques, and technologies of decreasing false alarms. His current research interests include fault diagnosis, reliability, testability, maintenance for complex systems, etc. E-mail: qiujing16@sina.com
Liu Guanjun is a professor and Ph.D. supervisor at the College of Mechatronic Engineering and Automation, NUDT. He received his B.S. and Ph.D. degrees from NUDT in 1994 and 2000, respectively. His research interests include fault diagnosis, testability, maintenance, etc. E-mail: gjliu342@qq.com
Zhang Yong received his B.S. degree in 2004 from Wuhan University, Wuhan, China, M.S. and Ph.D. degrees from NUDT in 2006 and 2012, respectively. His research interests include fault diagnosis, and testability demonstration and evaluation. E-mail: zhangy21cn@126.com
