基于测试覆盖的安全关键软件测试策略研究
|
摘要
在航空、航天及军事等安全关键系统中，软件发挥着至关重要的作用。软件一旦失效将导致系统处于危险状态、严重事故甚至灾难，这类软件被称为安全关键软件。高可靠性是对安全关键软件的基本要求。
软件测试作为保证软件质量和可靠性的重要手段已经备受关注。虽然目前在测试方法、测试过程组织和管理等方面的软件测试理论日趋成熟，但是关于“如何在时间、资源有限的条件下，对软件进行充分地测试”这个关键问题仍然没有解决。近年来，安全关键软件作为一个独立的分支被广泛地研究。由于安全关键软件具有时间紧、任务重和可靠性要求高的特点，所以软件测试亟待解决的关键问题在安全关键软件的测试中变得更为突出。
软件测试覆盖率可以有效地度量测试充分性和测试效率，其与软件缺陷、软件可靠性之间有着密切的关系。本文在分析国内外安全关键软件测试领域的研究现状基础上，对现有的软件测试理论、软件可靠性测试的研究成果以及安全关键软件的特点进行了系统全面的分析，提出了基于测试覆盖的安全关键软件测试策略，运用该测试策略指导安全关键软件的测试活动，能够提高测试效率，而且能够发现更多的软件缺陷。
本文主要贡献和研究成果如下：
一、制定了基于测试覆盖的安全关键软件测试策略。该策略的主要思想是依据测试覆盖率增长函数，选择出对测试覆盖率增长有贡献的测试用例；利用测试覆盖率信息和已经发现的缺陷数据预测出软件中残留的缺陷；通过软件缺陷分析，发现测试薄弱点，测试人员有针对性地补充测试用例，可以发现更多的软件缺陷。运用该测试策略能够有效地提高安全关键软件的测试效率和测试的充分性。
二、在研究使用测试覆盖率数据指导软件测试时，提出了测试用例对测试覆盖率贡献度的概念，并用数学方法推算出软件测试覆盖率增长函数的通用形式，提出了增强的Inflection-S型测试覆盖率函数。使用NASA公布出的测试数据对提出的函数进行验证，结果表明：本文提出的增强Inflection-S型测试覆盖率函数与已有的测试覆盖率函数相比具有更好的拟合精度，更符合实际测试工作中测试覆盖率增长的情况。
三、研究测试覆盖率与软件缺陷之间的关系。为了充分利用测试覆盖信息，本文在深入研究基于时间分布的软件缺陷预测模型基础上，建立了基于测试覆盖的软件缺陷预测模型，该模型能够克服饱和效应对缺陷预测的影响。使用NASA和ESA的共9组测试覆盖率数据和软件缺陷数据对模型进行验证，结果表明：本文提出的基于增强Inflection-S型测试覆盖率函数的缺陷预测模型与已有的软件缺陷预测模型相比，具有预测结果稳定性好、与实际缺陷的吻合度高的优点。
四、本文结合航天某型号软件的测试任务进行实验验证。通过详细地分析测试策略的适用条件和指导测试的方法，建立起理论研究与工程实践的桥梁。运用该测试策略对航天某型号软件的测试过程进行指导，实验结果表明：运用本文提出的基于测试覆盖的安全关键软件测试策略指导测试，不仅提高了测试效率，而且能够发现出更多的软件缺陷，保证有效且高效地完成了安全关键软件的测试任务。
“如何在时间、资源有限的条件下，对软件进行充分地测试”一直是软件测试领域的难题之一，本文提出的基于测试覆盖的安全关键软件测试策略在解决该难题方面进行了有益的探索，并取得了一定的进展，为相关领域的研究提供了思路。
Software plays a fatal role in safety critical systems such as aviation system,aerospace system and the military system. Once it becomes invalid, the system will beat dangerous condition or suffer from serious accidents even disasters. Such softwareis called safety critical software whose basic requirement is high reliability.
Software testing, as an important means for ensuring quality and reliability ofsoftware, has been under the spotlight. How to cover more defects of the software byexecuting fewer test cases under the urgent conditions is a puzzle confronted bysoftware testing. Measuring the sufficiency and effectiveness of software testing isalso needed to verify the high reliability of the tested safety critical software.
The testing coverage is an effective measurement index for the sufficiency andefficiency of testing, and it is closely related to the defects and reliability of thesoftware. In the paper, based on the current research situations, the testing strategiesof safety critical software based on testing coverage are proposed and used to instructtesting the certain safety critical software, which raised the efficiency of testing andthe reliability of safety critical software.
This paper mainly makes the following contributions:
Ⅰ.The testing strategies about testing safety critical software based on testingcoverage are proposed. The main idea of the testing strategies is generating the test cases by Black-box testing, and choosing the test cases contributing to testingcoverage according to its growth rate, and predicting defects remained in the softwarewith the help of the information about testing coverage and the discovered defects,and replenishing the test cases at pertinence after analyzing the properties and thesocial phenomenon of defects.
Ⅱ.After studying how to instruct the software testing with the help of the testingcoverage, the concept of the contribution of test cases to testing coverage is proposed,and the general forms of the growth function of testing coverage is given bymathematical methods, and the enhanced Inflection-S testing coverage function isproposed, which is verified by the test data issued by NASA. The results show thatthe enhanced Inflection-S testing coverage function proposed in the paper has thebetter fitting accuracy than the existing testing coverage functions, and is moreconsistent with the growth of the testing coverage in the real testing work.
Ⅲ.The paper studies the relationship between the testing coverage and thedefects of softwares. In order to make full use of the testing coverage, the predictionmodels for software defects based on the time distribution are studied, and theprediction models for software defects are proposed based on it. And it is verified bythe data of the testing coverage and the software defects issued by NASA and ESA.The results show that the enhanced Inflection-S testing coverage function proposedin the paper is more stable and more consistent with the actual number of the defects.
Ⅳ.The experiments are combined with the test tasks of the certain space software,and the theoretical researches are applied to instruct the real test tasks. The resultsshow that the testing strategies about safety critical software based on testingcoverage proposed in the paper are able to raise the efficiency of testing, and discoverthe more defects existing in the software by executing the fewer test cases under theurgent conditions, and finish the testing tasks more effectively and efficiently.
软件测试作为保证软件质量和可靠性的重要手段已经备受关注。虽然目前在测试方法、测试过程组织和管理等方面的软件测试理论日趋成熟，但是关于“如何在时间、资源有限的条件下，对软件进行充分地测试”这个关键问题仍然没有解决。近年来，安全关键软件作为一个独立的分支被广泛地研究。由于安全关键软件具有时间紧、任务重和可靠性要求高的特点，所以软件测试亟待解决的关键问题在安全关键软件的测试中变得更为突出。
软件测试覆盖率可以有效地度量测试充分性和测试效率，其与软件缺陷、软件可靠性之间有着密切的关系。本文在分析国内外安全关键软件测试领域的研究现状基础上，对现有的软件测试理论、软件可靠性测试的研究成果以及安全关键软件的特点进行了系统全面的分析，提出了基于测试覆盖的安全关键软件测试策略，运用该测试策略指导安全关键软件的测试活动，能够提高测试效率，而且能够发现更多的软件缺陷。
本文主要贡献和研究成果如下：
一、制定了基于测试覆盖的安全关键软件测试策略。该策略的主要思想是依据测试覆盖率增长函数，选择出对测试覆盖率增长有贡献的测试用例；利用测试覆盖率信息和已经发现的缺陷数据预测出软件中残留的缺陷；通过软件缺陷分析，发现测试薄弱点，测试人员有针对性地补充测试用例，可以发现更多的软件缺陷。运用该测试策略能够有效地提高安全关键软件的测试效率和测试的充分性。
二、在研究使用测试覆盖率数据指导软件测试时，提出了测试用例对测试覆盖率贡献度的概念，并用数学方法推算出软件测试覆盖率增长函数的通用形式，提出了增强的Inflection-S型测试覆盖率函数。使用NASA公布出的测试数据对提出的函数进行验证，结果表明：本文提出的增强Inflection-S型测试覆盖率函数与已有的测试覆盖率函数相比具有更好的拟合精度，更符合实际测试工作中测试覆盖率增长的情况。
三、研究测试覆盖率与软件缺陷之间的关系。为了充分利用测试覆盖信息，本文在深入研究基于时间分布的软件缺陷预测模型基础上，建立了基于测试覆盖的软件缺陷预测模型，该模型能够克服饱和效应对缺陷预测的影响。使用NASA和ESA的共9组测试覆盖率数据和软件缺陷数据对模型进行验证，结果表明：本文提出的基于增强Inflection-S型测试覆盖率函数的缺陷预测模型与已有的软件缺陷预测模型相比，具有预测结果稳定性好、与实际缺陷的吻合度高的优点。
四、本文结合航天某型号软件的测试任务进行实验验证。通过详细地分析测试策略的适用条件和指导测试的方法，建立起理论研究与工程实践的桥梁。运用该测试策略对航天某型号软件的测试过程进行指导，实验结果表明：运用本文提出的基于测试覆盖的安全关键软件测试策略指导测试，不仅提高了测试效率，而且能够发现出更多的软件缺陷，保证有效且高效地完成了安全关键软件的测试任务。
“如何在时间、资源有限的条件下，对软件进行充分地测试”一直是软件测试领域的难题之一，本文提出的基于测试覆盖的安全关键软件测试策略在解决该难题方面进行了有益的探索，并取得了一定的进展，为相关领域的研究提供了思路。
Software plays a fatal role in safety critical systems such as aviation system,aerospace system and the military system. Once it becomes invalid, the system will beat dangerous condition or suffer from serious accidents even disasters. Such softwareis called safety critical software whose basic requirement is high reliability.
Software testing, as an important means for ensuring quality and reliability ofsoftware, has been under the spotlight. How to cover more defects of the software byexecuting fewer test cases under the urgent conditions is a puzzle confronted bysoftware testing. Measuring the sufficiency and effectiveness of software testing isalso needed to verify the high reliability of the tested safety critical software.
The testing coverage is an effective measurement index for the sufficiency andefficiency of testing, and it is closely related to the defects and reliability of thesoftware. In the paper, based on the current research situations, the testing strategiesof safety critical software based on testing coverage are proposed and used to instructtesting the certain safety critical software, which raised the efficiency of testing andthe reliability of safety critical software.
This paper mainly makes the following contributions:
Ⅰ.The testing strategies about testing safety critical software based on testingcoverage are proposed. The main idea of the testing strategies is generating the test cases by Black-box testing, and choosing the test cases contributing to testingcoverage according to its growth rate, and predicting defects remained in the softwarewith the help of the information about testing coverage and the discovered defects,and replenishing the test cases at pertinence after analyzing the properties and thesocial phenomenon of defects.
Ⅱ.After studying how to instruct the software testing with the help of the testingcoverage, the concept of the contribution of test cases to testing coverage is proposed,and the general forms of the growth function of testing coverage is given bymathematical methods, and the enhanced Inflection-S testing coverage function isproposed, which is verified by the test data issued by NASA. The results show thatthe enhanced Inflection-S testing coverage function proposed in the paper has thebetter fitting accuracy than the existing testing coverage functions, and is moreconsistent with the growth of the testing coverage in the real testing work.
Ⅲ.The paper studies the relationship between the testing coverage and thedefects of softwares. In order to make full use of the testing coverage, the predictionmodels for software defects based on the time distribution are studied, and theprediction models for software defects are proposed based on it. And it is verified bythe data of the testing coverage and the software defects issued by NASA and ESA.The results show that the enhanced Inflection-S testing coverage function proposedin the paper is more stable and more consistent with the actual number of the defects.
Ⅳ.The experiments are combined with the test tasks of the certain space software,and the theoretical researches are applied to instruct the real test tasks. The resultsshow that the testing strategies about safety critical software based on testingcoverage proposed in the paper are able to raise the efficiency of testing, and discoverthe more defects existing in the software by executing the fewer test cases under theurgent conditions, and finish the testing tasks more effectively and efficiently.
引文
[1] Leveson N G. Software safety-why, what and how [J]. ACM Computing Surveys,1986,18(2):125-163
[2] Knight J C.Safety Critical Systems-Challenges and Directions [C]. Proc.of Futureof Software Engineering(FOSE07).New York:IEEE Computer Society,2007
[3] Mike F. Advances in Safety Critical Systems[R].Report for Results andAchievements from the DTI/EPSRC R&D Programme in Safety Criticalsystems,1997
[4]杨启亮,邢建春,王平.安全关键系统及其软件方法[J].计算机应用与软件,2011,28(2):129-138
[5] GJB/Z1391-2006故障模式、影响及危害性分析指南[S].
[6] GJB/Z142-2004军用软件安全性分析指南[S].
[7]刘彬彬.软件FMEA与FTA综合分析方法研究[D]:[硕士学位论文].北京:北京航空航天大学,2008
[8]宋晓秋.软件安全性分析的Petri网方法[J].质量与可靠性,1998,(1):33-36
[9]王金波.安全关键软件的测试与评估过程模型研究[D]:[博士学位论文].北京:中国科学院研究生院,2008
[10]郦萌.软件安全性3M评价中评价因子修正的研究[J].同济大学学报,2002,10:28-29
[11]郦萌.计算机软件的可靠性[M].北京:国防工业出版社,1988
[12]郦萌.安全性苛求系统中关于软件安全性评价的研究[J].计算机工程与科学,2002,24(2):59-61
[13]黎忠文,熊光泽.形式化方法与安全关键系统[J].计算机应用,2000,20(9):36-38
[14]黎忠文,陈亮,熊光泽.基于防危核/壳机制的安全关键硬实时系统响应时间分析[J].电子学报,2006(4):27-29
[15]杨仕平,桑楠,吴勇新等.基于多级关键度的高可信安全关键系统[J].系统工程与电子技术,2004,26(2):277-282
[16]杨仕平,桑楠,熊光泽.安全关键软件的防危性测评技术研究[J].计算机学报,2004,27(4):442-450
[17]覃志东.高可信软件可靠性和防危性测试与评价理论研究[D]:[博士学位论文].成都:电子科技大学,2005
[18]张德平,聂长海,徐宝文.基于Markov决策过程用交叉熵方法优化软件测试[J].软件学报,2008,19(10):2270-2279
[19] John Musa. Software Reliability Engineering [M]. McGraw-Hill,1999
[20]黄锡滋.软件可靠性、安全性与质量保证[M].北京:电子工业出版社,2002
[21]蔡开元.软件可靠性工程基础[M].清华大学出版社,1995
[22]蔡开元.关于软件可靠性和软件控制论的若干认识[J].中国科学基金.2004,年第1期:201-204
[23]徐仁佐.软件可靠性工程[M].北京:清华大学出版社,2007
[24] Jelinski Z, Moranda PB. Software Reliability Research[C]. Proceedings of theStatistical Methods for the Evaluation of Computer System Performance, AcademicPress,1972:485-502
[25] Schneidewind N F. Analysis of Error Processes in Computer Software[J].Sigplan Note,1975,10(6):337-346
[26] Goel A L, Okumoto K.Time-Dependent Error-Detection Rate Model forSoftware and Other Performance Measures [J]. IEEE Transactions on Reliability,1979,28(3):206-211
[27] Musa J D. Software Reliability: Handbook of software engineering[M].1984:392-412
[28] M. Ohba, S. Yamada. S-shaped software reliability growth models[C].Proceedings of the4th International Conference on Reliability and Maintainability,1984:430-436
[29] M. Ohba. Software reliability analysis models [J]. IBM Journal of ResearchDevelopment,1984,28:428-443
[30] M. Ohba. Inflection S-shaped software reliability growth models, in: S. Osaki, Y.Hatoyama (Eds.), Stochastic Models in Reliability Theory, Springer, Berlin,1984,pp.144-162.
[31] D. Musa and K. Okumoto. A Logarithmic PoissonExecution Time Model ForSoftware Reliability Measurement[C]. In Proc.7th International Conference onSoftware Engineering,1983:230-237
[32] M. Ohba, X.M. Chou. Does imperfect debugging affect software reliabilitygrowth[C]. Proceedings of the11th IEEE International Conference on SoftwareEngineering,1989:237-244.
[33] Miller D R. Exponential order statistic models of software reliability growth [J].IEEE Transaction on Software Engineering,1986,12(1):12-24
[34] John Musa.软件可靠性工程[M].北京:机械工业出版社,2003
[35] Everett W. Software component reliability analysis[C]. Proc. Symp. ApplicationSpecific systems and sonftware engineering technology (ASSET99),1999:204-211
[36] Singh H, CortellessaV,Cukic B. A Bayesian approach to reliability predictionand assessment of component based systems[C]. Proc.12th internationalSymp.software reliability engineering(ISSRE2001),2001:12-21
[37]刘斌.软件可靠性测试研究[D]:[博士后研究工作报告].北京:北京航空航天大学,2002
[38]李秋英,姜梦岑.软件可靠性验证测试最小测试量的必要条件[J].北京航空航天大学学报,2010,36(2):239-247
[39] Thierry L, Thierry S, Guilhem P.Formal Methods in Safety-Critical RailwaySystems[C]. l0th Brasilian Symposium on Formal Methods,August31,2007
[40] Myer G. The Art of Software Testing [M]. Wiley,1989
[41] IEEE. IEEE Standard Glossary[S].1983
[42] R.Mandl. Orthogonal Latin Squares: An Application of Experimental Design tocompiler testing communications of the ACM.1985,28(10):1054-1058
[43] D.Cohen et al. The AETG system: An approach to testing based oncombinatorial design [J]. IEEE Trans. on software engineering.1997.23(7):437-444
[44] Y.Lei, K.Tai. A Test Generation Strategy for Pairwise testing[R]. TechnicalReport TR-2001-03Dept. of Computer Science,2001
[45] K.Tai,Y.Lei. A Test Generation Strategy for Pairwise Testing [J]. IEEETransaction on Software Engineering,2002,28(1):109-111
[46] M.Cohen, C.Colbouns, P.Gibbons. Constructing Test Suites For InteractionTesting[C]. Proceedings of the Intl. Conf. on software Engineering (ICSE2003).2003:38-48
[47] M.Roper. Computer-Aided Software Testing using Genetic Algorithms[C]. The10th International Software Quality Week,1997
[48] G.McGraw, C.Michael, M.Schatz. Generating software test data by evolution[R].Technical Report RSTR-018-97-01.RST Corporation Sterling,1998
[49] J.Wegener, A.Baresel, H.Sthamer. Evolutionary Test Environment for AutomaticStructural Testing Information and Software Technology Special Issue on SoftwareEng[J]. Using Metaheuristic Innovative Algorithms.2001,43(14):841-854
[50]史亮.测试数据自动生成技术研究[D]:[博士学位论文].南京:东南大学计算机软件与理论,2006
[51] J.b.Goodenough S.L.G. Toward a theory of test data selection[J]. IEEE Trans onSoftware Engineering.1975, SE-(2)
[52]支定庸.航天型号软件测试回顾与展望[R]
[53] A.Dalal, S.R.; Horgan, J.R.; Kettenring, J.R.Reliable software andcommunication: software quality, reliability, and safety[C]. Proceedings15thInternational Conference on Software Engineering.1993:425-435
[54] T. Rivers and M. A. Vouk. An empirical evaluation of testing efficiency duringnon-operational testing[C]. Proceedings of the4th Software Engineering ResearchForum,1995:111-120
[55] M. A. Vouk. Using reliability models during testing with non-operationalprofile[C]. In Proc. of2nd Bellcore/Purdue Symposium on Issues in SoftwareReliability Estimation,1993:103-110
[56] Y. K. Malaiya, N. Karunanithi, and P. Verma. Predictability of softwarereliability models[C]. IEEE Trans. Reliability,1992,41:539-546
[57] Y. K. Malaiya, N. Li, and J. Bieman et al.. The relationship between testcoverage and reliability[C]. Proc. Int. Symp. Software Reliability Engineering,1994:186-195
[58] Malaiya, Y.K.; Li, M.N.; Bieman, J.M.; Karcich, R. Software reliability growthwith test coverage [J]. IEEE Transactions on Reliability,2002,51(4):420-426
[59] Hoang Pham. System software Reliability [M]. Springer,2006
[60] Hoang Pham, Xuemei Zhang. NHPP software reliability and cost models withtesting coverage[J]. European Journal of Operational Research,2003,145(2):443-454
[61] H. Pham, L. Nordmann, X. Zhang. A General imperfect software debuggingmodel with S-shaped fault detection rate[J]. IEEE Transactions on Reliability,1999,48(2):169-175
[62] H. Pham. Software Reliability and Testing[J]. IEEE Computer Society Press,Silver Spring, MD,1995
[63] H. Pham. Software reliability assessment: Imperfect debugging and multiplefault types in software development[J],EG&G-RAAM-10737, Idaho NationalLaboratory,1993
[64] H. Pham, X. Zhang. An NHPP software reliability model and its comparison[J].International Journal of Reliability, Quality and Safety Engineering,1997,4(3):269-282
[65] H. Pham, X. Zhang. A software cost model with warranty and risk costs[J].IEEE Transactions on Computers,1999,48(1):71-75
[66] H. Pham, Software Reliability[M], Springer, Berlin,2000.
[67] H. Pham, X. Zhang. Software release policies with gain in reliability justifyingthe costs.[] Annals of Software Engineering8(1999)147-166
[68] H. Pham, Software reliability, in: J.G. Webster (Ed.), Wiley Encyclopedia ofElectrical and Electronics Engineering[J], Wiley, New York,2000
[69] H. Pham, H. Wang. Software reliability and cost modeling by a quasi renewalprocess[J]. IEEE Transactions on Systems Man, and Cybernetics,2001,31(6):623-631
[70] S. Yamada, M. Onha, S. Osaki. S-shaped reliability growth modeling forsoftware error detection[J]. IEEE Transactions on Reliability,1983,12:475-484.
[71] S. Yamada, H. Ohtera, M. Ohba. Testing-domain dependent software reliabilitymodel[J]. Computers and Mathematics with Applications,1992,24:79-86
[72] S. Yamada, H. Ohtera, H. Narihisa. Software reliability growth models withtesting effort[J]. IEEE Transactions on Reliability,1986,4:19-23
[73] S. Yamada. Software quality/reliability measurement and assessment: Softwarereliability growth models and data analysis[J]. Journal of Information Processing,1991,14(3):254-266
[74] S. Yamada, K. Tokuno, S. Osaki. Imperfect debugging models with faultintroduction rate for software reliability assessment[J]. International Journal ofSystems Science,1992,23(12)
[75] S. Yamada, S. Osaki. Optimal software release policies with simultaneous costand reliability criteria[J]. European Journal of Operational Research,1987,31(1):46-51
[76] Yamamoto, T., Inoue, S.,&Yamada, S.. A software reliability growth modelwith testing-coverage maturity process[C]. Proceedings of10th international societyof science and applied technologies,2004:299-303
[77] Fujiwara, T., Inoue, S.,&Yamada, S. A software reliability growth model withmodule composition[C]. Proceedings of11th international society of science andapplied technologies2005:266-270.
[78] Joong-Yang Park, Gyemin Lee, Jae Heung Park. A class of coverage growthfunctions and its practical application[J]. Journal of the Korean Statistical Society,2008,37(3):241-247
[79] Xia Cai; Lyu, M. R..Software Reliability Modeling with Test Coverage:Experimentation and Measurement with A Fault-Tolerant Software Project[C] The18th IEEE International Symposium on Software Reliability Engineering,2007:17-26
[80] Weyuker.E.J. Axiomatiozing software test data adequacy[J]. IEEE.Transactionson software engineering.1986, Vol.SE-12.1128-1138
[81]朱鸿,金凌紫.软件质量保障与测试[M].科学出版社,1997
[82]朱鸿.软件可靠性估计与计算复杂性的关系浅析[J].软件学报.1998,
[83]李秋英,陆民燕,阮镰.软件可靠性测试充分性问题的理论研究[J].北京航空航天大学学报,2003,29(4):312-316
[84]李秋英.软件可靠性测试充分性的理论研究[D]:[博士学位论文].北京:北京航空航天大学,2004
[85] Villemeur A. Reliability, Availability, Maintainability and Safety Assessment[M].John Willey&Sons,1991
[86]樊林波.软件可靠性与安全性的相互关系及其转换规律研究[D]:[博士学位论文].贵州:贵州大学,2009
[87] GJB/Z102-97软件可靠性和安全性设计准则[S]
[88] IEEE Recommended Practice on Software Reliability[S]. IEEE Std1633-2008.
[89]刘宏伟,杨孝宗,等.非齐次泊松过程类软件可靠性增长模型[J].同济大学学报（自然科学版）,2004,32(8):1071-1074
[90] Littlewood B. A software Reliability model for modular program structure[J].IEEE Transactions on Reliability,1979,28(2):241-246
[91] Mayer J, Beydeda S. Message of the program chairs of STEV’07[C]. Proc. ofthe7th Int’l Conf. on Quality Software (QSIC2007),2007,370-371
[92] Pasquini A., Crespo,A.N.&Matrella,P. Sensitivity of reliability growth modelsto operational profile errors vs testing accuracy[J]. IEEE Transactions onReliability,1996,45:531-540
[93] Gokhale S S,Philip,T., Marinos,P.N.,&Trivedi K S.Unification of finite failurenon-honmogeneous Poisson models through test coverage[C].In Proceeding of7thIEEE international symposium on software reliability engineering,1996,299-307
[94] Gokhale S S,Trivedi K S. A time/structure based software reliabilitymodel[J].Annals of Software Engineering,1999,8(1):85-121
[95] Gokhale S S,M ullen R E. From test count to code coverage using the lognormalfailure rate[C]. Proc of the15th IEEE Int Syrup on Software Reliability Engineering.IEEE Computer Society,2004:295-305
[96] Grottke M. A vector Markov model for structural coverage growth and thenumber of failure occurrences[C]. Proc of the13th IEEE Int Symp on SoftwareReliability Engineering,IEEE Computer Society,2002:304-315
[97] Sedigh-Ali,S.,Ghafoor,A.,&Paul,R.A. Temporal modeling of software testcoverage[C]. In Proceeding of IEEE26th annual computer software and applocationsconference,2002,823-828
[98] Yamamoto,T., Inoue,S.,&Yamada,S. A software reliability growth model withtesting-coverage maturity process[C]. In Proceeding of10th international society ofscience and applied technologies,2004,299-303
[99]李海峰,李秋英,陆民燕.基于Logistic测试覆盖率函数的软件可靠性建模研究[J].计算机研究与发展,2011,48(2):232-240
[100] Park J Y,Lee G,Park J H. A class of coverage growth functions and its practicalapplication[J].Journal of the Korean Statistical Society,2008,37(3):241-247
[101]游达章.最小二乘法在威布尔分布的可靠性评估[J].湖北工业大学学报,2009,24(4):34-36
[102]王青,伍书剑,李明树.软件缺陷预测技术[J].软件学报,2008,19(7):1565-1580
[103]王德胜,宫云战.论软件缺陷[J].计算机应用研究,2008,25(12):3531-3533
[104] Akiyama F. An Example of software system debugging[J].InformationProcessing.1974,71:353-379
[105] Lipow M. Number of fault per line of code[J]. IEEE Transactions softwareengineering.1982,8(4):437-439
[106] Gaffney J R. Estimating the number of faults in code[J]. IEEE Transactionssoftware engineering.1984,10(4):437-439
[107] P.piwowarski, M.ohba J C. Coverage measurement experience during functiontest[C]. Proceeding of15th International Conference of Software Engineering.1993:287-300
[108]解敏,任平安,马建峰.改进的软件测试预测模型[J].计算机工程与应用,2010,46(31):64-67
[109]任卫军,贺昱曜,张卫钢.基于Gompertz模型的软件质量与测试过程评估[J].计算机工程与应用,2008,44(9):87-89
[110] Fenton NE, Martain N, Willam M, Peter H, Lukrad R, Paul K. Predictingsoftware defects in varying development lifecycles using Bayesian nets[J].Information and Software Technology,2007,49(1):32-43
[111] Mills H. On the statistical validation of computer programs[R]. TechnicalReport,FSC-72-6015,IBM Federal Systems Division,1972
[112]朱永春,徐红.一种基于历史数据的软件缺陷预测方法改进[J].北京航空航天大学学报,2003,29(10):947-950
[113] Cai X, Lyu MR. Software reliability modeling with test coverage:Experimentation and measurement with a fault-tolerant software project[C]. Proc. ofthe18th IEEE Int’l Symp. on Software Reliability Engineering.2007,17-26
[114]周瑞,徐仁佐,杨晓青.基于测试覆盖的NHPP模型[J].小型微型计算机系统.1999,20(8):587-592
[115]袁春如,廖泰安,贺红卫.基于测试覆盖的嵌入式软件可靠性评估[J].计算机工程与设计,2009,30(9):2198-2200
[116] Shuanqi Wang, Yumei Wu, Minyan Lu. Discrete nonhomogeneous poissonprocess software reliability growth Models based on test coverage[J]. Quality andReliability Engineering International,2012
[117]谈维新.考虑测试效率的软件可靠性模型[D].南京:南京邮电大学,2011
[118]丁晓光.基于测试覆盖的软件可靠性模型[D].南京:南京邮电大学,2008
[119]胡海宏.软件可靠性模型与软件最优发布问题的研究[D].南京:南京邮电大学,2011
[120]顾林君.一种错误发现率为时间函数的软件可靠性模型[D].南京:南京邮电大学,2009
[121]郑垒.考虑非理性排错过程的软件可靠性模型研究[D].南京:南京邮电大学,2011
[122]周涛.航天型号软件测试[M].宇航出版社,1999
[123]郑人杰.计算机软件测试技术[M].北京:清华大学出版社,1990
[124]李秋英,刘斌,阮镰.灰盒测试方法在软件可靠性测试中的应用[J].航空学报,2002,23(5):455-458
[125]蔡开元,李永超,景涛,白成刚.软件测试的控制论方法[J].航空学报,2002,23(5):448-454
[126] He YT, Hecht H, Paul RA. Measuring and assessing software test processesusing test data[C]. Proc. of the5th IEEE Int’l Symp. on High Assurance SystemsEngineering (HASE2000).2000.259-264
[127]安金霞,王国庆,李树芳,朱纪洪.基于多维度覆盖率的软件测试动态评价方法[J].软件学报,2010,21(9):2135-2147
[128] IEEE Computer Society, IEEE Standard Classification for SoftwareAnomalies1993[S]. IEEE Std1044-1993
[2] Knight J C.Safety Critical Systems-Challenges and Directions [C]. Proc.of Futureof Software Engineering(FOSE07).New York:IEEE Computer Society,2007
[3] Mike F. Advances in Safety Critical Systems[R].Report for Results andAchievements from the DTI/EPSRC R&D Programme in Safety Criticalsystems,1997
[4]杨启亮,邢建春,王平.安全关键系统及其软件方法[J].计算机应用与软件,2011,28(2):129-138
[5] GJB/Z1391-2006故障模式、影响及危害性分析指南[S].
[6] GJB/Z142-2004军用软件安全性分析指南[S].
[7]刘彬彬.软件FMEA与FTA综合分析方法研究[D]:[硕士学位论文].北京:北京航空航天大学,2008
[8]宋晓秋.软件安全性分析的Petri网方法[J].质量与可靠性,1998,(1):33-36
[9]王金波.安全关键软件的测试与评估过程模型研究[D]:[博士学位论文].北京:中国科学院研究生院,2008
[10]郦萌.软件安全性3M评价中评价因子修正的研究[J].同济大学学报,2002,10:28-29
[11]郦萌.计算机软件的可靠性[M].北京:国防工业出版社,1988
[12]郦萌.安全性苛求系统中关于软件安全性评价的研究[J].计算机工程与科学,2002,24(2):59-61
[13]黎忠文,熊光泽.形式化方法与安全关键系统[J].计算机应用,2000,20(9):36-38
[14]黎忠文,陈亮,熊光泽.基于防危核/壳机制的安全关键硬实时系统响应时间分析[J].电子学报,2006(4):27-29
[15]杨仕平,桑楠,吴勇新等.基于多级关键度的高可信安全关键系统[J].系统工程与电子技术,2004,26(2):277-282
[16]杨仕平,桑楠,熊光泽.安全关键软件的防危性测评技术研究[J].计算机学报,2004,27(4):442-450
[17]覃志东.高可信软件可靠性和防危性测试与评价理论研究[D]:[博士学位论文].成都:电子科技大学,2005
[18]张德平,聂长海,徐宝文.基于Markov决策过程用交叉熵方法优化软件测试[J].软件学报,2008,19(10):2270-2279
[19] John Musa. Software Reliability Engineering [M]. McGraw-Hill,1999
[20]黄锡滋.软件可靠性、安全性与质量保证[M].北京:电子工业出版社,2002
[21]蔡开元.软件可靠性工程基础[M].清华大学出版社,1995
[22]蔡开元.关于软件可靠性和软件控制论的若干认识[J].中国科学基金.2004,年第1期:201-204
[23]徐仁佐.软件可靠性工程[M].北京:清华大学出版社,2007
[24] Jelinski Z, Moranda PB. Software Reliability Research[C]. Proceedings of theStatistical Methods for the Evaluation of Computer System Performance, AcademicPress,1972:485-502
[25] Schneidewind N F. Analysis of Error Processes in Computer Software[J].Sigplan Note,1975,10(6):337-346
[26] Goel A L, Okumoto K.Time-Dependent Error-Detection Rate Model forSoftware and Other Performance Measures [J]. IEEE Transactions on Reliability,1979,28(3):206-211
[27] Musa J D. Software Reliability: Handbook of software engineering[M].1984:392-412
[28] M. Ohba, S. Yamada. S-shaped software reliability growth models[C].Proceedings of the4th International Conference on Reliability and Maintainability,1984:430-436
[29] M. Ohba. Software reliability analysis models [J]. IBM Journal of ResearchDevelopment,1984,28:428-443
[30] M. Ohba. Inflection S-shaped software reliability growth models, in: S. Osaki, Y.Hatoyama (Eds.), Stochastic Models in Reliability Theory, Springer, Berlin,1984,pp.144-162.
[31] D. Musa and K. Okumoto. A Logarithmic PoissonExecution Time Model ForSoftware Reliability Measurement[C]. In Proc.7th International Conference onSoftware Engineering,1983:230-237
[32] M. Ohba, X.M. Chou. Does imperfect debugging affect software reliabilitygrowth[C]. Proceedings of the11th IEEE International Conference on SoftwareEngineering,1989:237-244.
[33] Miller D R. Exponential order statistic models of software reliability growth [J].IEEE Transaction on Software Engineering,1986,12(1):12-24
[34] John Musa.软件可靠性工程[M].北京:机械工业出版社,2003
[35] Everett W. Software component reliability analysis[C]. Proc. Symp. ApplicationSpecific systems and sonftware engineering technology (ASSET99),1999:204-211
[36] Singh H, CortellessaV,Cukic B. A Bayesian approach to reliability predictionand assessment of component based systems[C]. Proc.12th internationalSymp.software reliability engineering(ISSRE2001),2001:12-21
[37]刘斌.软件可靠性测试研究[D]:[博士后研究工作报告].北京:北京航空航天大学,2002
[38]李秋英,姜梦岑.软件可靠性验证测试最小测试量的必要条件[J].北京航空航天大学学报,2010,36(2):239-247
[39] Thierry L, Thierry S, Guilhem P.Formal Methods in Safety-Critical RailwaySystems[C]. l0th Brasilian Symposium on Formal Methods,August31,2007
[40] Myer G. The Art of Software Testing [M]. Wiley,1989
[41] IEEE. IEEE Standard Glossary[S].1983
[42] R.Mandl. Orthogonal Latin Squares: An Application of Experimental Design tocompiler testing communications of the ACM.1985,28(10):1054-1058
[43] D.Cohen et al. The AETG system: An approach to testing based oncombinatorial design [J]. IEEE Trans. on software engineering.1997.23(7):437-444
[44] Y.Lei, K.Tai. A Test Generation Strategy for Pairwise testing[R]. TechnicalReport TR-2001-03Dept. of Computer Science,2001
[45] K.Tai,Y.Lei. A Test Generation Strategy for Pairwise Testing [J]. IEEETransaction on Software Engineering,2002,28(1):109-111
[46] M.Cohen, C.Colbouns, P.Gibbons. Constructing Test Suites For InteractionTesting[C]. Proceedings of the Intl. Conf. on software Engineering (ICSE2003).2003:38-48
[47] M.Roper. Computer-Aided Software Testing using Genetic Algorithms[C]. The10th International Software Quality Week,1997
[48] G.McGraw, C.Michael, M.Schatz. Generating software test data by evolution[R].Technical Report RSTR-018-97-01.RST Corporation Sterling,1998
[49] J.Wegener, A.Baresel, H.Sthamer. Evolutionary Test Environment for AutomaticStructural Testing Information and Software Technology Special Issue on SoftwareEng[J]. Using Metaheuristic Innovative Algorithms.2001,43(14):841-854
[50]史亮.测试数据自动生成技术研究[D]:[博士学位论文].南京:东南大学计算机软件与理论,2006
[51] J.b.Goodenough S.L.G. Toward a theory of test data selection[J]. IEEE Trans onSoftware Engineering.1975, SE-(2)
[52]支定庸.航天型号软件测试回顾与展望[R]
[53] A.Dalal, S.R.; Horgan, J.R.; Kettenring, J.R.Reliable software andcommunication: software quality, reliability, and safety[C]. Proceedings15thInternational Conference on Software Engineering.1993:425-435
[54] T. Rivers and M. A. Vouk. An empirical evaluation of testing efficiency duringnon-operational testing[C]. Proceedings of the4th Software Engineering ResearchForum,1995:111-120
[55] M. A. Vouk. Using reliability models during testing with non-operationalprofile[C]. In Proc. of2nd Bellcore/Purdue Symposium on Issues in SoftwareReliability Estimation,1993:103-110
[56] Y. K. Malaiya, N. Karunanithi, and P. Verma. Predictability of softwarereliability models[C]. IEEE Trans. Reliability,1992,41:539-546
[57] Y. K. Malaiya, N. Li, and J. Bieman et al.. The relationship between testcoverage and reliability[C]. Proc. Int. Symp. Software Reliability Engineering,1994:186-195
[58] Malaiya, Y.K.; Li, M.N.; Bieman, J.M.; Karcich, R. Software reliability growthwith test coverage [J]. IEEE Transactions on Reliability,2002,51(4):420-426
[59] Hoang Pham. System software Reliability [M]. Springer,2006
[60] Hoang Pham, Xuemei Zhang. NHPP software reliability and cost models withtesting coverage[J]. European Journal of Operational Research,2003,145(2):443-454
[61] H. Pham, L. Nordmann, X. Zhang. A General imperfect software debuggingmodel with S-shaped fault detection rate[J]. IEEE Transactions on Reliability,1999,48(2):169-175
[62] H. Pham. Software Reliability and Testing[J]. IEEE Computer Society Press,Silver Spring, MD,1995
[63] H. Pham. Software reliability assessment: Imperfect debugging and multiplefault types in software development[J],EG&G-RAAM-10737, Idaho NationalLaboratory,1993
[64] H. Pham, X. Zhang. An NHPP software reliability model and its comparison[J].International Journal of Reliability, Quality and Safety Engineering,1997,4(3):269-282
[65] H. Pham, X. Zhang. A software cost model with warranty and risk costs[J].IEEE Transactions on Computers,1999,48(1):71-75
[66] H. Pham, Software Reliability[M], Springer, Berlin,2000.
[67] H. Pham, X. Zhang. Software release policies with gain in reliability justifyingthe costs.[] Annals of Software Engineering8(1999)147-166
[68] H. Pham, Software reliability, in: J.G. Webster (Ed.), Wiley Encyclopedia ofElectrical and Electronics Engineering[J], Wiley, New York,2000
[69] H. Pham, H. Wang. Software reliability and cost modeling by a quasi renewalprocess[J]. IEEE Transactions on Systems Man, and Cybernetics,2001,31(6):623-631
[70] S. Yamada, M. Onha, S. Osaki. S-shaped reliability growth modeling forsoftware error detection[J]. IEEE Transactions on Reliability,1983,12:475-484.
[71] S. Yamada, H. Ohtera, M. Ohba. Testing-domain dependent software reliabilitymodel[J]. Computers and Mathematics with Applications,1992,24:79-86
[72] S. Yamada, H. Ohtera, H. Narihisa. Software reliability growth models withtesting effort[J]. IEEE Transactions on Reliability,1986,4:19-23
[73] S. Yamada. Software quality/reliability measurement and assessment: Softwarereliability growth models and data analysis[J]. Journal of Information Processing,1991,14(3):254-266
[74] S. Yamada, K. Tokuno, S. Osaki. Imperfect debugging models with faultintroduction rate for software reliability assessment[J]. International Journal ofSystems Science,1992,23(12)
[75] S. Yamada, S. Osaki. Optimal software release policies with simultaneous costand reliability criteria[J]. European Journal of Operational Research,1987,31(1):46-51
[76] Yamamoto, T., Inoue, S.,&Yamada, S.. A software reliability growth modelwith testing-coverage maturity process[C]. Proceedings of10th international societyof science and applied technologies,2004:299-303
[77] Fujiwara, T., Inoue, S.,&Yamada, S. A software reliability growth model withmodule composition[C]. Proceedings of11th international society of science andapplied technologies2005:266-270.
[78] Joong-Yang Park, Gyemin Lee, Jae Heung Park. A class of coverage growthfunctions and its practical application[J]. Journal of the Korean Statistical Society,2008,37(3):241-247
[79] Xia Cai; Lyu, M. R..Software Reliability Modeling with Test Coverage:Experimentation and Measurement with A Fault-Tolerant Software Project[C] The18th IEEE International Symposium on Software Reliability Engineering,2007:17-26
[80] Weyuker.E.J. Axiomatiozing software test data adequacy[J]. IEEE.Transactionson software engineering.1986, Vol.SE-12.1128-1138
[81]朱鸿,金凌紫.软件质量保障与测试[M].科学出版社,1997
[82]朱鸿.软件可靠性估计与计算复杂性的关系浅析[J].软件学报.1998,
[83]李秋英,陆民燕,阮镰.软件可靠性测试充分性问题的理论研究[J].北京航空航天大学学报,2003,29(4):312-316
[84]李秋英.软件可靠性测试充分性的理论研究[D]:[博士学位论文].北京:北京航空航天大学,2004
[85] Villemeur A. Reliability, Availability, Maintainability and Safety Assessment[M].John Willey&Sons,1991
[86]樊林波.软件可靠性与安全性的相互关系及其转换规律研究[D]:[博士学位论文].贵州:贵州大学,2009
[87] GJB/Z102-97软件可靠性和安全性设计准则[S]
[88] IEEE Recommended Practice on Software Reliability[S]. IEEE Std1633-2008.
[89]刘宏伟,杨孝宗,等.非齐次泊松过程类软件可靠性增长模型[J].同济大学学报（自然科学版）,2004,32(8):1071-1074
[90] Littlewood B. A software Reliability model for modular program structure[J].IEEE Transactions on Reliability,1979,28(2):241-246
[91] Mayer J, Beydeda S. Message of the program chairs of STEV’07[C]. Proc. ofthe7th Int’l Conf. on Quality Software (QSIC2007),2007,370-371
[92] Pasquini A., Crespo,A.N.&Matrella,P. Sensitivity of reliability growth modelsto operational profile errors vs testing accuracy[J]. IEEE Transactions onReliability,1996,45:531-540
[93] Gokhale S S,Philip,T., Marinos,P.N.,&Trivedi K S.Unification of finite failurenon-honmogeneous Poisson models through test coverage[C].In Proceeding of7thIEEE international symposium on software reliability engineering,1996,299-307
[94] Gokhale S S,Trivedi K S. A time/structure based software reliabilitymodel[J].Annals of Software Engineering,1999,8(1):85-121
[95] Gokhale S S,M ullen R E. From test count to code coverage using the lognormalfailure rate[C]. Proc of the15th IEEE Int Syrup on Software Reliability Engineering.IEEE Computer Society,2004:295-305
[96] Grottke M. A vector Markov model for structural coverage growth and thenumber of failure occurrences[C]. Proc of the13th IEEE Int Symp on SoftwareReliability Engineering,IEEE Computer Society,2002:304-315
[97] Sedigh-Ali,S.,Ghafoor,A.,&Paul,R.A. Temporal modeling of software testcoverage[C]. In Proceeding of IEEE26th annual computer software and applocationsconference,2002,823-828
[98] Yamamoto,T., Inoue,S.,&Yamada,S. A software reliability growth model withtesting-coverage maturity process[C]. In Proceeding of10th international society ofscience and applied technologies,2004,299-303
[99]李海峰,李秋英,陆民燕.基于Logistic测试覆盖率函数的软件可靠性建模研究[J].计算机研究与发展,2011,48(2):232-240
[100] Park J Y,Lee G,Park J H. A class of coverage growth functions and its practicalapplication[J].Journal of the Korean Statistical Society,2008,37(3):241-247
[101]游达章.最小二乘法在威布尔分布的可靠性评估[J].湖北工业大学学报,2009,24(4):34-36
[102]王青,伍书剑,李明树.软件缺陷预测技术[J].软件学报,2008,19(7):1565-1580
[103]王德胜,宫云战.论软件缺陷[J].计算机应用研究,2008,25(12):3531-3533
[104] Akiyama F. An Example of software system debugging[J].InformationProcessing.1974,71:353-379
[105] Lipow M. Number of fault per line of code[J]. IEEE Transactions softwareengineering.1982,8(4):437-439
[106] Gaffney J R. Estimating the number of faults in code[J]. IEEE Transactionssoftware engineering.1984,10(4):437-439
[107] P.piwowarski, M.ohba J C. Coverage measurement experience during functiontest[C]. Proceeding of15th International Conference of Software Engineering.1993:287-300
[108]解敏,任平安,马建峰.改进的软件测试预测模型[J].计算机工程与应用,2010,46(31):64-67
[109]任卫军,贺昱曜,张卫钢.基于Gompertz模型的软件质量与测试过程评估[J].计算机工程与应用,2008,44(9):87-89
[110] Fenton NE, Martain N, Willam M, Peter H, Lukrad R, Paul K. Predictingsoftware defects in varying development lifecycles using Bayesian nets[J].Information and Software Technology,2007,49(1):32-43
[111] Mills H. On the statistical validation of computer programs[R]. TechnicalReport,FSC-72-6015,IBM Federal Systems Division,1972
[112]朱永春,徐红.一种基于历史数据的软件缺陷预测方法改进[J].北京航空航天大学学报,2003,29(10):947-950
[113] Cai X, Lyu MR. Software reliability modeling with test coverage:Experimentation and measurement with a fault-tolerant software project[C]. Proc. ofthe18th IEEE Int’l Symp. on Software Reliability Engineering.2007,17-26
[114]周瑞,徐仁佐,杨晓青.基于测试覆盖的NHPP模型[J].小型微型计算机系统.1999,20(8):587-592
[115]袁春如,廖泰安,贺红卫.基于测试覆盖的嵌入式软件可靠性评估[J].计算机工程与设计,2009,30(9):2198-2200
[116] Shuanqi Wang, Yumei Wu, Minyan Lu. Discrete nonhomogeneous poissonprocess software reliability growth Models based on test coverage[J]. Quality andReliability Engineering International,2012
[117]谈维新.考虑测试效率的软件可靠性模型[D].南京:南京邮电大学,2011
[118]丁晓光.基于测试覆盖的软件可靠性模型[D].南京:南京邮电大学,2008
[119]胡海宏.软件可靠性模型与软件最优发布问题的研究[D].南京:南京邮电大学,2011
[120]顾林君.一种错误发现率为时间函数的软件可靠性模型[D].南京:南京邮电大学,2009
[121]郑垒.考虑非理性排错过程的软件可靠性模型研究[D].南京:南京邮电大学,2011
[122]周涛.航天型号软件测试[M].宇航出版社,1999
[123]郑人杰.计算机软件测试技术[M].北京:清华大学出版社,1990
[124]李秋英,刘斌,阮镰.灰盒测试方法在软件可靠性测试中的应用[J].航空学报,2002,23(5):455-458
[125]蔡开元,李永超,景涛,白成刚.软件测试的控制论方法[J].航空学报,2002,23(5):448-454
[126] He YT, Hecht H, Paul RA. Measuring and assessing software test processesusing test data[C]. Proc. of the5th IEEE Int’l Symp. on High Assurance SystemsEngineering (HASE2000).2000.259-264
[127]安金霞,王国庆,李树芳,朱纪洪.基于多维度覆盖率的软件测试动态评价方法[J].软件学报,2010,21(9):2135-2147
[128] IEEE Computer Society, IEEE Standard Classification for SoftwareAnomalies1993[S]. IEEE Std1044-1993
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |