流特征下的在线知识发现研究
|
摘要
相对于静态特征空间下的在线知识发现算法,动态特征空间下的在线知识发现算法的研究并没有引起足够的关注。数据特征空间的动态性是指在学习算法开始之前,问题的特征空间不是或不能事先给定,而是随时间动态变化。因而,数据特征空间的动态性对传统的静态特征空间下的在线知识发现算法提出诸多新问题和新挑战。为探索动态特征空间下的在线知识发现问题,本文提出了流特征(Streaming Features)的概念来建模特征空间的高维度和动态性而无需在学习算法开始前给定问题的全部特征空间。流特征定义为样本(示例)空间不变的条件下,问题的特征空间中的特征随时间逐个流入,且每个新流入的特征被立即在线处理。以流特征概念为核心,本文开展了流特征下的在线知识发现算法研究,主要取得了如下创新性成果：
(1)针对流特征下的在线特征选择问题,本文提出了两种新颖的在线特征选择算法OSFS(Online Streaming Feature Selection)和Fast-OSFS。实验表明,OSFS和Fast-OSFS不仅能很好的处理训练数据特征维数信息预先未知条件下的特征选择问题,而且比已有的在线特征选择算法用较小的特征子集获得更高的分类精度。
(2)针对流特征下的局部因果关系发现问题,本文提出了流特征下的局部因果关系发现算法(CDFSF, Causal Discovery From Streaming Features)。为提高CDFSF的效率,在忠实性因果贝叶斯网络条件下,利用父结点(原因)和子结点(结果)之间的对称性,提出了S-CDFSF (Symmetrical CDFSF)算法。与已有的局部因果发现算法(算法开始前需要训练数据的整个特征空间)的实验比较结果表明,我们提出的算法可以有效处理流特征下的局部因果发现问题。
(3)针对流特征下的新兴模式(Emerging Patterns,以下简称EP模式)的挖掘问题,我们的研究工作分为以下两个方面。首先,基于因果贝叶斯网络中的因果相关性,本文提出了两种EP模式分类器：CE-EP和MB-EP来解决静态高维特征空间下的EP模式挖掘问题,CE (direct Causes and direct Effects)表示直接因果,而MB (Markov Blanket)表示马尔科夫毯。进而,针对CE-EP和MB-EP分类器不能处理动态高维特征空间下的EP模式挖掘问题,本文进一步提出了流特征下的在线EP模式挖掘算法,EPSF(mining Emerging Patterns using Streaming Feature selection)算法。实验表明,CE-EP和MB-EP可以有效解决静态高维特征空间下的EP模式挖掘问题,而EPSF算法不仅能够处理静态高维数据下的EP模式挖掘问题,而且可以有效的处理动态高维特征空间下的EP模式挖掘问题。
(4)以火星图片上的陨石坑自动检测为应用研究,本文提出了流特征下的在线陨石坑检测算法,实验验证了本文所提出的流特征下的在线学习算法(包括OSFS、Fast-OSFS、 CE-EP和EPSF算法)在实际应用问题处理的有效性。
Compared to the traditional online knowledge discovery with a static feature space, online knowledge discovery with a dynamic feature space has not attracted much attention. A feature space is dynamic when not all features are available before learning begins or when the feature space changes dynamically over time. Therefore, a dynamic feature space might make the feature space of traning data become high dimensional and uncertain, which is challenging for traditional online knowledge discovery algorithms.In order to explore online knowledge discovery with a dynamic feature space, we define the concept of streaming features to model high yet dynamic feature dimensions without the necessity of a whole feature space before learning starts. With streaming features, the features flow in one by one and each feature is online processed upon its arrival while the number of instances is fixed. With streaming features, in this dissertation, we study online knowledge discovery with a dynamic feature space and our main contributions are as follows.
(1) We propose a new online feature selection framework for applications with streaming features where the knowledge of the full feature space is unknown in advance. With this framework, we present a novel Online Streaming Feature Selection (OSFS) method to select strongly relevant and non-redundant features on the fly. An efficient Fast-OSFS algorithm is proposed to improve feature selection performance. Experimental results demonstrate that the algorithms achieve better compactness and higher prediction accuracy than existing streaming feature selection algorithms.
(2) We study a new research problem of discovery of local causal relationships in the context of streaming features. With a causal Bayesian network to represent causal relationships, we propose a novel algorithm, called CDFSF (Causal Discovery From Streaming Features) to discover local causal relationships from streaming features. In order to improve the efficiency of CDFSF, using the symmetry properties between parents (causes) and children (effects) in a faithful Bayesian network, we present a variant of CDFSF, S-CDFSF (Symmetrical CDFSF). Experimental results validate our algorithms in comparison with the existing algorithms for causal relationship discovery.
(3) Mining emerging patterns (EP for short) is a challenging issue in the context of streaming features. To address this challenging problem, we propose two EP miners for mining emerging patterns from a high yet static feature space, called CE-EP and MB-EP, where CE stands for direct Causes and direct Effects, and MB for Markov Blanket. To mine EPs from a high yet dynamic feature space, we present a novel streaming pattern mining technique, called EPSF (mining Emerging Patterns with Streaming Feature selection). Compared to CE-EP and MB-EP, EPSF can mine EPs from not only a high yet static feature space, but also a high yet dynamic feature space. Extensive experiments on a broad range of datasets show the effectiveness of the CE-EP MB-EP, and EPSF classifiers against other well-established methods, in terms of predictive accuracy, pattern numbers, running time, and sensitivity analysis.
(4) Also, we apply our proposed methods, including OSFS, Fast-OSFS, CE-EP, and EPSF, to a case study on automatic impact crater detection in real planetary images. Extensive studies reveal the advantages of our methods over existing streaming feature selection algorithms, crater detection methods, and well-known feature selection algorithms. Meanwhile, this case study validates our proposed methods on real-world data.
(1)针对流特征下的在线特征选择问题,本文提出了两种新颖的在线特征选择算法OSFS(Online Streaming Feature Selection)和Fast-OSFS。实验表明,OSFS和Fast-OSFS不仅能很好的处理训练数据特征维数信息预先未知条件下的特征选择问题,而且比已有的在线特征选择算法用较小的特征子集获得更高的分类精度。
(2)针对流特征下的局部因果关系发现问题,本文提出了流特征下的局部因果关系发现算法(CDFSF, Causal Discovery From Streaming Features)。为提高CDFSF的效率,在忠实性因果贝叶斯网络条件下,利用父结点(原因)和子结点(结果)之间的对称性,提出了S-CDFSF (Symmetrical CDFSF)算法。与已有的局部因果发现算法(算法开始前需要训练数据的整个特征空间)的实验比较结果表明,我们提出的算法可以有效处理流特征下的局部因果发现问题。
(3)针对流特征下的新兴模式(Emerging Patterns,以下简称EP模式)的挖掘问题,我们的研究工作分为以下两个方面。首先,基于因果贝叶斯网络中的因果相关性,本文提出了两种EP模式分类器：CE-EP和MB-EP来解决静态高维特征空间下的EP模式挖掘问题,CE (direct Causes and direct Effects)表示直接因果,而MB (Markov Blanket)表示马尔科夫毯。进而,针对CE-EP和MB-EP分类器不能处理动态高维特征空间下的EP模式挖掘问题,本文进一步提出了流特征下的在线EP模式挖掘算法,EPSF(mining Emerging Patterns using Streaming Feature selection)算法。实验表明,CE-EP和MB-EP可以有效解决静态高维特征空间下的EP模式挖掘问题,而EPSF算法不仅能够处理静态高维数据下的EP模式挖掘问题,而且可以有效的处理动态高维特征空间下的EP模式挖掘问题。
(4)以火星图片上的陨石坑自动检测为应用研究,本文提出了流特征下的在线陨石坑检测算法,实验验证了本文所提出的流特征下的在线学习算法(包括OSFS、Fast-OSFS、 CE-EP和EPSF算法)在实际应用问题处理的有效性。
Compared to the traditional online knowledge discovery with a static feature space, online knowledge discovery with a dynamic feature space has not attracted much attention. A feature space is dynamic when not all features are available before learning begins or when the feature space changes dynamically over time. Therefore, a dynamic feature space might make the feature space of traning data become high dimensional and uncertain, which is challenging for traditional online knowledge discovery algorithms.In order to explore online knowledge discovery with a dynamic feature space, we define the concept of streaming features to model high yet dynamic feature dimensions without the necessity of a whole feature space before learning starts. With streaming features, the features flow in one by one and each feature is online processed upon its arrival while the number of instances is fixed. With streaming features, in this dissertation, we study online knowledge discovery with a dynamic feature space and our main contributions are as follows.
(1) We propose a new online feature selection framework for applications with streaming features where the knowledge of the full feature space is unknown in advance. With this framework, we present a novel Online Streaming Feature Selection (OSFS) method to select strongly relevant and non-redundant features on the fly. An efficient Fast-OSFS algorithm is proposed to improve feature selection performance. Experimental results demonstrate that the algorithms achieve better compactness and higher prediction accuracy than existing streaming feature selection algorithms.
(2) We study a new research problem of discovery of local causal relationships in the context of streaming features. With a causal Bayesian network to represent causal relationships, we propose a novel algorithm, called CDFSF (Causal Discovery From Streaming Features) to discover local causal relationships from streaming features. In order to improve the efficiency of CDFSF, using the symmetry properties between parents (causes) and children (effects) in a faithful Bayesian network, we present a variant of CDFSF, S-CDFSF (Symmetrical CDFSF). Experimental results validate our algorithms in comparison with the existing algorithms for causal relationship discovery.
(3) Mining emerging patterns (EP for short) is a challenging issue in the context of streaming features. To address this challenging problem, we propose two EP miners for mining emerging patterns from a high yet static feature space, called CE-EP and MB-EP, where CE stands for direct Causes and direct Effects, and MB for Markov Blanket. To mine EPs from a high yet dynamic feature space, we present a novel streaming pattern mining technique, called EPSF (mining Emerging Patterns with Streaming Feature selection). Compared to CE-EP and MB-EP, EPSF can mine EPs from not only a high yet static feature space, but also a high yet dynamic feature space. Extensive experiments on a broad range of datasets show the effectiveness of the CE-EP MB-EP, and EPSF classifiers against other well-established methods, in terms of predictive accuracy, pattern numbers, running time, and sensitivity analysis.
(4) Also, we apply our proposed methods, including OSFS, Fast-OSFS, CE-EP, and EPSF, to a case study on automatic impact crater detection in real planetary images. Extensive studies reveal the advantages of our methods over existing streaming feature selection algorithms, crater detection methods, and well-known feature selection algorithms. Meanwhile, this case study validates our proposed methods on real-world data.
引文
[Agresti,1990] Agresti A. (1990) Categorical data analysis. New York:John Wiley and Sons.
[Akaike,1973] Akaike H. (1973) Information theory and an extension of the maximum likelihood principle. In:B. N. Petrov, F. Caki, eds. Second International Symposium on Information Theory. Budapest:AkademiaiKiado,267-281.
[Aliferisand Tsamardinos,2002] Aliferis C. F. and Tsamardinos Ⅰ. (2002) Algorithms for large-scale local causal discovery and feature selection in the presence of small sample or large causal neighborhoods. Technical Report DSL 02-08, Department of Biomedicallnformatics, Vanderbilt University.
[Aliferis et al.,2003a] Aliferis C.F., Tsamardinos I., Statnikov A., and Brown L. E. (2003) Causal explorer:a causal probabilistic network learning toolkit for biomedical discovery. METMBS'03.
[Aliferiset et al.,2003b] Aliferis C.F., Tsamardinos I., and Statnikov A. (2003) HITON:a novel markov blanket algorithm for optimal variable selection. American Medical Informatics Association (AMIA) Annual Symposium, Vancouver, British Columbia, November18-22, 2003,21-25. AMIA, Bethesda, Maryland, USA.
[Aliferis et al.,2010a] Aliferis, C. F., A. Statnikov, I. Tsamardinos, S. Mani, and X. Koutsoukos. (2010) Local causal and Markov blanket induction for causal discovery and feature selection for classification Part Ⅰ:algorithms and empirical evaluation. Journal of Machine Learning Research,11,171-234.
[Aliferis et al.,2010b] Aliferis C. F, Statnikov A., Tsamardinos I., Mani S., and Koutsoukos X. (2010) Local causal and Markov blanketinduction for causal discovery and feature selection forclassification part Ⅱ:analysis and extensions. Journal of Machine Learning Research,11,235-284.
[Aphinyanaphongs et al.,2006] Aphinyanaphongs Y., Statnikov A. and Aliferis C. F. (2006) A comparison of citation metrics to machine learning filters for the identification of high quality medline documents. J. Am. Med. Inform. Assoc.,13(4):446-455.
[Baralis et al.,2008] Baralis E., Chiusano S., and Garza P. (2008) A Lazy Approach to Associative Classification. IEEE Transactions on Knowledge and Data Engineering, 20(2):156-171.
[Brown et al.,2012] Brown G., Pocock A., Zhao M., and Luj'an M. (2012) Conditional likelihood maximisation:a unifying framework for information theoretic feature selection. Journal of Machine Learning Research,13:27-66.
[Bayardo,1998] Bayardo R. J. (1998) Efficiently mining long patterns from databases. SIGMOD'98,85-93.
[Bailey et al.,2002] Bailey J., Manoukian T., and Ramamohanarao K. (2002) Fast algorithms for mining emerging pat-terns. PKDD'02,39-50.
[Bailey et al.,2003] Bailey J., Manoukian T., and Ramamohanarao K. (2003) A fast algorithm for computing hypergraph transversals and its application in mining emerging patterns. ICDM'03,485-488.
[Beinlich et al.,1989] Beinlich, I.A., Suernondt, H., Chavez, R. and Cooper, G. (1989) The ALARM Monitoring System:A Case Study with Two Probabilistic Inference Techniques for Belief Networks. Proc.2nd European Conf. AI and Medicine, London, August, pp. 247-256. Springer, Berlin.
[Binder et al.,1997]Binder, J., Koller, D., Russell, S. and Kanazawa, K. (1997). Adaptive probabilistic networks with hidden variables. Machine Learning,29,213-244.
[Blake and Merz,1998] Blake C. L. and Merz C. J. (1998) UCI Repository of machine learning databases.
[Candes and Tao,2007] Candes E. and Tao T. (2007) The Dantzig selector:statistical estimation when p is much larger than n. Ann Statist,35,2313-2351.
[Cheng et al.,2002] Cheng J., Greiner R., Kelly J., Bell D., and Liu W. (2002) Learning Bayesian networks from data:an information-theorybased approach. Artificial intelligence, 137,43-90.
[Chickering,2002] Chickering D. M. (2002) Learning equivalence classes of Bayesian network structures. Journal of Machine Learning Research,2,445-498.
[Chickeringet al.,2004] Chickering D. M., Heckerman D. and Meek C. (2004) Large sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research,5, 1287-1330.
[Campos and Ji,2011] Campos de C.P. and Ji Q. (2011) Efficient structure learning ofBayesian networks using constraints. Journal of Machine Learning Research,12,663-689.
[Conrads et al.,2004] Conrads T. P. et al. (2004) High-resolution serum proteomic features for ovarian cancer detection. Endocr.Relat Cancer,11:163-178.
[Cowell et al.,1999]Cowell, R.G., Dawid, A.P., Lauritzen, S.L. and Spiegelhalter, D.J. (1999) Probabilistic Networks and Expert Systems. Springer, NewYork.
[Daly and Shen,2009] Daly R. and Shen Q. (2009) Learning Bayesian network equivalence classes with ant colony optimization. Journal of Artificial intelligence Research,35,391-447.
[Dash and Druzdzel,2003] Dash D. and Druzdzel M. (2003) Robust independence testing for constraint-based learning of causal structure. Proc. UAI'03, Acapulco, Mexico, August 8- 10,2003,167-174, MorganKaufmann Publishers, Inc., San Francisco.
[Ding et al.,2011] Ding W., Stepinski T., Mu Y., Bandeira L., Vilalta R., Wu Y., Lu Z., Cao T. and Wu X. (2011) Sub-kilometer crater discovery with boosting and transfer learning. ACM Transactions on Intelligent Systems and Technology,2(4),1-22.
[Dong and Li,1999] Dong G. and Li J. (1999) Efficient mining of emerging patterns: discovering trends and differences. KDD'99,43-52.
[Dong et al.,1999] Dong G., Zhang X., Wong L., and Li J. (1999) CAEP:classification by aggregating emerging patterns. DS'99,30-42.
[Donoho and Huo,2001] Donoho D. L and Huo X. (2001) Uncertainty principles and ideal atomic decomposition. IEEE Transactions on Information Theory,47,2845-2862
[Efron et al.,2004] Efron B., Haistie T., Johnstone I., and Tibshirani R. (2004) Least angle regression. Ann Statist,2004,32,407-499.
[Friedman et al.,2002] Friedman J., Hastie T., and Tibshirani R. (2002). Additive logistic regression:a statistical view of boosting. Ann Statist,28,337-407.
[Friedman et al.,1999] Friedman N., Nachman I., and Peer D. (1999) Learning Bayesian network structure from massive datasets:the "Sparse Candidate". Proc. UAI'99, Stockholm, Sweden, July 30-August 1,1999,206-215. Morgan Kaufmann Publishers, Inc., SanFrancisco.
[Friedman et al.,2000] Friedman, N., Nachman, I. and Peer, D. (2000) Using Bayesian networks to analyze expression data. Comput. Biol.,7,601-620.
[Fan and Ramamohanarao,2002] Fan H. and Ramamohanarao K. (2002) An efficient single-scan algorithm for mining essential jumping emerging patterns for classification. PAKDD'02,456-462.
[Fan and Ramamohanarao,2006] Fan H. and Ramamohanarao K. (2006) Fast discovery and the generalization of strong jumping emerging patterns for building compact and accurate classifiers. IEEE Transactions on Knowledge and Data Engineering 18(6),721-737.
[Fang et al.,2012] Fang G., Pandey G., Wang W., Gupta M., Steinbach M., and Kumar V. (2012) Mining low-support discriminative patterns from dense and high-dimensional data. IEEE Transactions on Knowledge and Data Engineering,24(2),279-294.
[Glocer et al.,2005] Glocer K., Eads D. and Theiler J. (2005) Online feature selection for pixel classification. ICML'05,249-256.
[Guyonand Elisseeff,2003] Guyon, I. and A. Elisseeff. (2003) An introduction to variable and feature selection. Journal of Machine Learning Research,3,1157-1182
[Heckerman et al.,1995] Heckerman, D., Geiger, D. and Chickering, D.M. (1995) Learning Bayesian networks:the combination of knowledge andstatistical data. Mach. Learn.,20, 197-243.
[Joachims,2002] Joachims T. (2002) Learning to classify text using support vector machines. Boston:Kluwer Academic.
[Kojima et al.,2010] Kojima K., Perrier E., Imoto S., and Miyano S. (2010) Optimal search on clustered structural constraint for learning Bayesian network structure. Journal of Machine Learning Research,11,285-310.
[Kohavi and John,1997] Kohavi R. and John G. H. (1997) Wrappers for feature subset selection. Artificial Intelligence,97:273-324.
[Koller and Sahami,1996] Koller, D. and Sahami M. (1996) Toward optimal feature selection. ICML'96,284-292.
[Lee et al.,2011] Lee J. G., Han J., Li X., and Cheng H. (2011) Mining discriminative patterns for classifying trajectories on road networks. IEEE Transactions on Knowledge and Data Engineering,23(5):713-726.
[Li et al.,2000] Li J., Dong G., and Ramamohanarao K. (2000) Making use of the most expressive jumping emerging patterns for classification. PAKDD'00,220-232.
[Li et al.,2001] Li W., Han J., and Pei J. (2001) CMAR:accurate and efficient classification based on multiple-class association rule. ICDM'01,369-376.
[Liu et al.,2011] Liu H., Lin Y. and Han J. (2011) Methods for mining frequent items in data streams:an overview. Knowledge and Information Systems,26(1),1-30.
[Liu et al.,1998] Liu B., Hsu W., and Ma Y. (1998) Integrating classification and association rule mining. KDD'98,80-86.
[Lo et al.,2009] Lo D., Cheng H., Han J., Khoo S., and Sun C. (2009) Classification of software behaviors for failure detection:a discriminative pattern mining approach. KDD'09, 557-566.
[Loekitoand and Bailey,2006] Loekito E. and Bailey J. (2006) Fast mining of high dimensional expressive contrast patterns using zero suppressed binary decision diagrams. KDD'06, 307-316.
[Li et al.,2000a] Li J., Dong G., and Ramamohanarao K. (2000) Making use of the most expressive jumping emerging patterns for classification. PAKDD'00,220-232.
[Li et al.,2000b] Li J., Dong G. and Ramamohanarao K. (2000). Instance-based classification by emerging patterns. PKDD'00,191-200.
[Ma et al.,2009] Ma J., Saul L. K., Savage S., and Voelker G. M. (2009) Identifying Suspicious URLs:An Application of Large-Scale Online Learning. ICML2009.
[Margaritis and Thrun,2000] Margaritis D. and Thrun S. (2000) Bayesian network induction via local neighborhoods. In Solla, S.A., Leen, T.K. and Muller, K.R. (eds), Advances in Neural InformationProcessing Systems,12,505-511. MIT Press, Cambridge, MA.
[Mao and Dong,2005] Mao S. and Dong G. (2005) Discovery of highly differentiative gene groups from microarray gene expression data using the gene club approach. J. Bioinformatics and Computational Biology,3(6):1263-1280.
[Masud et al.,2010] Masud M., Chen Q., Khan L., Aggarwal C., Gao J., Han J., and Thuraisingham B.I. (2010) Addressing concept-evolution in concept-drifting data stream. Proceedings of ICDM'10, Sydney,3-17 Dec.,2010,929-934, IEEE Computer Society, Washington, DC.
[Neapolitan,2003] Neapolitan R. (2003) Learning Bayesian networks. Upper Saddle River, NJ: Prentice Hall.
[Papageorgiou et al.,1998] Papageorgiou, C., Oren, M., and Poggio, T. (1998) A general framework for object detection. In Sixth International Conference on Computer Vision (CVPR'98). Computer Vision,555-562.
[Peartl,1988] Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems.Morgan Kaufmann, San Francisco, CA.
[Pearl,1995] Pearl, J. (1995) Causal diagrams for empirical research (withdiscussion).Biometrika,82,669-710.
[Pearl,2000] Pearl, J. (2000) Causality:Models, Reasoning, and Inference. Cambridge University Press, NewYork.
[Patnaik et al.,2011] Patnaik, D., Laxman, S. and Ramakrishnan, N. (2011) Discovering excitatory relationships using dynamic Bayesian networks. Knowl. Inf. Syst.,29,273-303.
[Pena et al.,2007] Pena J. M., Nilsson R., Bjdrkegren J., and Tegner J. (2007) Towards scalable and data efficient learning of Markov boundaries. Int. J. Approx. Reason.,45,211-232.
[Perkins and Theiler,2003] Perkins S. and Theiler J. (2003) Online feature selection using grafting. ICML'03,592-599.
[Peng et al.,2005] Peng H., Long F. and Ding C. (2005) Feature selection based on mutual information:Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence,27(8):1226-1238.
[Rakotomamonjy,2012] Rakotomamonjy A. (2012) Sparse support vector infinite push. ICML'12.
[Rossetand Zhu,2007] Rosset S. and Zhu J. (2007) Piecewise linear regularization solution paths, Ann Statist,35,1012-1030.
[Rosenwald et al.,2002] Rosenwald A. et al. (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell lymphoma. N. Engl. J Med.,346, 1937-1947.
[Science,2011] The Special Section on Dealing with Data.Science,2011,331:692.
[Schwarz,1978] Schwarz G. (1978) Estimating the dimension of a model. Ann Statist,6, 461-464.
[Shah et al.,2012] Shah M., Marchand M., and Corbeil J. (2012) Feature selection with conjunctions of decision stumps and learning from microarray data. IEEE Transactions on Pattern Analysis and Machine Intelligence,34(1),174-186.
[Song et al.,2012] Song L., Smola A., Gretton A., Bedo J., and Borgward K. (2012) Feature Selection via Dependence Maximization. Journal of Machine Learning Research, 13:1393-1434.
[Spirtes et al.,2000] Spirtes P., Glymour C., and Scheines R. (2000) Causation, Prediction, and Search (2nd edn). MIT Press, Cambridge, MA.
[Statnikov et al.,2003] Statnikov, A., Tsamardinos, I. and Aliferis, C.F. (2003) An Algorithm for the Generation of Large Bayesian Networks. Technical Report DSL-03-01, Vanderbilt University.
[Tibshirani,1996] Tibshirani R. (1996) Regression Shrinkage and Selection via the Lasso. J. Roy. Stat. Soc. B,58(1):267-288.
[Tsamardinos et al.,2006] Tsamardinos, I., Brown, L.E. and Aliferis, C.F. (2006) The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn.,65,31-78.
[Tsamardinos et al.,2003] Tsamardinos I., Aliferis C. F., and Statnikov A. (2003) Time and sample efficient discovery of markov blankets and directcausal relations. Proc. KDD'03, Washington, DC, August24-27,2003,673-678. ACM, NewYork.
[Urbach et al.,2007] Urbach E. R., Roerdink J. B. T. M., and Wilkinson M. H. F. (2007) Connected shapesize pattern spectra for rotation and scale-invariant classication of gray-scale images. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 272-285.
[Urbach and Stepinski,2009] Urbach E. R. and Stepinski T. F. (2009) Automatic detection of sub-km craters in high resolution planetary images. Planetary and Space Science 57, 880-887.
[Wang et al.,2005] Wang Y. et al. (2005) Gene-expression profiles to predict distant metastasis of lymph-node negative primary breast cancer. Lancet,365,671-679.
[Wang and Karypis,2005] Wang J. and Karypis G. (2005) HARMONY:efficiently mining the best rules for classification. SDM'05,205-216.
[Wang et al.2013] Wang D., Ding W., Yu K., Wu X., Cheng P., Small D., and Islam S. (2013) Towards Long-Lead Forecasting of Extreme Flood Events:a Data Mining Framework for Precipitation Cluster Precursors Identification. KDD'13, Chicago, USA, August,2013.
[Wu et al.,2010] Wu X., Yu K., Wang H., and Ding W. (2010) Online Streaming FeatureSelection. Proc. Int'l Conf. Machine Learning (ICML'10),1159-1166.
[Wu et al.,2013] Wu X., Yu K., Ding W., Wang H., and Zhu X. (2013) Online Feature Selection with Streaming Features. IEEE Transactions on Pattern Analysis and Machine Intelligence,35,5:1178-1192.
[Xu et al.,2010] Xu Z., Zhang H., Wang Y., and Chang X. (2010) L1/2regularizer. Science in China (Information Sciences),53:1159-1169.
[Yin and Han,2003] Yin X. and Han J. (2003) CPAR:classification based on predictive association rule. SDM'03,369-376.
[Yu et al.,2011] Yu K., Wu X., Ding W., Wang H., and Yao H. (2011) Causal associative classification. ICDM'll,914-923.
[Yu et al.,2013] Yu K., Ding W., Wang H., and Wu X. (2013) Bridging causal relevance and pattern discriminability:mining emerging patterns from high-dimensional data. IEEE Transactions on Knowledge and Data Engineering, in press.
[Yu and Liu,2004] Yu L. and Liu H. (2004) Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research,5:1205-1224.
[Yu et al.,2008] Yu L., Ding C., and Loscalzo S. (2008) Stable feature selection via dense feature groups. KDD'08,803-811.
[Yuan et al.,2011] Yuan, C., Malone, B. and Wu, X. (2011) Learning optimal bayesian networks using A* search. Proc. IJCAI'll, Barcelona, Catalonia, July 16-22,2011,2186-2191, AAAI Press, Menlo Park, CA.
[Zhao and Yu,2006] P. Zhao and B. Yu. (2006) On model selection consistency of Lasso. Journal of Machine Learning Research,7:2541-2567.
[Zhao and Yu,2007] Zhao P. and Yu B. (2007) Stagewiselasso. Journal of Machine Learning Research,8,2701-2726.
[Zhang et al.,2000a] Zhang X., Dong G., and Ramamohanarao K. (2000) Exploring constraints to efficiently mine emerg-ing patterns from large high-dimensional datasets. KDD'00, 310-314.
[Zhang et al.,2000b] Zhang X., Dong G., and Ramamohanarao K. (2000) Information-based classification by aggregating emerging patterns. IDEAL'00,48-53.
[Zhang,2009] Zhang T. (2009) On the consistency of feature selection using greedy least squares regression. Journal of Machine Learning Research,10:555-568.
[Zhang et al.,2011] Zhang Y, Zhu X., Wu X., and Jeffrey P. B. (2011) Corrective classification: learning from data imperfections with aggressive and diverse classifier ensembling. Information Systems,36,8:1135-1157.
[Zhou et al.,2005] Zhou J., Foster D. P., Stine R., and Ungar L.H. (2005) Streaming feature selection using Alpha-investing. KDD'05,384-393.
[Zhou et al.,2006] Zhou J., Foster D., Stine R.A., and Ungar L.H. (2006) Streamwise feature selection. Journal of Machine Learning Research,7:1861-1885.
[Akaike,1973] Akaike H. (1973) Information theory and an extension of the maximum likelihood principle. In:B. N. Petrov, F. Caki, eds. Second International Symposium on Information Theory. Budapest:AkademiaiKiado,267-281.
[Aliferisand Tsamardinos,2002] Aliferis C. F. and Tsamardinos Ⅰ. (2002) Algorithms for large-scale local causal discovery and feature selection in the presence of small sample or large causal neighborhoods. Technical Report DSL 02-08, Department of Biomedicallnformatics, Vanderbilt University.
[Aliferis et al.,2003a] Aliferis C.F., Tsamardinos I., Statnikov A., and Brown L. E. (2003) Causal explorer:a causal probabilistic network learning toolkit for biomedical discovery. METMBS'03.
[Aliferiset et al.,2003b] Aliferis C.F., Tsamardinos I., and Statnikov A. (2003) HITON:a novel markov blanket algorithm for optimal variable selection. American Medical Informatics Association (AMIA) Annual Symposium, Vancouver, British Columbia, November18-22, 2003,21-25. AMIA, Bethesda, Maryland, USA.
[Aliferis et al.,2010a] Aliferis, C. F., A. Statnikov, I. Tsamardinos, S. Mani, and X. Koutsoukos. (2010) Local causal and Markov blanket induction for causal discovery and feature selection for classification Part Ⅰ:algorithms and empirical evaluation. Journal of Machine Learning Research,11,171-234.
[Aliferis et al.,2010b] Aliferis C. F, Statnikov A., Tsamardinos I., Mani S., and Koutsoukos X. (2010) Local causal and Markov blanketinduction for causal discovery and feature selection forclassification part Ⅱ:analysis and extensions. Journal of Machine Learning Research,11,235-284.
[Aphinyanaphongs et al.,2006] Aphinyanaphongs Y., Statnikov A. and Aliferis C. F. (2006) A comparison of citation metrics to machine learning filters for the identification of high quality medline documents. J. Am. Med. Inform. Assoc.,13(4):446-455.
[Baralis et al.,2008] Baralis E., Chiusano S., and Garza P. (2008) A Lazy Approach to Associative Classification. IEEE Transactions on Knowledge and Data Engineering, 20(2):156-171.
[Brown et al.,2012] Brown G., Pocock A., Zhao M., and Luj'an M. (2012) Conditional likelihood maximisation:a unifying framework for information theoretic feature selection. Journal of Machine Learning Research,13:27-66.
[Bayardo,1998] Bayardo R. J. (1998) Efficiently mining long patterns from databases. SIGMOD'98,85-93.
[Bailey et al.,2002] Bailey J., Manoukian T., and Ramamohanarao K. (2002) Fast algorithms for mining emerging pat-terns. PKDD'02,39-50.
[Bailey et al.,2003] Bailey J., Manoukian T., and Ramamohanarao K. (2003) A fast algorithm for computing hypergraph transversals and its application in mining emerging patterns. ICDM'03,485-488.
[Beinlich et al.,1989] Beinlich, I.A., Suernondt, H., Chavez, R. and Cooper, G. (1989) The ALARM Monitoring System:A Case Study with Two Probabilistic Inference Techniques for Belief Networks. Proc.2nd European Conf. AI and Medicine, London, August, pp. 247-256. Springer, Berlin.
[Binder et al.,1997]Binder, J., Koller, D., Russell, S. and Kanazawa, K. (1997). Adaptive probabilistic networks with hidden variables. Machine Learning,29,213-244.
[Blake and Merz,1998] Blake C. L. and Merz C. J. (1998) UCI Repository of machine learning databases.
[Candes and Tao,2007] Candes E. and Tao T. (2007) The Dantzig selector:statistical estimation when p is much larger than n. Ann Statist,35,2313-2351.
[Cheng et al.,2002] Cheng J., Greiner R., Kelly J., Bell D., and Liu W. (2002) Learning Bayesian networks from data:an information-theorybased approach. Artificial intelligence, 137,43-90.
[Chickering,2002] Chickering D. M. (2002) Learning equivalence classes of Bayesian network structures. Journal of Machine Learning Research,2,445-498.
[Chickeringet al.,2004] Chickering D. M., Heckerman D. and Meek C. (2004) Large sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research,5, 1287-1330.
[Campos and Ji,2011] Campos de C.P. and Ji Q. (2011) Efficient structure learning ofBayesian networks using constraints. Journal of Machine Learning Research,12,663-689.
[Conrads et al.,2004] Conrads T. P. et al. (2004) High-resolution serum proteomic features for ovarian cancer detection. Endocr.Relat Cancer,11:163-178.
[Cowell et al.,1999]Cowell, R.G., Dawid, A.P., Lauritzen, S.L. and Spiegelhalter, D.J. (1999) Probabilistic Networks and Expert Systems. Springer, NewYork.
[Daly and Shen,2009] Daly R. and Shen Q. (2009) Learning Bayesian network equivalence classes with ant colony optimization. Journal of Artificial intelligence Research,35,391-447.
[Dash and Druzdzel,2003] Dash D. and Druzdzel M. (2003) Robust independence testing for constraint-based learning of causal structure. Proc. UAI'03, Acapulco, Mexico, August 8- 10,2003,167-174, MorganKaufmann Publishers, Inc., San Francisco.
[Ding et al.,2011] Ding W., Stepinski T., Mu Y., Bandeira L., Vilalta R., Wu Y., Lu Z., Cao T. and Wu X. (2011) Sub-kilometer crater discovery with boosting and transfer learning. ACM Transactions on Intelligent Systems and Technology,2(4),1-22.
[Dong and Li,1999] Dong G. and Li J. (1999) Efficient mining of emerging patterns: discovering trends and differences. KDD'99,43-52.
[Dong et al.,1999] Dong G., Zhang X., Wong L., and Li J. (1999) CAEP:classification by aggregating emerging patterns. DS'99,30-42.
[Donoho and Huo,2001] Donoho D. L and Huo X. (2001) Uncertainty principles and ideal atomic decomposition. IEEE Transactions on Information Theory,47,2845-2862
[Efron et al.,2004] Efron B., Haistie T., Johnstone I., and Tibshirani R. (2004) Least angle regression. Ann Statist,2004,32,407-499.
[Friedman et al.,2002] Friedman J., Hastie T., and Tibshirani R. (2002). Additive logistic regression:a statistical view of boosting. Ann Statist,28,337-407.
[Friedman et al.,1999] Friedman N., Nachman I., and Peer D. (1999) Learning Bayesian network structure from massive datasets:the "Sparse Candidate". Proc. UAI'99, Stockholm, Sweden, July 30-August 1,1999,206-215. Morgan Kaufmann Publishers, Inc., SanFrancisco.
[Friedman et al.,2000] Friedman, N., Nachman, I. and Peer, D. (2000) Using Bayesian networks to analyze expression data. Comput. Biol.,7,601-620.
[Fan and Ramamohanarao,2002] Fan H. and Ramamohanarao K. (2002) An efficient single-scan algorithm for mining essential jumping emerging patterns for classification. PAKDD'02,456-462.
[Fan and Ramamohanarao,2006] Fan H. and Ramamohanarao K. (2006) Fast discovery and the generalization of strong jumping emerging patterns for building compact and accurate classifiers. IEEE Transactions on Knowledge and Data Engineering 18(6),721-737.
[Fang et al.,2012] Fang G., Pandey G., Wang W., Gupta M., Steinbach M., and Kumar V. (2012) Mining low-support discriminative patterns from dense and high-dimensional data. IEEE Transactions on Knowledge and Data Engineering,24(2),279-294.
[Glocer et al.,2005] Glocer K., Eads D. and Theiler J. (2005) Online feature selection for pixel classification. ICML'05,249-256.
[Guyonand Elisseeff,2003] Guyon, I. and A. Elisseeff. (2003) An introduction to variable and feature selection. Journal of Machine Learning Research,3,1157-1182
[Heckerman et al.,1995] Heckerman, D., Geiger, D. and Chickering, D.M. (1995) Learning Bayesian networks:the combination of knowledge andstatistical data. Mach. Learn.,20, 197-243.
[Joachims,2002] Joachims T. (2002) Learning to classify text using support vector machines. Boston:Kluwer Academic.
[Kojima et al.,2010] Kojima K., Perrier E., Imoto S., and Miyano S. (2010) Optimal search on clustered structural constraint for learning Bayesian network structure. Journal of Machine Learning Research,11,285-310.
[Kohavi and John,1997] Kohavi R. and John G. H. (1997) Wrappers for feature subset selection. Artificial Intelligence,97:273-324.
[Koller and Sahami,1996] Koller, D. and Sahami M. (1996) Toward optimal feature selection. ICML'96,284-292.
[Lee et al.,2011] Lee J. G., Han J., Li X., and Cheng H. (2011) Mining discriminative patterns for classifying trajectories on road networks. IEEE Transactions on Knowledge and Data Engineering,23(5):713-726.
[Li et al.,2000] Li J., Dong G., and Ramamohanarao K. (2000) Making use of the most expressive jumping emerging patterns for classification. PAKDD'00,220-232.
[Li et al.,2001] Li W., Han J., and Pei J. (2001) CMAR:accurate and efficient classification based on multiple-class association rule. ICDM'01,369-376.
[Liu et al.,2011] Liu H., Lin Y. and Han J. (2011) Methods for mining frequent items in data streams:an overview. Knowledge and Information Systems,26(1),1-30.
[Liu et al.,1998] Liu B., Hsu W., and Ma Y. (1998) Integrating classification and association rule mining. KDD'98,80-86.
[Lo et al.,2009] Lo D., Cheng H., Han J., Khoo S., and Sun C. (2009) Classification of software behaviors for failure detection:a discriminative pattern mining approach. KDD'09, 557-566.
[Loekitoand and Bailey,2006] Loekito E. and Bailey J. (2006) Fast mining of high dimensional expressive contrast patterns using zero suppressed binary decision diagrams. KDD'06, 307-316.
[Li et al.,2000a] Li J., Dong G., and Ramamohanarao K. (2000) Making use of the most expressive jumping emerging patterns for classification. PAKDD'00,220-232.
[Li et al.,2000b] Li J., Dong G. and Ramamohanarao K. (2000). Instance-based classification by emerging patterns. PKDD'00,191-200.
[Ma et al.,2009] Ma J., Saul L. K., Savage S., and Voelker G. M. (2009) Identifying Suspicious URLs:An Application of Large-Scale Online Learning. ICML2009.
[Margaritis and Thrun,2000] Margaritis D. and Thrun S. (2000) Bayesian network induction via local neighborhoods. In Solla, S.A., Leen, T.K. and Muller, K.R. (eds), Advances in Neural InformationProcessing Systems,12,505-511. MIT Press, Cambridge, MA.
[Mao and Dong,2005] Mao S. and Dong G. (2005) Discovery of highly differentiative gene groups from microarray gene expression data using the gene club approach. J. Bioinformatics and Computational Biology,3(6):1263-1280.
[Masud et al.,2010] Masud M., Chen Q., Khan L., Aggarwal C., Gao J., Han J., and Thuraisingham B.I. (2010) Addressing concept-evolution in concept-drifting data stream. Proceedings of ICDM'10, Sydney,3-17 Dec.,2010,929-934, IEEE Computer Society, Washington, DC.
[Neapolitan,2003] Neapolitan R. (2003) Learning Bayesian networks. Upper Saddle River, NJ: Prentice Hall.
[Papageorgiou et al.,1998] Papageorgiou, C., Oren, M., and Poggio, T. (1998) A general framework for object detection. In Sixth International Conference on Computer Vision (CVPR'98). Computer Vision,555-562.
[Peartl,1988] Pearl, J. (1988) Probabilistic Reasoning in Intelligent Systems.Morgan Kaufmann, San Francisco, CA.
[Pearl,1995] Pearl, J. (1995) Causal diagrams for empirical research (withdiscussion).Biometrika,82,669-710.
[Pearl,2000] Pearl, J. (2000) Causality:Models, Reasoning, and Inference. Cambridge University Press, NewYork.
[Patnaik et al.,2011] Patnaik, D., Laxman, S. and Ramakrishnan, N. (2011) Discovering excitatory relationships using dynamic Bayesian networks. Knowl. Inf. Syst.,29,273-303.
[Pena et al.,2007] Pena J. M., Nilsson R., Bjdrkegren J., and Tegner J. (2007) Towards scalable and data efficient learning of Markov boundaries. Int. J. Approx. Reason.,45,211-232.
[Perkins and Theiler,2003] Perkins S. and Theiler J. (2003) Online feature selection using grafting. ICML'03,592-599.
[Peng et al.,2005] Peng H., Long F. and Ding C. (2005) Feature selection based on mutual information:Criteria of max-dependency, max-relevance, and min-redundancy. IEEE Transactions on Pattern Analysis and Machine Intelligence,27(8):1226-1238.
[Rakotomamonjy,2012] Rakotomamonjy A. (2012) Sparse support vector infinite push. ICML'12.
[Rossetand Zhu,2007] Rosset S. and Zhu J. (2007) Piecewise linear regularization solution paths, Ann Statist,35,1012-1030.
[Rosenwald et al.,2002] Rosenwald A. et al. (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-B-cell lymphoma. N. Engl. J Med.,346, 1937-1947.
[Science,2011] The Special Section on Dealing with Data.Science,2011,331:692.
[Schwarz,1978] Schwarz G. (1978) Estimating the dimension of a model. Ann Statist,6, 461-464.
[Shah et al.,2012] Shah M., Marchand M., and Corbeil J. (2012) Feature selection with conjunctions of decision stumps and learning from microarray data. IEEE Transactions on Pattern Analysis and Machine Intelligence,34(1),174-186.
[Song et al.,2012] Song L., Smola A., Gretton A., Bedo J., and Borgward K. (2012) Feature Selection via Dependence Maximization. Journal of Machine Learning Research, 13:1393-1434.
[Spirtes et al.,2000] Spirtes P., Glymour C., and Scheines R. (2000) Causation, Prediction, and Search (2nd edn). MIT Press, Cambridge, MA.
[Statnikov et al.,2003] Statnikov, A., Tsamardinos, I. and Aliferis, C.F. (2003) An Algorithm for the Generation of Large Bayesian Networks. Technical Report DSL-03-01, Vanderbilt University.
[Tibshirani,1996] Tibshirani R. (1996) Regression Shrinkage and Selection via the Lasso. J. Roy. Stat. Soc. B,58(1):267-288.
[Tsamardinos et al.,2006] Tsamardinos, I., Brown, L.E. and Aliferis, C.F. (2006) The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn.,65,31-78.
[Tsamardinos et al.,2003] Tsamardinos I., Aliferis C. F., and Statnikov A. (2003) Time and sample efficient discovery of markov blankets and directcausal relations. Proc. KDD'03, Washington, DC, August24-27,2003,673-678. ACM, NewYork.
[Urbach et al.,2007] Urbach E. R., Roerdink J. B. T. M., and Wilkinson M. H. F. (2007) Connected shapesize pattern spectra for rotation and scale-invariant classication of gray-scale images. IEEE Transactions on Pattern Analysis and Machine Intelligence 29, 272-285.
[Urbach and Stepinski,2009] Urbach E. R. and Stepinski T. F. (2009) Automatic detection of sub-km craters in high resolution planetary images. Planetary and Space Science 57, 880-887.
[Wang et al.,2005] Wang Y. et al. (2005) Gene-expression profiles to predict distant metastasis of lymph-node negative primary breast cancer. Lancet,365,671-679.
[Wang and Karypis,2005] Wang J. and Karypis G. (2005) HARMONY:efficiently mining the best rules for classification. SDM'05,205-216.
[Wang et al.2013] Wang D., Ding W., Yu K., Wu X., Cheng P., Small D., and Islam S. (2013) Towards Long-Lead Forecasting of Extreme Flood Events:a Data Mining Framework for Precipitation Cluster Precursors Identification. KDD'13, Chicago, USA, August,2013.
[Wu et al.,2010] Wu X., Yu K., Wang H., and Ding W. (2010) Online Streaming FeatureSelection. Proc. Int'l Conf. Machine Learning (ICML'10),1159-1166.
[Wu et al.,2013] Wu X., Yu K., Ding W., Wang H., and Zhu X. (2013) Online Feature Selection with Streaming Features. IEEE Transactions on Pattern Analysis and Machine Intelligence,35,5:1178-1192.
[Xu et al.,2010] Xu Z., Zhang H., Wang Y., and Chang X. (2010) L1/2regularizer. Science in China (Information Sciences),53:1159-1169.
[Yin and Han,2003] Yin X. and Han J. (2003) CPAR:classification based on predictive association rule. SDM'03,369-376.
[Yu et al.,2011] Yu K., Wu X., Ding W., Wang H., and Yao H. (2011) Causal associative classification. ICDM'll,914-923.
[Yu et al.,2013] Yu K., Ding W., Wang H., and Wu X. (2013) Bridging causal relevance and pattern discriminability:mining emerging patterns from high-dimensional data. IEEE Transactions on Knowledge and Data Engineering, in press.
[Yu and Liu,2004] Yu L. and Liu H. (2004) Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research,5:1205-1224.
[Yu et al.,2008] Yu L., Ding C., and Loscalzo S. (2008) Stable feature selection via dense feature groups. KDD'08,803-811.
[Yuan et al.,2011] Yuan, C., Malone, B. and Wu, X. (2011) Learning optimal bayesian networks using A* search. Proc. IJCAI'll, Barcelona, Catalonia, July 16-22,2011,2186-2191, AAAI Press, Menlo Park, CA.
[Zhao and Yu,2006] P. Zhao and B. Yu. (2006) On model selection consistency of Lasso. Journal of Machine Learning Research,7:2541-2567.
[Zhao and Yu,2007] Zhao P. and Yu B. (2007) Stagewiselasso. Journal of Machine Learning Research,8,2701-2726.
[Zhang et al.,2000a] Zhang X., Dong G., and Ramamohanarao K. (2000) Exploring constraints to efficiently mine emerg-ing patterns from large high-dimensional datasets. KDD'00, 310-314.
[Zhang et al.,2000b] Zhang X., Dong G., and Ramamohanarao K. (2000) Information-based classification by aggregating emerging patterns. IDEAL'00,48-53.
[Zhang,2009] Zhang T. (2009) On the consistency of feature selection using greedy least squares regression. Journal of Machine Learning Research,10:555-568.
[Zhang et al.,2011] Zhang Y, Zhu X., Wu X., and Jeffrey P. B. (2011) Corrective classification: learning from data imperfections with aggressive and diverse classifier ensembling. Information Systems,36,8:1135-1157.
[Zhou et al.,2005] Zhou J., Foster D. P., Stine R., and Ungar L.H. (2005) Streaming feature selection using Alpha-investing. KDD'05,384-393.
[Zhou et al.,2006] Zhou J., Foster D., Stine R.A., and Ungar L.H. (2006) Streamwise feature selection. Journal of Machine Learning Research,7:1861-1885.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |