Acceleration of the EM Algorithm Using the Vector Aitken Method and Its Steffensen Form
|
摘要
Based on Vector Aitken(VA) method,we propose an acceleration Expectation-Maximization(EM)algorithm,VA-accelerated EM algorithm,whose convergence speed is faster than that of EM algorithm.The VA-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm,thus it keeps the flexibility and simplicity of the EM algorithm.Considering Steffensen iterative process,we have also given the Steffensen form of the VA-accelerated EM algorithm.It can be proved that the reform process is quadratic convergence.Numerical analysis illustrate the proposed methods are efficient and faster than EM algorithm.
Based on Vector Aitken(VA) method,we propose an acceleration Expectation-Maximization(EM)algorithm,VA-accelerated EM algorithm,whose convergence speed is faster than that of EM algorithm.The VA-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm,thus it keeps the flexibility and simplicity of the EM algorithm.Considering Steffensen iterative process,we have also given the Steffensen form of the VA-accelerated EM algorithm.It can be proved that the reform process is quadratic convergence.Numerical analysis illustrate the proposed methods are efficient and faster than EM algorithm.
Based on Vector Aitken(VA) method,we propose an acceleration Expectation-Maximization(EM)algorithm,VA-accelerated EM algorithm,whose convergence speed is faster than that of EM algorithm.The VA-accelerated EM algorithm does not use the information matrix but only uses the sequence of estimates obtained from iterations of the EM algorithm,thus it keeps the flexibility and simplicity of the EM algorithm.Considering Steffensen iterative process,we have also given the Steffensen form of the VA-accelerated EM algorithm.It can be proved that the reform process is quadratic convergence.Numerical analysis illustrate the proposed methods are efficient and faster than EM algorithm.
引文
[1]Brezinski,C.,Zagria,R.Extrapolation methods:theory and practice.Elsevier,Amsterdam,1991
[2]Dempster,A.P.,Laird,N.M.,Rubin,D.B.Maximum likelihood from incomplete data via the EM algorithm(with discussion).J.Roy.Statist.Soc.Ser.B.,39:1–38(1977)
[3]Gelfand,A.E.,Smith,A.F.M.Samping-based approaches to calcuating marginal densities.J.Am.Statist.Assoc.,85:398–409(1990)
[4]Jamshidian,M.,Jennrich,R.I.Conjugate gradient accleration of the EM algorithm.J.Am.Statist.Assoc.,88:221–228(1993)
[5]Jamshidian,M.,Jennrich,R.I.Acceleration of the EM algorithm by using quasi-newton methods.J.Roy.Statist.Soc.Ser.B.,59:569–587(1997)
[6]Kuroda,M.,Sakakihara,M.,Geng,Z.Acceleration of the EM and ECM algorithms using the Aitken 2method for log-linear models with partially classified data.Statist.and Prob.Lett.,78:2332–2338(2008)
[7]Lange,K.A gradient algorithm locally equlvalent to the EM algorithm.J.Roy.Statist.Soc.Ser.B.,57:425–437(1995)
[8]Lange,K.A quasi-newton acceleration of the EM alogorithm.Statist.Sinica.,5:1–18(1995)
[9]Liu,C.,Rubin,D.B.The ECME algorithm:a simple extension of EM and ECM with faster monotone convergence.Biometrika,81:633–648(1994)
[10]Liu,C.,Rubin,D.B.,Wu,Y.N.Parameter expansion to accelerate EM:The PX-EM algorithm.Biometrika,88:755–770(1998)
[11]Louis,T.A.Finding the observed information matrix when using the EM algorithm.J.Roy.Statist.Soc.Ser.B.,44:226–233(1982)
[12]Meng,X.L.,Rubin,D.B.Maximum likelihood estimation via the ECM algorithm:a general framework.Biometrika,80:267–278(1993)
[13]Schafer,J.L.Analysis of Incomplete Multivariate Data.Chapman Hall/CRC,London,1999
[14]Wang,M.F.,Kuroda,M.,Sakakihara,M.,Geng,Z.Acceleration of the EM algorithm using the vector epsilon algorithm.Comput.Statist.,23:469–486(2008)
[15]Wu,C.F.On the convergence properties of the EM algorithm.Ann.Statist.,11:95–103(1983)
[2]Dempster,A.P.,Laird,N.M.,Rubin,D.B.Maximum likelihood from incomplete data via the EM algorithm(with discussion).J.Roy.Statist.Soc.Ser.B.,39:1–38(1977)
[3]Gelfand,A.E.,Smith,A.F.M.Samping-based approaches to calcuating marginal densities.J.Am.Statist.Assoc.,85:398–409(1990)
[4]Jamshidian,M.,Jennrich,R.I.Conjugate gradient accleration of the EM algorithm.J.Am.Statist.Assoc.,88:221–228(1993)
[5]Jamshidian,M.,Jennrich,R.I.Acceleration of the EM algorithm by using quasi-newton methods.J.Roy.Statist.Soc.Ser.B.,59:569–587(1997)
[6]Kuroda,M.,Sakakihara,M.,Geng,Z.Acceleration of the EM and ECM algorithms using the Aitken 2method for log-linear models with partially classified data.Statist.and Prob.Lett.,78:2332–2338(2008)
[7]Lange,K.A gradient algorithm locally equlvalent to the EM algorithm.J.Roy.Statist.Soc.Ser.B.,57:425–437(1995)
[8]Lange,K.A quasi-newton acceleration of the EM alogorithm.Statist.Sinica.,5:1–18(1995)
[9]Liu,C.,Rubin,D.B.The ECME algorithm:a simple extension of EM and ECM with faster monotone convergence.Biometrika,81:633–648(1994)
[10]Liu,C.,Rubin,D.B.,Wu,Y.N.Parameter expansion to accelerate EM:The PX-EM algorithm.Biometrika,88:755–770(1998)
[11]Louis,T.A.Finding the observed information matrix when using the EM algorithm.J.Roy.Statist.Soc.Ser.B.,44:226–233(1982)
[12]Meng,X.L.,Rubin,D.B.Maximum likelihood estimation via the ECM algorithm:a general framework.Biometrika,80:267–278(1993)
[13]Schafer,J.L.Analysis of Incomplete Multivariate Data.Chapman Hall/CRC,London,1999
[14]Wang,M.F.,Kuroda,M.,Sakakihara,M.,Geng,Z.Acceleration of the EM algorithm using the vector epsilon algorithm.Comput.Statist.,23:469–486(2008)
[15]Wu,C.F.On the convergence properties of the EM algorithm.Ann.Statist.,11:95–103(1983)