神经动态规划在水泥分解炉温度控制中的应用研究
|
摘要
新型干法水泥预分解技术就是在预热器和回转窑之间增设分解炉，水泥生料在分解炉中要完成90％以上的碳酸盐预分解，对其温度的稳定和优化控制是保证正常分解的充分必要条件。
由于生料成分的多样性和热变换的持续性，分解炉的温度控制是一个非线性的复杂控制对象，影响它的外部因素很多，且各个因素之间存在耦合和不确定性，一直是分解炉动态稳定控制中难以解决的问题。
神经动态规划方法(NDP)是一种基于现场实际数据的不依赖于系统精确数学模型的在线控制方法。文中用于实现分解炉温度控制NDP方法中的双重启发式动态规划(DHP)，主要由三部分组成：模型网络(MODEL)、执行网络(ACTION)和评价网络(CRITIC)。
首先，利用机理与神经网络结合的方法建立分解炉温度控制的神经网络模型(NNM)，然后，将NNM作为DHP结构中的MODEL模块进行训练，再依次训练评价网络和执行网络。这三部分均由BP网络构成，使用函数逼近的方法，来控制分解炉的温度变化。最后，与生产实际中使用的实时PID控制、仿真PID控制、BP神经网络控制进行比较，显示了NDP方法在控制效果上的优越性。
In cement manufacturing, pre-calcining process (PCP) technology is an added decomposing furnace between the preheater and the kiln. Since raw meal decomposition rate in the kiln is over 90%, optimal control is the necessary and sufficient condition for keeping temperature stable to ensure normal decomposition.
Due to the diversity of raw meal composition and the consistent thermal transformation, temperature control in the decomposing furnace is a complex nonlinear control object. Many factors influence it, and coupling between various factors and uncertainties, it is difficult to solve the dynamic stability control problem for the temperature in decomposing furnace.
Neural Dynamic Programming (NDP) is an on line control approach that based on actual data not with the exact mathematical model. This paper addresses temperature control in the decomposing furnace using dual heuristic dynamic programming (DHP) which a method of NDP. The approach is mainly composed of three parts: model network (MODEL), action network (ACTION) and critic network (CRITIC).
Firstly, we establish the neural network model (NNM) of temperature control of decomposing furnace by the method of mechanism and neural network. Then, put NNM as a structure of the training module MODEL in DHP. Next follow training critic net and action net. All three parts use BP networks. Temperature changes Control with Function approximation methods. Finally, we compared to the real-time PID control in actual fieldwork, Simulation PID control and BP neural network control. Demonstrate the advantages of the NDP in control.
由于生料成分的多样性和热变换的持续性，分解炉的温度控制是一个非线性的复杂控制对象，影响它的外部因素很多，且各个因素之间存在耦合和不确定性，一直是分解炉动态稳定控制中难以解决的问题。
神经动态规划方法(NDP)是一种基于现场实际数据的不依赖于系统精确数学模型的在线控制方法。文中用于实现分解炉温度控制NDP方法中的双重启发式动态规划(DHP)，主要由三部分组成：模型网络(MODEL)、执行网络(ACTION)和评价网络(CRITIC)。
首先，利用机理与神经网络结合的方法建立分解炉温度控制的神经网络模型(NNM)，然后，将NNM作为DHP结构中的MODEL模块进行训练，再依次训练评价网络和执行网络。这三部分均由BP网络构成，使用函数逼近的方法，来控制分解炉的温度变化。最后，与生产实际中使用的实时PID控制、仿真PID控制、BP神经网络控制进行比较，显示了NDP方法在控制效果上的优越性。
In cement manufacturing, pre-calcining process (PCP) technology is an added decomposing furnace between the preheater and the kiln. Since raw meal decomposition rate in the kiln is over 90%, optimal control is the necessary and sufficient condition for keeping temperature stable to ensure normal decomposition.
Due to the diversity of raw meal composition and the consistent thermal transformation, temperature control in the decomposing furnace is a complex nonlinear control object. Many factors influence it, and coupling between various factors and uncertainties, it is difficult to solve the dynamic stability control problem for the temperature in decomposing furnace.
Neural Dynamic Programming (NDP) is an on line control approach that based on actual data not with the exact mathematical model. This paper addresses temperature control in the decomposing furnace using dual heuristic dynamic programming (DHP) which a method of NDP. The approach is mainly composed of three parts: model network (MODEL), action network (ACTION) and critic network (CRITIC).
Firstly, we establish the neural network model (NNM) of temperature control of decomposing furnace by the method of mechanism and neural network. Then, put NNM as a structure of the training module MODEL in DHP. Next follow training critic net and action net. All three parts use BP networks. Temperature changes Control with Function approximation methods. Finally, we compared to the real-time PID control in actual fieldwork, Simulation PID control and BP neural network control. Demonstrate the advantages of the NDP in control.
引文
[1] 曾学敏．中国水泥工业及其供求现状．新世纪水泥导报，2003(5)．
[2] 刘志江．新型干法水泥技术．中国建材工业出版社，2005．1
[3] 王益民．我国建材工业自动化现状与发展，中国建材报2001(2)
[4] D.P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996
[5] D.P. Bertsekas, M.L. Homer, D.A. Logan, S.D Patek, and N. R. Sandell, Missile Defense and Interceptor Allocation By Neuro-Dynamic Programming, IEEE Transactions on Systems, Man and Cybernetics, partA, 2000, Vol. 30(1), 42-51
[6] C.J.C.H.Watkins, Learning from Delayed Rewards, Ph.D. Thesis, Cambridge University, Cambridge, England, 1989.
[7] C.J.C.H. Watkins and P Dayan, Q-Learning, Machine.8, 279-292
[8] G.J. Tesauro, Practcal Issues in Temporal Difference Learning, Machine Learning, 1992, 257-277
[9] W. Zhang and T.G. Dietterich, A Reinforcement Learning Approach to Job Shop Scheduling, Proceedings of the 14th IJCAI, 1114-1220
[10] Danil V. Prokhorov, and Donald C. Wunsch, Adaptive Critic Designs. IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 8, NO. 5, SEPTEMBER 1997
[11] Liu D, Xiong X, Zhang Y. Action-dependent adaptive critic designs, In: Proceedings of the INNS-IEEE Intemational Joint Conference on Neural Networks, Washington, DC, 2001, 7:990～995
[12] Lendaris G G, Paintz C. Training strategies for critic and action neural networks in dual heuristic progrmming method, In: Proceedings of the 1997 IEEE International Conference on Neural Networks, Houston, TX, 1997,6:712～717
[13] Cox C, Stepniewski S, Jorgensen C, Saeks R, Lewis C. On the design of a neural network autolander, International Journal of Robust Nonlinear Control, 1999, 9:1071～1096
[14] Murray J J, Cox C J, Lendaris G G, Saeks R. Adaptive dynamic programming, IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 2002, 32(5): 140～153
[15] Murray J J, Cox C J, Saeks R E. The adaptive dynamic programming theorem, In: Stability and Control of Dynamical Systems with Applications, D. Liu and P. J. Antsaldins, Editors, Boston, MA: Birkhauser, 2003,379～394
[16] Xin Liul,S.N. Balakrishnan. CONVERGENCE ANALYSIS OF ADAPTIVE CRITIC BASED OPTIMAL CONTROL. Proceedings of the American Control Conference Chicago, Illinois June 2000
[17] Paul J. Werbos. Stable Adaptive Control Using New Critic Designs. arⅩⅳ:adap-org/9810001 vl 25 Sep 1998
[18] Remi Munos, Leemon C. Baird, Andrew W. Moore. Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation. International Joint Conference on Neural Networks, July, 1999.
[19] REMI MUNOS. Variable Resolution Discretization in Optimal Control. Machine Learning, 49, 291-323, 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.
[20] Paul J. Werbos. Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities. www.die.uchile.cl/ieee-cis/evic2005/files/AD2004Werbosv2
[21] Chang-Yun Seong, Bernard Widrow, Neural Dynamic Optimization for Control Systems, CYBERNETICS, VOL. 31, NO. 4, AUGUST 2001
[22] Van Roy B, Bertsekas D P, Lee Y, et al. Neuro-Dynamic Programming Approach to Retailer Inventory Management, Proceedings of the IEEE Conference on Decision and Control, 1997,4:4052～4057
[23] Bertsekas D P, Tsitsiklis J N. Neuro-dynamic programming. Belmont: Athena Scientific, 1996
[24] Enns R, Si J. Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Transactions on Neural Network, 2003, 14(4):929～939.
[25] Han D.Flight control with adaptive critic neural network. University of Missouri-Rolla. 2001.
[26] White D A,Sofge D A. Handbook of intelligent control -neural, fuzzy and adaptive approaches. New York: Van Nostrand Reinhold, 1992.
[27] Miller W T, Sutton R S, Werbos P J. Neural networks for control. Cambridge: MIT Press, 1990.
[28] Si J, Barto A, Powell W, et al.Handbook of learning and approximate dynamic programming. New York: John Wiley&Sons2004
[29] Mas P, Brooks R. Learning to coordinate behaviors. Proceedings of the 8th AAAI, Morgan Kaufmann, 1990,796～802
[30] Martin T.Hagan, Howard B.Demuth. Neural Network Design. China Machine Press.2002.
[31] Kalmar Z, Szepesvari C, Lorincz A. Module-based reinforcement learning: experiments with a real robot. Proceedings of the 6th European Workshop on Learning Robots. Machine Leaning, 1998,34:55～85
[32] Excelente-Toledo C B, Jennings N R. Learning when and how to coordinate. Web Intelligence and Agent System, 2003,1(3/4):203～218
[33] C.Anderson. Approximating a policy can be easier than approximating a value function. Technical Report CS-00-101, Colorado State University, 2000
[34] C. Anderson. R.M. Kretchner,EM Young, and D.C.Hittle. Robust reinforcement learning control with static and dynamic stability. International Journal of Robust and Nonlinear Control, 2001
[35] Z. Hung and S. N. Balakrishnan. Robust adaptive critic based neurocontrollers for missiles with model uncertainties. 2001 AAA Guidance, Navigation and control Conference, Montreal, Canada, 2001
[36] Z. Hung and S. N. Balakrishnan. Robust adaptive critic based neurocontrollers for systems with input uncertainties. Proceedings of IJCNN'2000, Como, Italy, 2000:B-263
[37] 望安全，陈宗海，文锋，一种基于强化学习的控制算法研究．计算机仿真，Vol．20，No．11，2003，
[38] 金辉宇，于海斌，神经元动态规划综述．信息与控制．第30卷第4期，2001年8月
[39] 陆超，谢小荣，童陆园，王仲鸿．使用直接神经动态规划方法的SVC附加阻尼控制．中国电机工程学报．第24卷第12期，2004年12月
[40] 钱征，孙亮，阮晓钢．一种基于递归神经网络的白适应控制方法研究．微计算机信息(测控白动化)2005年第21卷第11-1期
[41] 王科俊，王可成．神经网络建模、预报与控制．哈尔滨工程大学出版社．1996年12月第1版
[42] 吴旭光．系统建模和参数估计—理论与算法．机械工业出版社．2002年8月．
[43] 杨海威，基于结构的神经网络建模与优化方法在复杂M-K-B系统中的应用研究．上海交通大学2003年博士学位论文．
[44] 倪小华；王永清：李慧：杨明皓；一种实用的不良数据处理方法．现代电力，Modern Electric Power，2006年04期．
[45] 张冰；贺禹；数据采集和智能数据处理系统的分析和设计．计算机工程与设计，Computer Engineering and Design，2004年06期．
[46] S.Renganathan,J.Stanly Johnson.A Robust Adaptive Controller Using Fuzzy Logic Approach. 1995 IEEE/PAC Cement Industry Technical Conference,95-99
[47] T.Tomohiro. Fuzzy Identification of Systems and its applications to modeling and control,IEEE trans.on System,Man and Cybemetics,vol.SMC-15,pp. 116-132
[48] https://pcs.khe.siemens.com/index.aspx?nr=6836
[49] http://www.abb.com.cn/cawp/seitp 161/137bc605538cb8b2c 1256f89005483ba.aspx
[50] http://www.flsautomation.eom/ECS/FuzzyExpert-pulp recovety plants.
[51] http://www.pavtech.com/images/stories/docs/Cement
[52] Rolf Isermann. Digital Control Systema Vol 1&2,Springer Ve rlag Berlin, 1989
[53] Hiroshi Asayama, Phil Burton,Jurgen Gerstacker. Fuzzy Logic Control of a Perlite Plant.Computing&Control Engineering Journal December 1994,293-298
[54] 赵永君，田恒元，魏红光．分解炉温度自动控制系统．山东煤炭科技，1999(3)：44-46
[55] 姚维，孟睿，颜文俊，诸静．水泥回转窑生产过程的模糊控制．化工自动化及仪 表，2000，27(2)：15-18
[56] 徐德，诸静．湿磨干烧水泥生产线分解炉温度控制．硅酸盐学报，2001，29(2)：120-122
[57] 姚维．模糊控制技术在水泥回转窑分解炉温度控制中的应用．测控技术，2000，19(9)：36-40
[58] 王德富，邱文斗．自寻优多模式模糊逻辑控制算法在水泥窑分解炉温度控制中的应用．水泥科技，1999(4)：16-20
[59] 邹健，诸静．模糊预测函数控制在水泥回转窑分解炉温控系统中的应用研究．硅酸盐学报，2001，29(4)：318-321
[2] 刘志江．新型干法水泥技术．中国建材工业出版社，2005．1
[3] 王益民．我国建材工业自动化现状与发展，中国建材报2001(2)
[4] D.P. Bertsekas and J. N. Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996
[5] D.P. Bertsekas, M.L. Homer, D.A. Logan, S.D Patek, and N. R. Sandell, Missile Defense and Interceptor Allocation By Neuro-Dynamic Programming, IEEE Transactions on Systems, Man and Cybernetics, partA, 2000, Vol. 30(1), 42-51
[6] C.J.C.H.Watkins, Learning from Delayed Rewards, Ph.D. Thesis, Cambridge University, Cambridge, England, 1989.
[7] C.J.C.H. Watkins and P Dayan, Q-Learning, Machine.8, 279-292
[8] G.J. Tesauro, Practcal Issues in Temporal Difference Learning, Machine Learning, 1992, 257-277
[9] W. Zhang and T.G. Dietterich, A Reinforcement Learning Approach to Job Shop Scheduling, Proceedings of the 14th IJCAI, 1114-1220
[10] Danil V. Prokhorov, and Donald C. Wunsch, Adaptive Critic Designs. IEEE TRANSACTIONS ON NEURAL NETWORKS, VOL. 8, NO. 5, SEPTEMBER 1997
[11] Liu D, Xiong X, Zhang Y. Action-dependent adaptive critic designs, In: Proceedings of the INNS-IEEE Intemational Joint Conference on Neural Networks, Washington, DC, 2001, 7:990～995
[12] Lendaris G G, Paintz C. Training strategies for critic and action neural networks in dual heuristic progrmming method, In: Proceedings of the 1997 IEEE International Conference on Neural Networks, Houston, TX, 1997,6:712～717
[13] Cox C, Stepniewski S, Jorgensen C, Saeks R, Lewis C. On the design of a neural network autolander, International Journal of Robust Nonlinear Control, 1999, 9:1071～1096
[14] Murray J J, Cox C J, Lendaris G G, Saeks R. Adaptive dynamic programming, IEEE Transactions on Systems, Man, and Cybernetics-Part C: Applications and Reviews, 2002, 32(5): 140～153
[15] Murray J J, Cox C J, Saeks R E. The adaptive dynamic programming theorem, In: Stability and Control of Dynamical Systems with Applications, D. Liu and P. J. Antsaldins, Editors, Boston, MA: Birkhauser, 2003,379～394
[16] Xin Liul,S.N. Balakrishnan. CONVERGENCE ANALYSIS OF ADAPTIVE CRITIC BASED OPTIMAL CONTROL. Proceedings of the American Control Conference Chicago, Illinois June 2000
[17] Paul J. Werbos. Stable Adaptive Control Using New Critic Designs. arⅩⅳ:adap-org/9810001 vl 25 Sep 1998
[18] Remi Munos, Leemon C. Baird, Andrew W. Moore. Gradient Descent Approaches to Neural-Net-Based Solutions of the Hamilton-Jacobi-Bellman Equation. International Joint Conference on Neural Networks, July, 1999.
[19] REMI MUNOS. Variable Resolution Discretization in Optimal Control. Machine Learning, 49, 291-323, 2002 Kluwer Academic Publishers. Manufactured in The Netherlands.
[20] Paul J. Werbos. Backwards Differentiation in AD and Neural Nets: Past Links and New Opportunities. www.die.uchile.cl/ieee-cis/evic2005/files/AD2004Werbosv2
[21] Chang-Yun Seong, Bernard Widrow, Neural Dynamic Optimization for Control Systems, CYBERNETICS, VOL. 31, NO. 4, AUGUST 2001
[22] Van Roy B, Bertsekas D P, Lee Y, et al. Neuro-Dynamic Programming Approach to Retailer Inventory Management, Proceedings of the IEEE Conference on Decision and Control, 1997,4:4052～4057
[23] Bertsekas D P, Tsitsiklis J N. Neuro-dynamic programming. Belmont: Athena Scientific, 1996
[24] Enns R, Si J. Helicopter trimming and tracking control using direct neural dynamic programming. IEEE Transactions on Neural Network, 2003, 14(4):929～939.
[25] Han D.Flight control with adaptive critic neural network. University of Missouri-Rolla. 2001.
[26] White D A,Sofge D A. Handbook of intelligent control -neural, fuzzy and adaptive approaches. New York: Van Nostrand Reinhold, 1992.
[27] Miller W T, Sutton R S, Werbos P J. Neural networks for control. Cambridge: MIT Press, 1990.
[28] Si J, Barto A, Powell W, et al.Handbook of learning and approximate dynamic programming. New York: John Wiley&Sons2004
[29] Mas P, Brooks R. Learning to coordinate behaviors. Proceedings of the 8th AAAI, Morgan Kaufmann, 1990,796～802
[30] Martin T.Hagan, Howard B.Demuth. Neural Network Design. China Machine Press.2002.
[31] Kalmar Z, Szepesvari C, Lorincz A. Module-based reinforcement learning: experiments with a real robot. Proceedings of the 6th European Workshop on Learning Robots. Machine Leaning, 1998,34:55～85
[32] Excelente-Toledo C B, Jennings N R. Learning when and how to coordinate. Web Intelligence and Agent System, 2003,1(3/4):203～218
[33] C.Anderson. Approximating a policy can be easier than approximating a value function. Technical Report CS-00-101, Colorado State University, 2000
[34] C. Anderson. R.M. Kretchner,EM Young, and D.C.Hittle. Robust reinforcement learning control with static and dynamic stability. International Journal of Robust and Nonlinear Control, 2001
[35] Z. Hung and S. N. Balakrishnan. Robust adaptive critic based neurocontrollers for missiles with model uncertainties. 2001 AAA Guidance, Navigation and control Conference, Montreal, Canada, 2001
[36] Z. Hung and S. N. Balakrishnan. Robust adaptive critic based neurocontrollers for systems with input uncertainties. Proceedings of IJCNN'2000, Como, Italy, 2000:B-263
[37] 望安全，陈宗海，文锋，一种基于强化学习的控制算法研究．计算机仿真，Vol．20，No．11，2003，
[38] 金辉宇，于海斌，神经元动态规划综述．信息与控制．第30卷第4期，2001年8月
[39] 陆超，谢小荣，童陆园，王仲鸿．使用直接神经动态规划方法的SVC附加阻尼控制．中国电机工程学报．第24卷第12期，2004年12月
[40] 钱征，孙亮，阮晓钢．一种基于递归神经网络的白适应控制方法研究．微计算机信息(测控白动化)2005年第21卷第11-1期
[41] 王科俊，王可成．神经网络建模、预报与控制．哈尔滨工程大学出版社．1996年12月第1版
[42] 吴旭光．系统建模和参数估计—理论与算法．机械工业出版社．2002年8月．
[43] 杨海威，基于结构的神经网络建模与优化方法在复杂M-K-B系统中的应用研究．上海交通大学2003年博士学位论文．
[44] 倪小华；王永清：李慧：杨明皓；一种实用的不良数据处理方法．现代电力，Modern Electric Power，2006年04期．
[45] 张冰；贺禹；数据采集和智能数据处理系统的分析和设计．计算机工程与设计，Computer Engineering and Design，2004年06期．
[46] S.Renganathan,J.Stanly Johnson.A Robust Adaptive Controller Using Fuzzy Logic Approach. 1995 IEEE/PAC Cement Industry Technical Conference,95-99
[47] T.Tomohiro. Fuzzy Identification of Systems and its applications to modeling and control,IEEE trans.on System,Man and Cybemetics,vol.SMC-15,pp. 116-132
[48] https://pcs.khe.siemens.com/index.aspx?nr=6836
[49] http://www.abb.com.cn/cawp/seitp 161/137bc605538cb8b2c 1256f89005483ba.aspx
[50] http://www.flsautomation.eom/ECS/FuzzyExpert-pulp recovety plants.
[51] http://www.pavtech.com/images/stories/docs/Cement
[52] Rolf Isermann. Digital Control Systema Vol 1&2,Springer Ve rlag Berlin, 1989
[53] Hiroshi Asayama, Phil Burton,Jurgen Gerstacker. Fuzzy Logic Control of a Perlite Plant.Computing&Control Engineering Journal December 1994,293-298
[54] 赵永君，田恒元，魏红光．分解炉温度自动控制系统．山东煤炭科技，1999(3)：44-46
[55] 姚维，孟睿，颜文俊，诸静．水泥回转窑生产过程的模糊控制．化工自动化及仪 表，2000，27(2)：15-18
[56] 徐德，诸静．湿磨干烧水泥生产线分解炉温度控制．硅酸盐学报，2001，29(2)：120-122
[57] 姚维．模糊控制技术在水泥回转窑分解炉温度控制中的应用．测控技术，2000，19(9)：36-40
[58] 王德富，邱文斗．自寻优多模式模糊逻辑控制算法在水泥窑分解炉温度控制中的应用．水泥科技，1999(4)：16-20
[59] 邹健，诸静．模糊预测函数控制在水泥回转窑分解炉温控系统中的应用研究．硅酸盐学报，2001，29(4)：318-321