基于改进的神经网络自回归模型的非线性时间序列建模和预测
|
摘要
本文研究应用于控制的时间序列的建模和预测问题。时间序列可认为是表征系统在不同时间点动态特征或是由非线性动态方程描述的被控对象在某特定采样间隔的的一连串数据,最早可追溯到公元前5000年。时间序列广泛存在于工业过程、社会科学、金融学、医疗学、天文学、气象学、物理学、生物学、经济学、环境研究、工程等其他一些领域。对时间序列的研究对于我们的生活具有重要的意义,它们是人们了解过去和预测未来的基础工具,如为人们提供商品的买卖计划、避免货物的过剩或脱销、提供产品的需求预测、管理交通运输、探索宇宙空间等。时间序列在人类生活各个方面的成功应用归功于数学模型及其对它们的分析。时间序列分析方法一般包括提取数据有效的统计特征及其它相关特征并根据历史观测数据建立预测未来数据的模型。非线性时间序列属于一类随机顺序排列的数据,对它们的分析也与不具有随机性的普通数据分析问题不同。同时,时间序列与空间数据也有不同,它们的一个重要特性是连续的观测数据比较长时间间隔的观测数据更具有相关性,这个特性经常用以通过历史数据来预测未来数据。
最近几年,针对时间序列的分析问题,许多线性模型和非线性模型被相继提出。本文提出了四种新的模型,包括两种改进的神经网络非线性模型和两种更为一般的神经网络非线性模型与线性模型的组合模型,并将这些非线性模型作为内模应用于实际船舶的轨迹跟踪控制。分别为局部变权重径向基神经网络模型(LVW-RBFN)、基于状态相依自回归结构的局部多项式径向基神经网络模型(LPRBF-AR)、基于结构化非线性参数优化方法的局部多项式小波神经网络模型(LPWNN-SNPOM)和基于状态相依自回归模型的函数型权重小波神经网络模型(FWWNN-AR)、最后,这些全局非线性模型作为预测控制器的内模应用于实际船舶的轨迹跟踪控制。本文的主要工作和创新成果概括如下：
第一,针对非线性时间序列的建模和预测问题,提出了一种特定的径向基神经网络类模型——局部变权重径向基神经网络模型(LVW-RBFN)。局部变权重径向基神经网络可认为是普通径向基神经网络的改进,它是由输入层、输出层、隐含层和权重层构成的四层前馈人工神经网络,隐含层的每个节点即为一个径向基函数并具有一个称之为网络中心的参数矢量,网络中心用以比对径向基函数的输入以得到相应的径向几何响应,这些响应通过与网络输入相关的局部变权重系数进行缩放,最后由它们组成整个网络的输出。局部变权重径向基神经网络模型(LVW-RBFN)相对于标准径向基网络需使用更多的线性参数,但这更有利于通过隐含层和权重层获得非线性时间序列的动态特性。一种结构化的非线性参数优化方法(SNPOM)被用来估计此模型,这种方法将需要的估计的参数分为线性参数空间和非线性参数空间,并通过LMM方法优化非线性参数,通过最小二乘法优化线性参数,且在参数的每次优化迭代过程中,非线性参数(线性参数)的更新紧紧跟随着线性参数(非线性参数)的更新。最后,利用提出的LVW-RBFN模型预测了三组著名的时间序列,分别为Mackey-Glass序列、the electroencephalography (EEG)和the Box-Jenkins data(请参考甘敏论文).并与其它一些最新提出的模型进行比较,研究结果表明了该模型的优越性。
1)Mackey-Glass (MG)时间序列；为了与以前的工作保持一致,我们基于四阶龙格-库塔方法生成了两千个数据,时间步长为0.1,初始值x(0)=1.2。从生成的数据集来看,1000个数据中提取从第118到第1117个数据。前500个数据用于训练模型,而其余的500个用来测试该模型。LVW-RBF(10,4)模型被用来预测这个复杂的非线性时间序列的六步向前输出。得到的结果表明,与列表中的其他方法相比,该模型需要更少的隐含层神经元以及更少的中心就能产生更好的预测精度。它也显示,预测误差均值几乎为零,揭示了误差类似白噪声。
2)EEG时间序列,也称为脑电图,记录短时间内大脑的自发性电活动。我们使用LVW-RBF(8,5)来评估对脑电图的建模和预测性能。使用630个数据,前350个用于模型扩展,其余280个专门用于测试模型。与一些著名方法实现的结果相比,实际值与用测试集估计的输出值之差的均方根误差MSE及AIC都比较小。这表明,该模型优于那些模型。
3)Box-Jenkins煤气炉时间序列；在甲烷气体混合物燃烧过程中记录Box-Jenkins数据。这组数据众所周知,是常用的测试非线性建模与预测方法的基准。该数据集包括296组输入输出对,采样周期为9s。输入u(t)是入炉的气体流量,x(t)输出是出口气体中的二氧化碳浓度。进行了评估,通过与回归权重的RBF比较来评估我们所提出的模型的性能,u(t-4)和x(t-1)作为输入变量,x(t)作为输出变量,且考虑不同节点的数量n=(3,6,...30)。均方根误差(MSE)作为性能指标。给定相同的数量或有时给更少的隐含层神经元,该模型比回归权RBF模型取得更好的性能。
第二,为了提高LVW-RBF的性能和进一步减少中心数量,本文提出基于局部多项式RBF神经网络的状态相依AR (LPRBF-AR)模型。LPRBFN是RBF的改进型,它的权重由输入的多项式函数表示。与LLRBFN不同之处在于,它的每个权重包含一个自由参数且是一个幂函数,其幂为正整数,而LLRBFN每个权重的幂等于1。它与回归权重RBF的不同之处在于,其权重不包含任何误差。实际上该误差被添加到网络输出。而回归权重RBF的每个权重的幂等于1,与LLRBFN情况一样。结构化非线性参数优化方法(SNPOM)用于估计该模型的参数。我们所提出的模型的性能和有效性用Mackey-Glass时间序列、Lorenz混沌时间序列、EEC和Box-Jenkins煤气炉时间序列评估。
1)Mackey-Glass(MG)时间序列：对两种不同的情况进行了研究。第一种情况下,选取τ=20,生成1000个数据,其中前500个数据用于训练模型,后500个用于测试该模型。从而得到三种不同滞后时间的LPRBF-AR模型的一步前向预测误差。结果显示,与具有相同滞后时间、相同数量的节点且每个节点具有相同数量中心的模型相比,LPRBF-AR模型能够得到最小的预测误差和AIC。此外,具有相同数量节点以及每个节点的中心数时,LPRBF-AR比RBF-AR模型表现更好,且滞后时间更短。与一些知名模型相比,LPRBF-AR表现出最好的模型和预测精度。第二种情况下,考虑与(1)中相同的数据,比较LPRBF-AR(4,10,4)与LPRBF-AR(4,8,4)两种模型,获取六步前向预测,并将它们的模型和预测性能与早期出版物中所做的工作进行比较。跟它们相比,我观察到LPRBF-AR需要更少的隐藏神经元与中心数来达到好得多的预测精度。
2)洛伦茨吸引子是洛伦茨振子的长期行为对应的分形结构。洛伦茨振子是一种能产生混沌流的三维动力系统。这一洛伦茨模型不只对非线性数学有重要性,对于气候和天气预报来说也很重要。声明变量σ,ρ,二者均为正数。为与早期的工作进行公平的比较,选取普朗特数α=10,瑞利数ρ=28,常数β=3/8。通过调节这些参数,并设置采样时间为0.01s,用55秒的时间来生成一个洛伦茨吸引子的长期轨迹,从而创建了一个单变量时间序列。为避免瞬态响应,应选取第35s到55s之间的数据集,得到2500个数据构成的集合。前1500个数据用于训练,后1000个用于测试。性能指标选取标准均方误差(Normalized Mean Square Error, NMSE)。将LPRBF-AR(4,3,2)和LPRBF-AR(5,3,2)的性能与其他知名方法进行比较。实际输出与估计输出的NMSE如下：使用训练集的LPRBF-AR(4,3,2)为1.53×10-12,LPRBF-AR(5,3,2)为1.10×10-14,而使用测试集的LPRBF-AR(4,3,2)为2.96×10-12,LPRBF-AR(5,3,2)为3.650×10-14。与其他方法相比,这些NMSE很小。可见LPRBF-AR(4,3,2)的表现比除了LPRBF-AR(5,3,2)之外的其他方法都要好。因此与其他模型相比,LPRBF-AR需要更少的中心数目。所提出模型表现出来的好的统计特性同样证明了模型能够有效地描述原始数据的动态特性。因此提供了一个更加精确的预测结果。
3)对于EEG时间序列(正如(1)中描述的一样),LPRBF-AR (8,3,4)与其他所列举的算法相比有更小的预测误差和AIC。这证明通过LPRBF来逼近状态自回归方程的系数比目前所知道的其他类型的SD-AR和RBF算法有更加优越的性能。其次,训练数据和测试数据的预测误差都接近于白噪声。而且对于模拟震动时间序列常常发生大的建模误差都没有发现。所有的这些结果都显示LRBF-AR模型能够很好的揭示原始数据的动态性能。
4)此部分给出了两个关于Box-Jenkins燃气炉时间序列的例子分析。第一个例子中,(1)给出了数据和条件。通过对比我们的模型和regressive weights RBF的建模结果,我们发现：所提出模型的MSE和这些regressive weights RBF模型一样,随着结点数目的增多而减少。而且相比其他的regressive weights RBF模型,LPRBF-AR模型只需要较少的隐含层节点数来取得更好的建模性能。在第二个例子中,考虑和(1)一样的原始数据,为了更加公平的对比我们的模型和其他模型的优缺点,我们采用同样的输入和相关条件。
对比建模结果显示：LPRBF-AR有更小的训练数据均方误差。其次,LPRBF-AR有最小的测试数据均方误差。这表明,相比所列的其他模型,我们的模型优越不仅表现在建模上,而且表现在预测未知情况的数据上。同样能看到LPRBF-AR只需要较少的节点数目便能取得很好的建模效果。
第三,此部分介绍了增强型的小波神经网络,即LPWNN-SNPOM。这是一个5层的神经网络包括了输入层、输出层,小波层,乘积层和多项式层,多项式层是作为函数的输入来计算权重的,这些权重是动态的,而且对于动态系统的学习过程很重要。LPWNN-SNPOM是WNN-HLA的推广与发展。这两个模型主要有下面的三个不同：i)LPWNN-SNPOM包含了一个偏置。ii)在LPWNN-SNPOM中,我们用输入的多项式函数作为参数权重,而WNN-HLA的参数权重是不变的,这样能更好地让权重随着输入的变化而变化和反应系统的动态性能。iii)不像WNN-HLA,用在线优化的方法,LPWNN-SNPOM采用离线优化的方法,即SNPOM。所提出模型的性能和效率通过了几个例子来详细阐述了(Mackey-Glass时间序列,非线性动态系统,混沌信号,澳大利亚的气泡酒时间序列,Box-Jenkins燃气炉时间序列和太阳黑子时间序列)。这些结果都显示了所提出模型的有效性和优越性,同时证明了所提出模型在WNN-HLA有提高并且比其他著名的模型性能好。
1) Mackey-Glass时间序列。采用和(第一)中相同的数据,利用LPWNN—SNPOM (4,10)模型对Mackey-Glass这一复杂非线性时间序列的未来六个步骤的输出进行了预测。结果显示,相比其它预测技术包括如LLWNN, LLNF和PG-RBF在内的工艺水平,该模型获得了一个训练和测试数据集小很多的RMSE。
2)非线性动态系统,它可认为是一个输出非线性依赖于其过去值及输入,并附加了输入输出值影响的被控对象。为了和以前的成果进行公平对比,选用了200组数据来训练LPWNN-SNPOM(2,6)模型,以根的平均平方误差(RMSE)作为效果评估,将该模型与WNN-HLA, FALCON, SONFIN和SCFNN模型进行了比较。据观察可得,LPWNN-SNPOM (2,6)模型所产生的预测误差远远小于上文所提到的其它模型。另外还可得知,由于该模型的权重结构,该模型获得较好建模效果所需的参数和迭代次数更少。
3)混沌信号。将LPWNN-SNPOM模型应用在一个从等式衍生出来的简单混沌信号的典型一步向前预测问题上。令系数a=3.6,x(1)=0.001,首先生成60个数据的训练集。再令a=3.8,初始值x(1)=0.9,生成100个数据用于模型的验证。LPWNN-SNPOM(2,2)模型则用来预测该复杂混沌时间序列的一步向前输出。据观察可得,LPWNN-SNPOM(2,2)模型的预测误差在经过多次迭代之后可以减小到一个非常小的值。它表明,虽然相对小波,权重有更高的振幅,但两者都对该非线性时间序列的动态表现有贡献,而且该模型几乎完全符合原始输出。结果显示,与其他所列的预测技术相比,该模型只需要很少的隐含神经元和小波基就能获得很好的预测精度。
4)非线性函数的逼近。在这部分,通过使用二维函数对该方法的性能进行了研究。为了公平对比,从均匀分布在D=[10,10]×[10,10]范围内的一个随机输入中采集了400个数据作为训练集。根据WNN-HLA,RBF,WNN,WN以及LPWNN-SNPOM模型的比较结果可知,相比其他方法,该模型用更少的参数和迭代次数就可以获得更好的模型精度。这样的性能主要是由于非线性函数中新的小波结构的代表能力以及最优化技术对该模型的训练效率。
5)澳大利亚一个烈酒公司的气泡酒时间序列：这个时间序列是从http://robjhyndman.com/tsdldata/data/sparklng.dat获得,为了在实际数据的基础上测试这个设定模型的结果。这个时间序列有很强的年度季节性,可以在设计一个更好的合适的模型中得以应用。在这部分中,使用了一个有固定输入向量的LPWNN-SNPOM(2,2)模型去预测这个非线性时间序列的一步前向输出。两个流行的策略即均方差(MSE)和平均绝对误差百分比(MAPE),都作为使用指标。通过假定模型获得的结果和通过SARIMA模型获得的结果进行相对照,可以看到除了在第六年通过SARIMA模型获得比较好的均方差外,其他年份设定模型比SARIMA模型获得的结果都要好。如果在设定模型的基础上,澳大利亚烈酒公司将会获得更大的经济效益。
6)Box-Jenkins煤气炉时间序列：如在(第一)中考虑的一样。这主要目的是把设定模型应用起来获得一步前向预测,建立在y(t-1),y(t-2),y(t-3),y(t-4),u(t-1),u(t-2),u(t-3),u(t-4),u(t-5)和u(t-6)基础上。这样的输入结构使用的更广泛。为了使和其它使用同样输入结构的方法公平对照,数据集减到了290。前145个数据用来训练设定模型而剩下的145个数据用来在还不能看到的一些数据的基础上测试它们的预测能力。获得的结果表明,在与其他例举出来用于比较的预测技术相比,所提出模型需要一个很小数量的隐藏神经元来获得更好的预测精度,于是提出了一个LPWNN-SNPOM(10,6)模型。也有人指出,该模型只需要几个步骤去汇合。
7)太阳黑子时间序列：太阳黑子是出现在太阳光球部位的一种与周围区域相比起来有明显黑斑的瞬时自然现象。它们是由于强磁场活动引起的,该磁场活动阻止了边缘涡流产生的对流,形成了一些表面温度降低的区域。太阳黑子的数目是太阳活动的一个很好的检测,并且太阳黑子活动的周期是11年,称为太阳活动周期。太阳活动对地球,气候,天气,卫星以及太空任务都有强烈的影响。因此预测太阳黑子时间序列是非常重要的。然而,由于系统的复杂性和数据模型的缺乏,预测太阳周期不是一个简单的任务。为了使设定模型与现有的模型在同等条件下相比较,选择了从1834年11月到2001年六月的太阳黑子时间序列,并且使2000个点缩放到[0,1]这个范围,其中前1000个点用来训练设定模型,而剩下的1000个点用来在不能看到的情况下测试设定模型的预测准确性。在LPWNN-SNPOM(4,2)里面带有不同的季节性。在这个例子中,季节性因素分别标记为1,3,6和12,分别代表月,季度,期和年。把1分配到季节性因素将会使预测超前一步,所以在太阳黑子时间序列中常常被使用。为了评价在测试数据基础上的设定模型的一步前向预测结果和与在文献中汇报的结果以及看到该模型的季节性影响,均方差(MSE)、二次均方差(RMSE)和规范均方差(NMSE)作为性能指标。在文献中提到的一步前向预测差,MSE,RMSE和NMSE的比较以及设定模型对1000个太阳黑子的测试例子表明：当设定方法用来预测太阳黑子时间序列的未来值时,与另一种提到的预测方法相比设定方法能产生更好的结果。在测试例子中,对于四个不同的季节性即月、季度、期和年的1000个太阳黑子时间序列的误差指数,MSE,RMSE和NMSE的结果表明,当与月相比,季度、期和年展示了一个巨大的跌幅。总之,可以看到期季度有一个更小的误差指数,这说明太阳黑子有个强烈的期季度可以在预测过程中探索。
第四,提出了用一种称为基于状态相依AR模型的函数权重小波神经网络(FWWNN-AR)模型来对非线性时间序列进行建模和预测。这种模型利用改进的函数型权重小波神经网络(FWWNN)来逼近状态相依AR模型的状态相依系数,使其结合了状态相依AR模型利用少量节点描述非线性动态特性和FWWNN对时域和频域特性的函数逼近能力的优势。FWWNN是包含有输入层、输出层、小波层、规划层和多项式层的五层小波神经网络,其中多项式层是与输入有关的函数,用以计算输出的动态权重。该模型通过三个不同层次来学习非线性时间序列的动态特性,分别为AR层、小波网络层和函数型权重层。函数型权重层可使用不同的基函数,本文选用标准的WNN为函数型权重层的函数变量,从而使得FWWNN-AR模型的性能总优于WNN-AR模型,因此,FWWNN-AR模型可认为是基于WNN-AR模型的改进,FWWNN-AR的线性形式可通过选择极大平移因子来得到。对数值实验和实际非线性时间序列的预测效果表明了该模型的有效性和优越性。
1)扭曲长记忆AR时间序列；DLM绘制出市场或经济泡沫。为进行一次适当比较,我们采用在之前已发表的文章中所应用的同一数据。此时间序列由500个数据点组成,其中的400个数据点用于训练模型,而剩余的100个数据点用于检验所提出模型在未知情况下的性能。主要任务是利用含有3种不同权重基函数的FWWNN-ARX(4,2,2)模型来得到DLM的一步前向预测。所得结果显示出,作为最好模型出现的FWWNN-AR1模型而言,所提出的模型有更好的性能。并且所提出的模型能够超越WNN-AR模型,这也证明了所提出的模型是基于WNN-AR模型的改进。这些性能一部分是因为所提出模型权重分担非线性与小波间隔的能力。此外,对于扰动数据,所提出模型所获得预测精确的水平显示出其对于扰动数据的鲁棒性。
2) Mackey-Glass时间序列；在这一部分,两个方案和如在(2)一样的条件下进行研究,在第一种方案,通过利用不同的延迟得到的结果和那些在参考文献中的模型得到的结果进行比较来计算这个模型的性能值。三种不同权值基函数用于进行比较。结果表明当考虑同一滞后时,和RBF, RBF-AR, WNN-AR以及LLRBF-AR模型比较,所提出模型不仅会有一个更小的预测误差,而且有更小的AIC值。这也表明所提出模型比RBF, RBF-AR, WNN-AR以及LLRBF-AR模型需要更少的时间序列延迟来使性能达到更好。此外,我们可以看出FWWNN-AR模型未知数据的预测精度会降低为标准WNN-AR模型未知数据预测精度的40,47%,38,78%和56,21%分别对于5,6和7个滞后。这也证实FWWNN-AR模型是基于WNN-AR模型的一个改进。在第二个方案,此项方案的任务是从一个固定的输入中预测出y(t+6)。和其他例举的方法相比较,所提出模型能够得到更好的预测精度。在学习的过程中,通过SNPOM对基于3种权值基础函数的FWWNN-AR模型预测误差的反复估算,使预测误差会减小到一个很小的值。
3)洛伦兹时间序列：如(2)中使用的洛伦兹因子,本章节中,主要工作是通过FWWNN-ARX(5,2,2)三种不同的权重因子获得非线性时间序列一步预测。NMSE用来训练和测试参考文献中的这些关于相同时间序列的模型,并且用来做对比。与别的模型对比可以看出FWWNN-ARX模型只需要更少的节点就能得到最小的方差。结果显示了很好地统计特性也证明了处理动态原始数据更有效率。FWWNN-AR模型优于其他的模型,这个模型减少了98.90%检验误差,而WNN-AR模型只减少了97.86%。
4)太阳黑子时间序列：如(3)中所描述的数据。在这个例子中一步向前预测使用FWWNN-ARX模型使用三种不同权重函数。MSE,RMSE和NMSE测试前面模型得来的太阳黑子时间序列,用来与参考文献中其他的一些经典方法做对比。对照可以发现在模型中输入层、隐含层、输出层均只要一个神经元,AR时间延迟为4。而第二个模型输入层需要五个神经元,隐含层需要7个,输出层需要5个神经元和7个recurrent neurons.因此本模型只需要更少的非线性参数就可以得到比其他模型更好的结果。在例子中,提出的所有的模型得到了相似的结果,同时也比标准的WNN-AR和其他模型效果更好。
5)Australian sparkling wine time series:这个时间序列和章节(2)中提到一样,在2.2.2FWWNN-ARX中使用三种不同的权重的基函数。通过连续五年的平均绝对误差百分比(MAPE)和均方误差(MSE),并且按月,半年,一年得到平均值,以及与SARIMA的年平均作对比。从中得出本模型的半年平均和年平均有更好的效果,即使本模型中使用的是半年的数据值而SARIMA采用的年平均值。但是也有一些年份SRIMA模型有更好的效果。此外,所有模型中效果最好的年平均值,这是SARIMA作者的建议,数据按年来做计算。按照年数据FWWNN-AR1、FWWNN-AR4、FWWNN-AR5和SARIMA对比MAPE和MSE参数,FWWNN-AR5是效果最好的模型。该模型比标准的WNN-AR有着更好的效果,进一步证明了是比WNN-AR更好的模型。
6)煤气炉时间序列：尽管煤气炉的数据可以近似线性的输入-输出行为,然而很多非线性的建模方法用在对线性模型的建模和预测上。这种异常主要是由输入结构引起的。这个例子的主要目的是对于这些线性数据,本文提出的模型与其他的线性与非线性模型的效果对比。为了对比,这里使用图3中的数据,同时列出了一些比较所用到的条件。总共用到了五组测试数据,五组数据的输入及用来测试和检验的数据长度不同。模型的输入包括过去的输入及输出。每一个例子使用本文提出的模型进行一步预测。第一个例子用y(t-1)和u(t-4)作为模型的输入来计算y(t),u1,(x(t-q))=x(t-q)2用来衡量基函数的权重。
从AIC来看,本文提出的线性模型的AIC明显大于非线性模型。这说明非线性模型对这些数据的建模效果更好。这里同样列出了本文提出的非线性模型与其他的非线性模型的对比结果。第二个例子使用,y(t-1),u(t-3)及u(t-4)作为模型的输入来获得模型的输出y(t),u1(x(t-q))=x(t-q)2用来衡量基函数的权重。同样可以看到线性模型的AIC大于非线性模型的AIC。从结果可以看出来,非线性模型的更加适合用于对煤气炉的建模及预测。此外,与其他的非线性模型相比较,在获得同样的精度的条件下,本文提出的非线性模型使用了更少的神经网络节点。在第三个例子中使用y(t-1),u(t-3),u(t-4)和u(t-5)作为模型的输入,u1(x(t-q))=x(t-q)2用来衡量基函数的权重。同样可以看出,本文提出的线性模型的AIC比本文所提出的非线性模型的AIC要大,非线性模型更加适合对煤气炉模型进行建模和预测。同样可以看到本文提出的非线性模型的MSE是0.086,而Nie所提出来的模型的MSE是57.98。在第四个例子中使用y(t-1),u(1-2),u(t-3),u(t-5)和u(t-6)作为模型的输入,u1(x(t-q))=x(t-q)2用来衡量基函数的权重。前200个数据用来训练模型,后40个数据用来测试模型。获得的线性模型的AIC与非线性模型比较,在训练数据中线性模型的AIC较大,而训练数据中的线性模型的AIC较小。这意味着线性模型比非线性模型更加适合。这是因为输入和输出有很强的线性结构。因而在这个例子中提出的线性模型的MSE比非线性模型的MSE更小。最后一个例子中使用y(t-1),y(t-2),y(t-3),y(t-4),u(t-1),u(t-2),u(t-3), u(t-4),u(t-5)和u(t-6)。这种结构广泛使用。u1(x(t-q))=x(t-q)11用来衡量基函数的权重。为了跟以前的工作进行比较,使用的数据为290个,前145个数据用来训练本文提出的模型。可以看出来本文提出的线性模型的AIC远大于非线性模型的AIC。这意味着对于这一组输入的煤气炉的数据,非线性模型更加适合用来建模和预测。此外,本文提出的非线性模型因为只用到了一层隐含层,因而用到的参数比其他的模型更少。与第二好的模型对比,FWWNN-AR1(1,1,10)模型在训练数据上的MSE比第二好的模型小77.34%,在测试数据上比第二好的模型小36.74%。有这么好的结果,主要是因为本文提出的模型的权重是动态的。
最后,将提出的FWWNN-AR模型在控制中进行了应用,一种基于ARX模型的扩展指数型小波神经网络(EW-WNN-ARX)模型作为模型预测控制的内模用以对船舶进行控制。为了得到稳定的系统,用航向偏离角角速度作为模型输出,建立EW-WNN-ARX模型来描述动态船舶航向偏离角和舵角之间的差异。为了描述船舶的动态非线性特征,滚动角作为状态特征量用以引导模型参数变化以反映船舶的运动状态。采用SNPOM离线辨识EW-WNNN-ARX模型以避免在线优化带来的诸多问题,估计出的模型实质是一个以航向偏离角描述船舶位置轨迹误差特征的数学模型,这种模型是统计模型和数学模型的结合体,称为E-EW-WNN-ARX-MM模型。通过该模型可建立起船舶运动的状态空间模型并将其作为模型预测控制(MPC)的内模以实现船舶跟踪期望轨迹连续运动。E-EW-WNN-ARX-MM数据来自于Shioji-MARU experimental ship of Tokyo University of Marine Science and Technology of Japan。建模与控制结果表明了基于离线辨识的E-EW-WNN-ARX-MM能描述大范围过程的全局非线性特性。
Time series are an integral part of our lives. They are found in the field of social and natural sciences, engineering and economy. The need for their analysis, modeling, prediction and control has become more urgent. Generally, the main objective of time series analysis is to develop models that can establish the relation between variables. To serve these purposes, various models ranging from linear to nonlinear have been developed throughout the time. Usually, when the relation among different time series variables is linear, linear models are more suitable. On the other hand, when this relation is nonlinear, nonlinear models are more suitable. Linear models are easier to identify and implement. However, most real world systems are governed by nonlinear rules, hence, the importance of nonlinear modeling technique in dealing with time series.
In this thesis, the problematic of modeling and prediction of time series together with the control issues are considered. Four new models were developed including two fully nonlinear models that are enhanced type of neural networks and two more generalized models that are a combination of enhanced type of neural network with a linear model, thus having both a linear and nonlinear capability. Furthermore, a generalized nonlinear model is used as internal model to address the tracking issue of the real ship. The main work and innovations of this thesis are summarized as follows.
Firstly, a special type of RBF namely the Local Variable-Weights RBF Network (LVW-RBFN) was developed to investigate the modeling and forecasting problem of nonlinear time series. The LVW-RBFN is a four layered RBFN comprising of input layer, hidden layer, weight layer and output layer. It is an enhanced type of RBF network, in which the constant weights that connect the hidden layer with the output layer in the standard RBFN are replaced by functions of the RBFN's inputs computed via the weight layer. A structured nonlinear parameter estimation method (SNPOM) is applied to estimate the model. Case studies on several benchmark time series show that the prediction performance of the LVW-RBFN model is superior to other newly developed models including the RBF thus confirming that the proposed model is an enhanced type of RBF.
Secondly, the modeling and prediction problems of nonlinear time series are addressed using radial basis function network with polynomial type weights (LPRBF) to approximate the functional coefficients of the general state-dependent autoregressive (SD-AR) model. The LPRBF differs from the RBF by its weight form; the LPRBF has polynomial type weights, which are dynamic and vary with the network input, while the weights of the RBF are constants they are not affected by the change in the input of the network. The LPRBF is a more generalized form of RBF. It may be considered as an improved version of the RBF. Thus the LPRBF.-AR can be regarded as an enhanced version of previous RBF-AR model and local linearly RBF based AR (LLRBF-AR) model. A fast-converging estimation method known as the structured nonlinear parameter optimization method (SNPOM) is applied to estimate the LPRBF-AR. Case studies on various time series and chaotic systems show that the LPRBF-AR modeling approach exhibits much better prediction accuracy compared with some other existing methods.
Thirdly, an enhanced type of wavelet neural network, known as Local Polynomial Wavelet Neural Network with a Structured Nonlinear Parameter Optimization Method (LPWNN-SNPOM) is introduced. This is a five layered wavelet neural network comprising of input, output, wavelet, product and polynomial layers, which computes the weight as a function of inputs. These weights are dynamic and play an active part in the prediction. The LPWNN-SNPOM is an improvement of the Wavelet Neural Network with a Hybrid Learning Approach (WNN-HLA). These two models have mainly three differences:i) The LPWNN-SNPOM method contains a bias, ii) The single parameter weights connecting the hidden layer with the output in the WNN-HLA are replaced by polynomial functions of the inputs in the LPWNN-SNPOM, allowing the weights to vary with the changes in the inputs and share the dynamics with the wavelet compartment, iii) Unlike the WNN-HLA, which uses an online optimization approach, the LPWNN-SNPOM makes usage of an offline optimization approach known as the Structured Nonlinear Parameter Optimization Method (SNPOM). The performance and effectiveness of the proposed model are illustrated using several examples, whose results show the feasibility of the proposed model and demonstrate that it improved upon the WNN-HLA and performed better than some other well-known models.
Fourthly, the problems of modeling and prediction of time series are investigated with a newly introduced model namely the functional weights wavelet neural network-based state-dependent AR (FWWNN-AR) model. This model consists of combining an enhanced WNN named as a Functional Weights Wavelet Neural Network (FWWNN) to approximate the state dependant coefficients of the AR. The FWWNN-AR model possesses both the advantages of the state-dependent AR model in the description of nonlinear dynamics using few nodes and of the FWWN network in functional approximation considering mutually the time and frequency spaces. The FWWNN is a five-layered network structure comprising of an input layer, wavelet layer, product layer, output layer and functional weight layer. The functional weight layer computes the weight as a function of inputs, making the weights to vary with the inputs and share the dynamics with the wavelet compartment. The proposed model learns the dynamics of nonlinear time series from three distinct levels:AR level, wavelet compartment level and weights level. Different basis functions could be used as functional weight. However, in this thesis, these functions are chosen in such a way that the FWWNN has the standard WNN as one of its component which means that the FWWNN-AR will always perform better than the WNN-AR. Thus the FWWNN-AR can be seen as an improvement of the WNN-AR. Furthermore, the linear version of the FWWNN-AR may be obtained by choosing the translation to tend to infinity. The proposed model is validated by comparing its performances and effectiveness with those achieved by some well-known models on both generated and real nonlinear time series.
Finally, the proposed models are investigated for the control issue. To this ends, the FWWNN-AR model, specifically the expanded exponential weight wavelet network based ARX (EW-WNN-ARX) model, which is used as based for developing the internal model of MPC with the aim to control a maritime vehicle. In order to achieve an almost stationary system, the difference of heading angle deviation is used instead of the heading angle. The EW-WNN-ARX model is used to describe the ship motion between the difference of heading angle deviation and the rudder angle of the ship. To represent the ship motion's nonlinearity, rolling angle is used as model index to allow the model parameters vary with ship moving state. After identification of The EW-WNN-ARX by an offline method known as SNPOM to avoid a possible failure that may occur during online identification procedure, the difference of heading angle deviation are expanded and integrated into mathematical model (MM) to characterize the position tracking error of the ship. The model resulting is a combination of statistical and mathematical model namely the E-EW-WNN-ARX-MM model. The E-EW-WNN-ARX-MM model is then used to develop the state-space type tracking motion model which is used as internal model of the MPC to steer ship moving forward with constant velocity along a predefined reference path. The E-EW-WNN-ARX-MM is developed using data obtained from the Shioji-MARU experimental ship of Tokyo University of Marine Science and Technology of Japan. The modeling and control results achieved demonstrate that the proposed E-EW-WNN-ARX-MM-identified off-line can describe the global nonlinear property of the process over a wide region.
最近几年,针对时间序列的分析问题,许多线性模型和非线性模型被相继提出。本文提出了四种新的模型,包括两种改进的神经网络非线性模型和两种更为一般的神经网络非线性模型与线性模型的组合模型,并将这些非线性模型作为内模应用于实际船舶的轨迹跟踪控制。分别为局部变权重径向基神经网络模型(LVW-RBFN)、基于状态相依自回归结构的局部多项式径向基神经网络模型(LPRBF-AR)、基于结构化非线性参数优化方法的局部多项式小波神经网络模型(LPWNN-SNPOM)和基于状态相依自回归模型的函数型权重小波神经网络模型(FWWNN-AR)、最后,这些全局非线性模型作为预测控制器的内模应用于实际船舶的轨迹跟踪控制。本文的主要工作和创新成果概括如下：
第一,针对非线性时间序列的建模和预测问题,提出了一种特定的径向基神经网络类模型——局部变权重径向基神经网络模型(LVW-RBFN)。局部变权重径向基神经网络可认为是普通径向基神经网络的改进,它是由输入层、输出层、隐含层和权重层构成的四层前馈人工神经网络,隐含层的每个节点即为一个径向基函数并具有一个称之为网络中心的参数矢量,网络中心用以比对径向基函数的输入以得到相应的径向几何响应,这些响应通过与网络输入相关的局部变权重系数进行缩放,最后由它们组成整个网络的输出。局部变权重径向基神经网络模型(LVW-RBFN)相对于标准径向基网络需使用更多的线性参数,但这更有利于通过隐含层和权重层获得非线性时间序列的动态特性。一种结构化的非线性参数优化方法(SNPOM)被用来估计此模型,这种方法将需要的估计的参数分为线性参数空间和非线性参数空间,并通过LMM方法优化非线性参数,通过最小二乘法优化线性参数,且在参数的每次优化迭代过程中,非线性参数(线性参数)的更新紧紧跟随着线性参数(非线性参数)的更新。最后,利用提出的LVW-RBFN模型预测了三组著名的时间序列,分别为Mackey-Glass序列、the electroencephalography (EEG)和the Box-Jenkins data(请参考甘敏论文).并与其它一些最新提出的模型进行比较,研究结果表明了该模型的优越性。
1)Mackey-Glass (MG)时间序列；为了与以前的工作保持一致,我们基于四阶龙格-库塔方法生成了两千个数据,时间步长为0.1,初始值x(0)=1.2。从生成的数据集来看,1000个数据中提取从第118到第1117个数据。前500个数据用于训练模型,而其余的500个用来测试该模型。LVW-RBF(10,4)模型被用来预测这个复杂的非线性时间序列的六步向前输出。得到的结果表明,与列表中的其他方法相比,该模型需要更少的隐含层神经元以及更少的中心就能产生更好的预测精度。它也显示,预测误差均值几乎为零,揭示了误差类似白噪声。
2)EEG时间序列,也称为脑电图,记录短时间内大脑的自发性电活动。我们使用LVW-RBF(8,5)来评估对脑电图的建模和预测性能。使用630个数据,前350个用于模型扩展,其余280个专门用于测试模型。与一些著名方法实现的结果相比,实际值与用测试集估计的输出值之差的均方根误差MSE及AIC都比较小。这表明,该模型优于那些模型。
3)Box-Jenkins煤气炉时间序列；在甲烷气体混合物燃烧过程中记录Box-Jenkins数据。这组数据众所周知,是常用的测试非线性建模与预测方法的基准。该数据集包括296组输入输出对,采样周期为9s。输入u(t)是入炉的气体流量,x(t)输出是出口气体中的二氧化碳浓度。进行了评估,通过与回归权重的RBF比较来评估我们所提出的模型的性能,u(t-4)和x(t-1)作为输入变量,x(t)作为输出变量,且考虑不同节点的数量n=(3,6,...30)。均方根误差(MSE)作为性能指标。给定相同的数量或有时给更少的隐含层神经元,该模型比回归权RBF模型取得更好的性能。
第二,为了提高LVW-RBF的性能和进一步减少中心数量,本文提出基于局部多项式RBF神经网络的状态相依AR (LPRBF-AR)模型。LPRBFN是RBF的改进型,它的权重由输入的多项式函数表示。与LLRBFN不同之处在于,它的每个权重包含一个自由参数且是一个幂函数,其幂为正整数,而LLRBFN每个权重的幂等于1。它与回归权重RBF的不同之处在于,其权重不包含任何误差。实际上该误差被添加到网络输出。而回归权重RBF的每个权重的幂等于1,与LLRBFN情况一样。结构化非线性参数优化方法(SNPOM)用于估计该模型的参数。我们所提出的模型的性能和有效性用Mackey-Glass时间序列、Lorenz混沌时间序列、EEC和Box-Jenkins煤气炉时间序列评估。
1)Mackey-Glass(MG)时间序列：对两种不同的情况进行了研究。第一种情况下,选取τ=20,生成1000个数据,其中前500个数据用于训练模型,后500个用于测试该模型。从而得到三种不同滞后时间的LPRBF-AR模型的一步前向预测误差。结果显示,与具有相同滞后时间、相同数量的节点且每个节点具有相同数量中心的模型相比,LPRBF-AR模型能够得到最小的预测误差和AIC。此外,具有相同数量节点以及每个节点的中心数时,LPRBF-AR比RBF-AR模型表现更好,且滞后时间更短。与一些知名模型相比,LPRBF-AR表现出最好的模型和预测精度。第二种情况下,考虑与(1)中相同的数据,比较LPRBF-AR(4,10,4)与LPRBF-AR(4,8,4)两种模型,获取六步前向预测,并将它们的模型和预测性能与早期出版物中所做的工作进行比较。跟它们相比,我观察到LPRBF-AR需要更少的隐藏神经元与中心数来达到好得多的预测精度。
2)洛伦茨吸引子是洛伦茨振子的长期行为对应的分形结构。洛伦茨振子是一种能产生混沌流的三维动力系统。这一洛伦茨模型不只对非线性数学有重要性,对于气候和天气预报来说也很重要。声明变量σ,ρ,二者均为正数。为与早期的工作进行公平的比较,选取普朗特数α=10,瑞利数ρ=28,常数β=3/8。通过调节这些参数,并设置采样时间为0.01s,用55秒的时间来生成一个洛伦茨吸引子的长期轨迹,从而创建了一个单变量时间序列。为避免瞬态响应,应选取第35s到55s之间的数据集,得到2500个数据构成的集合。前1500个数据用于训练,后1000个用于测试。性能指标选取标准均方误差(Normalized Mean Square Error, NMSE)。将LPRBF-AR(4,3,2)和LPRBF-AR(5,3,2)的性能与其他知名方法进行比较。实际输出与估计输出的NMSE如下：使用训练集的LPRBF-AR(4,3,2)为1.53×10-12,LPRBF-AR(5,3,2)为1.10×10-14,而使用测试集的LPRBF-AR(4,3,2)为2.96×10-12,LPRBF-AR(5,3,2)为3.650×10-14。与其他方法相比,这些NMSE很小。可见LPRBF-AR(4,3,2)的表现比除了LPRBF-AR(5,3,2)之外的其他方法都要好。因此与其他模型相比,LPRBF-AR需要更少的中心数目。所提出模型表现出来的好的统计特性同样证明了模型能够有效地描述原始数据的动态特性。因此提供了一个更加精确的预测结果。
3)对于EEG时间序列(正如(1)中描述的一样),LPRBF-AR (8,3,4)与其他所列举的算法相比有更小的预测误差和AIC。这证明通过LPRBF来逼近状态自回归方程的系数比目前所知道的其他类型的SD-AR和RBF算法有更加优越的性能。其次,训练数据和测试数据的预测误差都接近于白噪声。而且对于模拟震动时间序列常常发生大的建模误差都没有发现。所有的这些结果都显示LRBF-AR模型能够很好的揭示原始数据的动态性能。
4)此部分给出了两个关于Box-Jenkins燃气炉时间序列的例子分析。第一个例子中,(1)给出了数据和条件。通过对比我们的模型和regressive weights RBF的建模结果,我们发现：所提出模型的MSE和这些regressive weights RBF模型一样,随着结点数目的增多而减少。而且相比其他的regressive weights RBF模型,LPRBF-AR模型只需要较少的隐含层节点数来取得更好的建模性能。在第二个例子中,考虑和(1)一样的原始数据,为了更加公平的对比我们的模型和其他模型的优缺点,我们采用同样的输入和相关条件。
对比建模结果显示：LPRBF-AR有更小的训练数据均方误差。其次,LPRBF-AR有最小的测试数据均方误差。这表明,相比所列的其他模型,我们的模型优越不仅表现在建模上,而且表现在预测未知情况的数据上。同样能看到LPRBF-AR只需要较少的节点数目便能取得很好的建模效果。
第三,此部分介绍了增强型的小波神经网络,即LPWNN-SNPOM。这是一个5层的神经网络包括了输入层、输出层,小波层,乘积层和多项式层,多项式层是作为函数的输入来计算权重的,这些权重是动态的,而且对于动态系统的学习过程很重要。LPWNN-SNPOM是WNN-HLA的推广与发展。这两个模型主要有下面的三个不同：i)LPWNN-SNPOM包含了一个偏置。ii)在LPWNN-SNPOM中,我们用输入的多项式函数作为参数权重,而WNN-HLA的参数权重是不变的,这样能更好地让权重随着输入的变化而变化和反应系统的动态性能。iii)不像WNN-HLA,用在线优化的方法,LPWNN-SNPOM采用离线优化的方法,即SNPOM。所提出模型的性能和效率通过了几个例子来详细阐述了(Mackey-Glass时间序列,非线性动态系统,混沌信号,澳大利亚的气泡酒时间序列,Box-Jenkins燃气炉时间序列和太阳黑子时间序列)。这些结果都显示了所提出模型的有效性和优越性,同时证明了所提出模型在WNN-HLA有提高并且比其他著名的模型性能好。
1) Mackey-Glass时间序列。采用和(第一)中相同的数据,利用LPWNN—SNPOM (4,10)模型对Mackey-Glass这一复杂非线性时间序列的未来六个步骤的输出进行了预测。结果显示,相比其它预测技术包括如LLWNN, LLNF和PG-RBF在内的工艺水平,该模型获得了一个训练和测试数据集小很多的RMSE。
2)非线性动态系统,它可认为是一个输出非线性依赖于其过去值及输入,并附加了输入输出值影响的被控对象。为了和以前的成果进行公平对比,选用了200组数据来训练LPWNN-SNPOM(2,6)模型,以根的平均平方误差(RMSE)作为效果评估,将该模型与WNN-HLA, FALCON, SONFIN和SCFNN模型进行了比较。据观察可得,LPWNN-SNPOM (2,6)模型所产生的预测误差远远小于上文所提到的其它模型。另外还可得知,由于该模型的权重结构,该模型获得较好建模效果所需的参数和迭代次数更少。
3)混沌信号。将LPWNN-SNPOM模型应用在一个从等式衍生出来的简单混沌信号的典型一步向前预测问题上。令系数a=3.6,x(1)=0.001,首先生成60个数据的训练集。再令a=3.8,初始值x(1)=0.9,生成100个数据用于模型的验证。LPWNN-SNPOM(2,2)模型则用来预测该复杂混沌时间序列的一步向前输出。据观察可得,LPWNN-SNPOM(2,2)模型的预测误差在经过多次迭代之后可以减小到一个非常小的值。它表明,虽然相对小波,权重有更高的振幅,但两者都对该非线性时间序列的动态表现有贡献,而且该模型几乎完全符合原始输出。结果显示,与其他所列的预测技术相比,该模型只需要很少的隐含神经元和小波基就能获得很好的预测精度。
4)非线性函数的逼近。在这部分,通过使用二维函数对该方法的性能进行了研究。为了公平对比,从均匀分布在D=[10,10]×[10,10]范围内的一个随机输入中采集了400个数据作为训练集。根据WNN-HLA,RBF,WNN,WN以及LPWNN-SNPOM模型的比较结果可知,相比其他方法,该模型用更少的参数和迭代次数就可以获得更好的模型精度。这样的性能主要是由于非线性函数中新的小波结构的代表能力以及最优化技术对该模型的训练效率。
5)澳大利亚一个烈酒公司的气泡酒时间序列：这个时间序列是从http://robjhyndman.com/tsdldata/data/sparklng.dat获得,为了在实际数据的基础上测试这个设定模型的结果。这个时间序列有很强的年度季节性,可以在设计一个更好的合适的模型中得以应用。在这部分中,使用了一个有固定输入向量的LPWNN-SNPOM(2,2)模型去预测这个非线性时间序列的一步前向输出。两个流行的策略即均方差(MSE)和平均绝对误差百分比(MAPE),都作为使用指标。通过假定模型获得的结果和通过SARIMA模型获得的结果进行相对照,可以看到除了在第六年通过SARIMA模型获得比较好的均方差外,其他年份设定模型比SARIMA模型获得的结果都要好。如果在设定模型的基础上,澳大利亚烈酒公司将会获得更大的经济效益。
6)Box-Jenkins煤气炉时间序列：如在(第一)中考虑的一样。这主要目的是把设定模型应用起来获得一步前向预测,建立在y(t-1),y(t-2),y(t-3),y(t-4),u(t-1),u(t-2),u(t-3),u(t-4),u(t-5)和u(t-6)基础上。这样的输入结构使用的更广泛。为了使和其它使用同样输入结构的方法公平对照,数据集减到了290。前145个数据用来训练设定模型而剩下的145个数据用来在还不能看到的一些数据的基础上测试它们的预测能力。获得的结果表明,在与其他例举出来用于比较的预测技术相比,所提出模型需要一个很小数量的隐藏神经元来获得更好的预测精度,于是提出了一个LPWNN-SNPOM(10,6)模型。也有人指出,该模型只需要几个步骤去汇合。
7)太阳黑子时间序列：太阳黑子是出现在太阳光球部位的一种与周围区域相比起来有明显黑斑的瞬时自然现象。它们是由于强磁场活动引起的,该磁场活动阻止了边缘涡流产生的对流,形成了一些表面温度降低的区域。太阳黑子的数目是太阳活动的一个很好的检测,并且太阳黑子活动的周期是11年,称为太阳活动周期。太阳活动对地球,气候,天气,卫星以及太空任务都有强烈的影响。因此预测太阳黑子时间序列是非常重要的。然而,由于系统的复杂性和数据模型的缺乏,预测太阳周期不是一个简单的任务。为了使设定模型与现有的模型在同等条件下相比较,选择了从1834年11月到2001年六月的太阳黑子时间序列,并且使2000个点缩放到[0,1]这个范围,其中前1000个点用来训练设定模型,而剩下的1000个点用来在不能看到的情况下测试设定模型的预测准确性。在LPWNN-SNPOM(4,2)里面带有不同的季节性。在这个例子中,季节性因素分别标记为1,3,6和12,分别代表月,季度,期和年。把1分配到季节性因素将会使预测超前一步,所以在太阳黑子时间序列中常常被使用。为了评价在测试数据基础上的设定模型的一步前向预测结果和与在文献中汇报的结果以及看到该模型的季节性影响,均方差(MSE)、二次均方差(RMSE)和规范均方差(NMSE)作为性能指标。在文献中提到的一步前向预测差,MSE,RMSE和NMSE的比较以及设定模型对1000个太阳黑子的测试例子表明：当设定方法用来预测太阳黑子时间序列的未来值时,与另一种提到的预测方法相比设定方法能产生更好的结果。在测试例子中,对于四个不同的季节性即月、季度、期和年的1000个太阳黑子时间序列的误差指数,MSE,RMSE和NMSE的结果表明,当与月相比,季度、期和年展示了一个巨大的跌幅。总之,可以看到期季度有一个更小的误差指数,这说明太阳黑子有个强烈的期季度可以在预测过程中探索。
第四,提出了用一种称为基于状态相依AR模型的函数权重小波神经网络(FWWNN-AR)模型来对非线性时间序列进行建模和预测。这种模型利用改进的函数型权重小波神经网络(FWWNN)来逼近状态相依AR模型的状态相依系数,使其结合了状态相依AR模型利用少量节点描述非线性动态特性和FWWNN对时域和频域特性的函数逼近能力的优势。FWWNN是包含有输入层、输出层、小波层、规划层和多项式层的五层小波神经网络,其中多项式层是与输入有关的函数,用以计算输出的动态权重。该模型通过三个不同层次来学习非线性时间序列的动态特性,分别为AR层、小波网络层和函数型权重层。函数型权重层可使用不同的基函数,本文选用标准的WNN为函数型权重层的函数变量,从而使得FWWNN-AR模型的性能总优于WNN-AR模型,因此,FWWNN-AR模型可认为是基于WNN-AR模型的改进,FWWNN-AR的线性形式可通过选择极大平移因子来得到。对数值实验和实际非线性时间序列的预测效果表明了该模型的有效性和优越性。
1)扭曲长记忆AR时间序列；DLM绘制出市场或经济泡沫。为进行一次适当比较,我们采用在之前已发表的文章中所应用的同一数据。此时间序列由500个数据点组成,其中的400个数据点用于训练模型,而剩余的100个数据点用于检验所提出模型在未知情况下的性能。主要任务是利用含有3种不同权重基函数的FWWNN-ARX(4,2,2)模型来得到DLM的一步前向预测。所得结果显示出,作为最好模型出现的FWWNN-AR1模型而言,所提出的模型有更好的性能。并且所提出的模型能够超越WNN-AR模型,这也证明了所提出的模型是基于WNN-AR模型的改进。这些性能一部分是因为所提出模型权重分担非线性与小波间隔的能力。此外,对于扰动数据,所提出模型所获得预测精确的水平显示出其对于扰动数据的鲁棒性。
2) Mackey-Glass时间序列；在这一部分,两个方案和如在(2)一样的条件下进行研究,在第一种方案,通过利用不同的延迟得到的结果和那些在参考文献中的模型得到的结果进行比较来计算这个模型的性能值。三种不同权值基函数用于进行比较。结果表明当考虑同一滞后时,和RBF, RBF-AR, WNN-AR以及LLRBF-AR模型比较,所提出模型不仅会有一个更小的预测误差,而且有更小的AIC值。这也表明所提出模型比RBF, RBF-AR, WNN-AR以及LLRBF-AR模型需要更少的时间序列延迟来使性能达到更好。此外,我们可以看出FWWNN-AR模型未知数据的预测精度会降低为标准WNN-AR模型未知数据预测精度的40,47%,38,78%和56,21%分别对于5,6和7个滞后。这也证实FWWNN-AR模型是基于WNN-AR模型的一个改进。在第二个方案,此项方案的任务是从一个固定的输入中预测出y(t+6)。和其他例举的方法相比较,所提出模型能够得到更好的预测精度。在学习的过程中,通过SNPOM对基于3种权值基础函数的FWWNN-AR模型预测误差的反复估算,使预测误差会减小到一个很小的值。
3)洛伦兹时间序列：如(2)中使用的洛伦兹因子,本章节中,主要工作是通过FWWNN-ARX(5,2,2)三种不同的权重因子获得非线性时间序列一步预测。NMSE用来训练和测试参考文献中的这些关于相同时间序列的模型,并且用来做对比。与别的模型对比可以看出FWWNN-ARX模型只需要更少的节点就能得到最小的方差。结果显示了很好地统计特性也证明了处理动态原始数据更有效率。FWWNN-AR模型优于其他的模型,这个模型减少了98.90%检验误差,而WNN-AR模型只减少了97.86%。
4)太阳黑子时间序列：如(3)中所描述的数据。在这个例子中一步向前预测使用FWWNN-ARX模型使用三种不同权重函数。MSE,RMSE和NMSE测试前面模型得来的太阳黑子时间序列,用来与参考文献中其他的一些经典方法做对比。对照可以发现在模型中输入层、隐含层、输出层均只要一个神经元,AR时间延迟为4。而第二个模型输入层需要五个神经元,隐含层需要7个,输出层需要5个神经元和7个recurrent neurons.因此本模型只需要更少的非线性参数就可以得到比其他模型更好的结果。在例子中,提出的所有的模型得到了相似的结果,同时也比标准的WNN-AR和其他模型效果更好。
5)Australian sparkling wine time series:这个时间序列和章节(2)中提到一样,在2.2.2FWWNN-ARX中使用三种不同的权重的基函数。通过连续五年的平均绝对误差百分比(MAPE)和均方误差(MSE),并且按月,半年,一年得到平均值,以及与SARIMA的年平均作对比。从中得出本模型的半年平均和年平均有更好的效果,即使本模型中使用的是半年的数据值而SARIMA采用的年平均值。但是也有一些年份SRIMA模型有更好的效果。此外,所有模型中效果最好的年平均值,这是SARIMA作者的建议,数据按年来做计算。按照年数据FWWNN-AR1、FWWNN-AR4、FWWNN-AR5和SARIMA对比MAPE和MSE参数,FWWNN-AR5是效果最好的模型。该模型比标准的WNN-AR有着更好的效果,进一步证明了是比WNN-AR更好的模型。
6)煤气炉时间序列：尽管煤气炉的数据可以近似线性的输入-输出行为,然而很多非线性的建模方法用在对线性模型的建模和预测上。这种异常主要是由输入结构引起的。这个例子的主要目的是对于这些线性数据,本文提出的模型与其他的线性与非线性模型的效果对比。为了对比,这里使用图3中的数据,同时列出了一些比较所用到的条件。总共用到了五组测试数据,五组数据的输入及用来测试和检验的数据长度不同。模型的输入包括过去的输入及输出。每一个例子使用本文提出的模型进行一步预测。第一个例子用y(t-1)和u(t-4)作为模型的输入来计算y(t),u1,(x(t-q))=x(t-q)2用来衡量基函数的权重。
从AIC来看,本文提出的线性模型的AIC明显大于非线性模型。这说明非线性模型对这些数据的建模效果更好。这里同样列出了本文提出的非线性模型与其他的非线性模型的对比结果。第二个例子使用,y(t-1),u(t-3)及u(t-4)作为模型的输入来获得模型的输出y(t),u1(x(t-q))=x(t-q)2用来衡量基函数的权重。同样可以看到线性模型的AIC大于非线性模型的AIC。从结果可以看出来,非线性模型的更加适合用于对煤气炉的建模及预测。此外,与其他的非线性模型相比较,在获得同样的精度的条件下,本文提出的非线性模型使用了更少的神经网络节点。在第三个例子中使用y(t-1),u(t-3),u(t-4)和u(t-5)作为模型的输入,u1(x(t-q))=x(t-q)2用来衡量基函数的权重。同样可以看出,本文提出的线性模型的AIC比本文所提出的非线性模型的AIC要大,非线性模型更加适合对煤气炉模型进行建模和预测。同样可以看到本文提出的非线性模型的MSE是0.086,而Nie所提出来的模型的MSE是57.98。在第四个例子中使用y(t-1),u(1-2),u(t-3),u(t-5)和u(t-6)作为模型的输入,u1(x(t-q))=x(t-q)2用来衡量基函数的权重。前200个数据用来训练模型,后40个数据用来测试模型。获得的线性模型的AIC与非线性模型比较,在训练数据中线性模型的AIC较大,而训练数据中的线性模型的AIC较小。这意味着线性模型比非线性模型更加适合。这是因为输入和输出有很强的线性结构。因而在这个例子中提出的线性模型的MSE比非线性模型的MSE更小。最后一个例子中使用y(t-1),y(t-2),y(t-3),y(t-4),u(t-1),u(t-2),u(t-3), u(t-4),u(t-5)和u(t-6)。这种结构广泛使用。u1(x(t-q))=x(t-q)11用来衡量基函数的权重。为了跟以前的工作进行比较,使用的数据为290个,前145个数据用来训练本文提出的模型。可以看出来本文提出的线性模型的AIC远大于非线性模型的AIC。这意味着对于这一组输入的煤气炉的数据,非线性模型更加适合用来建模和预测。此外,本文提出的非线性模型因为只用到了一层隐含层,因而用到的参数比其他的模型更少。与第二好的模型对比,FWWNN-AR1(1,1,10)模型在训练数据上的MSE比第二好的模型小77.34%,在测试数据上比第二好的模型小36.74%。有这么好的结果,主要是因为本文提出的模型的权重是动态的。
最后,将提出的FWWNN-AR模型在控制中进行了应用,一种基于ARX模型的扩展指数型小波神经网络(EW-WNN-ARX)模型作为模型预测控制的内模用以对船舶进行控制。为了得到稳定的系统,用航向偏离角角速度作为模型输出,建立EW-WNN-ARX模型来描述动态船舶航向偏离角和舵角之间的差异。为了描述船舶的动态非线性特征,滚动角作为状态特征量用以引导模型参数变化以反映船舶的运动状态。采用SNPOM离线辨识EW-WNNN-ARX模型以避免在线优化带来的诸多问题,估计出的模型实质是一个以航向偏离角描述船舶位置轨迹误差特征的数学模型,这种模型是统计模型和数学模型的结合体,称为E-EW-WNN-ARX-MM模型。通过该模型可建立起船舶运动的状态空间模型并将其作为模型预测控制(MPC)的内模以实现船舶跟踪期望轨迹连续运动。E-EW-WNN-ARX-MM数据来自于Shioji-MARU experimental ship of Tokyo University of Marine Science and Technology of Japan。建模与控制结果表明了基于离线辨识的E-EW-WNN-ARX-MM能描述大范围过程的全局非线性特性。
Time series are an integral part of our lives. They are found in the field of social and natural sciences, engineering and economy. The need for their analysis, modeling, prediction and control has become more urgent. Generally, the main objective of time series analysis is to develop models that can establish the relation between variables. To serve these purposes, various models ranging from linear to nonlinear have been developed throughout the time. Usually, when the relation among different time series variables is linear, linear models are more suitable. On the other hand, when this relation is nonlinear, nonlinear models are more suitable. Linear models are easier to identify and implement. However, most real world systems are governed by nonlinear rules, hence, the importance of nonlinear modeling technique in dealing with time series.
In this thesis, the problematic of modeling and prediction of time series together with the control issues are considered. Four new models were developed including two fully nonlinear models that are enhanced type of neural networks and two more generalized models that are a combination of enhanced type of neural network with a linear model, thus having both a linear and nonlinear capability. Furthermore, a generalized nonlinear model is used as internal model to address the tracking issue of the real ship. The main work and innovations of this thesis are summarized as follows.
Firstly, a special type of RBF namely the Local Variable-Weights RBF Network (LVW-RBFN) was developed to investigate the modeling and forecasting problem of nonlinear time series. The LVW-RBFN is a four layered RBFN comprising of input layer, hidden layer, weight layer and output layer. It is an enhanced type of RBF network, in which the constant weights that connect the hidden layer with the output layer in the standard RBFN are replaced by functions of the RBFN's inputs computed via the weight layer. A structured nonlinear parameter estimation method (SNPOM) is applied to estimate the model. Case studies on several benchmark time series show that the prediction performance of the LVW-RBFN model is superior to other newly developed models including the RBF thus confirming that the proposed model is an enhanced type of RBF.
Secondly, the modeling and prediction problems of nonlinear time series are addressed using radial basis function network with polynomial type weights (LPRBF) to approximate the functional coefficients of the general state-dependent autoregressive (SD-AR) model. The LPRBF differs from the RBF by its weight form; the LPRBF has polynomial type weights, which are dynamic and vary with the network input, while the weights of the RBF are constants they are not affected by the change in the input of the network. The LPRBF is a more generalized form of RBF. It may be considered as an improved version of the RBF. Thus the LPRBF.-AR can be regarded as an enhanced version of previous RBF-AR model and local linearly RBF based AR (LLRBF-AR) model. A fast-converging estimation method known as the structured nonlinear parameter optimization method (SNPOM) is applied to estimate the LPRBF-AR. Case studies on various time series and chaotic systems show that the LPRBF-AR modeling approach exhibits much better prediction accuracy compared with some other existing methods.
Thirdly, an enhanced type of wavelet neural network, known as Local Polynomial Wavelet Neural Network with a Structured Nonlinear Parameter Optimization Method (LPWNN-SNPOM) is introduced. This is a five layered wavelet neural network comprising of input, output, wavelet, product and polynomial layers, which computes the weight as a function of inputs. These weights are dynamic and play an active part in the prediction. The LPWNN-SNPOM is an improvement of the Wavelet Neural Network with a Hybrid Learning Approach (WNN-HLA). These two models have mainly three differences:i) The LPWNN-SNPOM method contains a bias, ii) The single parameter weights connecting the hidden layer with the output in the WNN-HLA are replaced by polynomial functions of the inputs in the LPWNN-SNPOM, allowing the weights to vary with the changes in the inputs and share the dynamics with the wavelet compartment, iii) Unlike the WNN-HLA, which uses an online optimization approach, the LPWNN-SNPOM makes usage of an offline optimization approach known as the Structured Nonlinear Parameter Optimization Method (SNPOM). The performance and effectiveness of the proposed model are illustrated using several examples, whose results show the feasibility of the proposed model and demonstrate that it improved upon the WNN-HLA and performed better than some other well-known models.
Fourthly, the problems of modeling and prediction of time series are investigated with a newly introduced model namely the functional weights wavelet neural network-based state-dependent AR (FWWNN-AR) model. This model consists of combining an enhanced WNN named as a Functional Weights Wavelet Neural Network (FWWNN) to approximate the state dependant coefficients of the AR. The FWWNN-AR model possesses both the advantages of the state-dependent AR model in the description of nonlinear dynamics using few nodes and of the FWWN network in functional approximation considering mutually the time and frequency spaces. The FWWNN is a five-layered network structure comprising of an input layer, wavelet layer, product layer, output layer and functional weight layer. The functional weight layer computes the weight as a function of inputs, making the weights to vary with the inputs and share the dynamics with the wavelet compartment. The proposed model learns the dynamics of nonlinear time series from three distinct levels:AR level, wavelet compartment level and weights level. Different basis functions could be used as functional weight. However, in this thesis, these functions are chosen in such a way that the FWWNN has the standard WNN as one of its component which means that the FWWNN-AR will always perform better than the WNN-AR. Thus the FWWNN-AR can be seen as an improvement of the WNN-AR. Furthermore, the linear version of the FWWNN-AR may be obtained by choosing the translation to tend to infinity. The proposed model is validated by comparing its performances and effectiveness with those achieved by some well-known models on both generated and real nonlinear time series.
Finally, the proposed models are investigated for the control issue. To this ends, the FWWNN-AR model, specifically the expanded exponential weight wavelet network based ARX (EW-WNN-ARX) model, which is used as based for developing the internal model of MPC with the aim to control a maritime vehicle. In order to achieve an almost stationary system, the difference of heading angle deviation is used instead of the heading angle. The EW-WNN-ARX model is used to describe the ship motion between the difference of heading angle deviation and the rudder angle of the ship. To represent the ship motion's nonlinearity, rolling angle is used as model index to allow the model parameters vary with ship moving state. After identification of The EW-WNN-ARX by an offline method known as SNPOM to avoid a possible failure that may occur during online identification procedure, the difference of heading angle deviation are expanded and integrated into mathematical model (MM) to characterize the position tracking error of the ship. The model resulting is a combination of statistical and mathematical model namely the E-EW-WNN-ARX-MM model. The E-EW-WNN-ARX-MM model is then used to develop the state-space type tracking motion model which is used as internal model of the MPC to steer ship moving forward with constant velocity along a predefined reference path. The E-EW-WNN-ARX-MM is developed using data obtained from the Shioji-MARU experimental ship of Tokyo University of Marine Science and Technology of Japan. The modeling and control results achieved demonstrate that the proposed E-EW-WNN-ARX-MM-identified off-line can describe the global nonlinear property of the process over a wide region.
引文
[1]G. Kitagawa. Introduction to time series modeling CRC press.2009.
[2]C.W.J. Granger, A.P. Andersen. An Introduction to Bilinear Time Series Models. Vandenhoek and Ruprecht, Gottingen.1978
[3]H. Tong. Threshold Models in Nonlinear Time Series Analysis. Springer Verlag, New York.1983.
[4]V. Haggan, T. Ozaki. Modeling nonlinear random vibrations using an amplitude-dependent autoregressive time series model. Biometrika.1981,68:189-196.
[5]T. Ozaki, H. Oda. Nonlinear time series model identification by Akaike's information criterion, in: B. Dubuisson (Ed.), Information and Systems,1978,83-91.
[6]K.S. Chan, H. Tong. On estimating thresholds in autoregressive models, Journal of Time Series Analysis,1986,7:179-190.
[7]M.B. Priestley. State dependent models:A general approach to nonlinear time series analysis, Journal of Time Series Analysis,1980,1:57-71.
[8]R. Chen, R.S. Tsay. Functional-coefficient autoregressive models, Journal of the American Statistical Association 1993,88:298-308.
[9]G.P. Zhang, A neural network ensemble method with jittered training data for time series forecasting, Information Sciences.2007,177:5329-5346.
[10]T.G. Barbounis, J.B. Teocharis, Locally recurrent neural networks for wind speed prediction using spatial correlation, Information Sciences.2007,177:5775-5797.
[11]Y. Chen, B. Yang, J. Dong. A. Abraham, Time series forecasting using flexible neural tree model, Information Sciences.2005,174 (3-4):219-235.
[12]A. Gholipour, B.N. Araabi, C. Lucas. Predicting chaotic time series using neural and neurofuzzy models:a comparative study, Neural Processing Letter.2006,24:217-239.
[13]C. Hamzacebi. Improving artificial neural networks'performance in seasonal time series forecasting. Information Sciences.2008,178 (23):4550-4559.
[14]F. Giordano, M.L. Rocca, C. Perna. Forecasting nonlinear time series with neural network sieve bootstrap. Computational Statistics and Data Analysis.2007,51:3871-3884
[15]V.M. Rivas, J.J. Merelo, P.A. Castillo, et al. Evolving RBF neural networks for time-series forecasting with EvRBF. Information Sciences.2005,165:207-220.
[16]H. Leung, T. Lo, S. Wang. Prediction of noisy chaotic time series using an optimal radial basis function neural network. IEEE Transactions on Neural Networks,2001,12 (5):1163-1172.
[17]I. Rojas, H. Pomares, J.L. Bernier, J. Ortega, et al. Time series analysis using normalized PG-RBF network with regression weights. Neurocomputing.2002,42:267-285.
[18]K.B. Cho, B.H. Wang. Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction:Fuzzy Sets and Systems.1996,83:325-329.
[19]A.F. Sheta, K.D. Jong. Time-series forecasting using GA-tuned radial basis functions, Information Sciences.2001,133:221-228.
[20]H. Du, N. Zhang. Time series prediction using evolving radial basis function networks with new encoding scheme. Neurocomputing.2008,71:1388-1400.
[21]C. Harpham, C.W. Dawson. The effect of different basis functions on a radial basis function network for time series prediction:a comparative study. Neurocomputing.2006,69:2161-2170.
[22]M. A. S. Potts and D. S. Broomhead, "Time series prediction with a radial basis function neural network," SPEE Adaptive Signal
[23]R. Langari, L. Wang and J. Yen. Radial Basis Function Networks, Regression Weights, and the Expectation-Maximization Algorithm. IEEE Trans on systems, man, and cybernetics—part a: systems and humans.1997,27(5):613-623.
[24]I. Rojas, J. Gonzalez, A Caflas, A. F. Diaz, F. J. Rojas and M. Rodriguez. Short-Term Prediction of Chaotic Time Series by Using RBF Network with Regression Weights. International Journal of Neural Systems.2000,10(5):353-64.
[25]I. Rojas, H. Pomares, J.L. Bernier, J. Ortega, et al. Time series analysis using normalized PG-RBF network with regression weights, Neurocomputing.2002,42:267-285.
[26]C.A. da Silva. Time series forecasting with a non-linear model and the scatter search meta-heuristic. Information Sciences.2008,178:3288-3299.
[27]O. Valenzuela, I. Rojas, F. Rojas, H. Pomares, et al. Hybridization of intelligent techniques and ARJMA models for time series prediction. Fuzzy Sets and Systems.2008,159:821-845.
[28]G. P. Zhang. Time series forecasting using a hybrid ARJMA and neural network model, Neurocomputing.2003,50:159-175.
[29]M. Ardalani-Farsa, S. Zolfaghari. Chaotic time series prediction with residual analysis method using hybrid Elman-NARX neural networks. Neurocomputing.2010,73:2540-2553
[30]J. Vesin. An amplitude-dependent autoregressive signal model based on a radial basis function expansion. In:Proceedings of International Conference on Acoustics, Speech, Signal Processing. 1993,3:129-132.
[31]Z. Shi, Y. Tamura, T. Ozaki. Nonlinear time series modeling with the radial basis function-based state-dependent autoregressive model. International Journal of System Science.1999,30 (7): 717-727.
[32]H. Peng, T. Ozaki, V. Haggan-Ozaki, Y. Toyoda. A parameter optimization method for radial basis function type models. IEEE Transactions on Neural Networks.2003,14:432-438.
[33]G. Min, P. Hui, P. Xiaoyang, C. Xiaohong, G. Inoussa. A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling. Information Sciences.2010,180: 4370-4383.
[34]Y. Chen, B. Yang, J. Dong. Time series prediction using a local linear wavelet neural network. Neurocomputing.2006,69:449-465.
[35]J. Cao, Z. Lin and G.-B. Huang. Composite function wavelet neural networks with extreme learning machine. Neurocomputing,2010,73(7-9):1405-1416.
[36]J. Cao, Z. Lin and G.-B. Huang. Composite Function Wavelet Neural Networks with Differential Evolution and Extreme Learning Machine. Neural Process Lett.2011,33:251-265.
[37]S.A. Billings and H.L. Wei. The Wavelet-NARMAX Representation:A Hybrid Model Structure Combining Polynomial Models with Multiresolution Wavelet Decompositions. International Journal of Systems Science.2005,36(3):137-152.
[38]Choi, T.M., Y. Yu, K.F. Au. A Hybrid SARIMA Wavelet Transform Method for Sales Forecasting. Decision Support Systems.2011,51:130-140.
[39]J. Needham. Science and Civilization in China. Mechanical Engineering. Taipei:Caves Books, Ltd.1986,4(2).
[40]R. T. Howard. Science in Medieval Islam:An Illustrated Introduction, p.181, University of Texas Press. ISBN 0-292-78149-0.1997,181
[41]H. Nyquist Regeneration theory. Bell System Technical.1932
[42]H.W Bode. Network Analysis and Feedback Amplifier Design, D. Van Nostrand, Princeton.1945, 39:321-335.
[43]W. R. Evans.:Control-System Dynamics. McGraw-Hill.1954.
[44]R. E Kalman,.A New Approach to Linear Filtering and Prediction Problems. Transaction of the ASME--Journal of Basic Engineering.1960,35-45.
[45]D. Landau, R Lozano, and M. M'Saad. Adaptive Control. London:Springer.1998.
[46]G. Zames. Feedback and optimal sensitivity:Model reference transformation, multiplicative seminonns and approximate inverses. IEEE Trans. Automat. Contr.1981,23:.301-320,
[47]K. Zhou, J. C. Doyle, K. Glover. Robust and Optimal Control. Prentice Hall, NJ,1996
[48]R. Marino and P. Tomei. Nonlinear Control Design:Geometric, Adaptive and Robust London: Prentice Hall.1995
[49]M. Krstic and H. Deng. Stabilization of Nonlinear Uncertain Systems. Springer.1998
[50]R A. Freeman, P. V. Kokotovic. Robust Nonlinear Control Design State-Space and Lyapunov Techniques Birkhauser Boston · Basel · Berlin 1996.
[51]H.Nakamura, H Akaike. Statistical Identification for Optimal Control for Supercritical Thermal Power Plants", Automatica.1981,17(1):143-155.
[52]H. Nakamura,Y. Toyoda. Statistical identification and optimal control of thermal power plants. Ann Inst Stat. Math.1988,40:1-28.
[53]G. Prasad, G. W. Irwin, E. Swidenbank, B.W. HoggBW. A local model networks. predictive control strategy for thermal power plants. Automatica.1998,34:1185-1204.
[54]H. H. J. Bloemen, T.J.J. van den Boom, H.B. Verbruggen. Model-based Predictive Control for Hammerstein systems. International Journal of Control.2001,74:482-485
[55]D. Q. Mayne, J. B. Rawlings, C. V. Rao, P. O. M. Scokaert. Constrained model predictive control: stability and optimality. Automatica.2000.36 (6):789-814.
[56]C. E. Garcia; M. Morari. Internal Model Control 1 A Unifying Review and Some New Results. Industrial & Engineering Chemistry Process Design and Development.1982,21 (2):308-323.
[57]D. W. Clarke, C. Mohtadi, and P. S. Tuffs. Generalized Predictive Control-Part Ⅰ and Ⅱ. Automatica.1987,.23(2):137-160.
[58]J. Richalet, A Rault, J. L. Testud, J. Papon. Model Predictive Heuristic Control:Application to Industrial Processes. Automatika.1978,14:413-428.
[59]R Rouhani, R.K Mehra, Model Algorithmic control (MAC), Basic Theoretical Properties, Automatics.1982,18(4); 401-414.
[60]C.R. Cutler, B.L. Ramaker, "Dynamic Matrix Control - A Computer Control Algorithm," in Proc. Joint Automatic Control Conference, San Francisco.1980.
[61]F. Allgower, A. Zheng (Eds.). Nonlinear Model Predictive Control Progress in system and control theory. Birkhauser Verlag,. Basel,2000,26
[62]S.Qin and T. Badgewell, A survey of industrial model predictive Control technology, Control Engineering Practice.2003,11:733-764.
[63]H. Chen, F. Allgower. A Quasi-infinite Horizon Non-linear Model Predictive Control Scheme with Guaranteed Stability. Automatica.1998,34:1205-1217.
[64]D. Q Mayne, H. Michalska. Receding Horizon Control of Non-Linear Systems, IEEE Transactions on Automatic Control.1990 35(5):814-824.
[65]T. Parisini, R. Zoppoli. A Receding-horizon Regulator for Nonlinear Systems and a Neural Approximation", Automatica, vol.31, n.10, pp.1443-1451,1995.
[66]G.De Nicolao, L.Magni, R.Scattolini:Stabilizing Predictive Control of Nonlinear ARX Models", Automatica.1997,.33(9); 1691-1697.
[67]G.De Nicolao, L.Magni, R.Scattolini:Stabilizing Receding-Horizon Control of Nonlinear Time-Varying Systems. IEEE Trans, on Automatic Control 1998,43:,1030-1036.
[68]L. Magni, G. De Nicolao, R. Scattolini. Output Feedback and Tracking of Nonlinear Systems with Model Predictive Control", Automatica.2001,37:1601-1607.
[69]L. Magni, G. De Nicolao, L. Magnani, and R. Scattolini. A Stabilizing Model-Based Predictive Controller for Nonlinear Systems. Automatica.2001,37:11351-1362.
[70]D.W.Clarke, R.Scattolini. Constrained Receding-Horizon Predictive Control. Proc. IEE Part D, 138(4):347-354.
[71]H. Peng, J. Wu, G. Inoussa, Q. Deng, K. Nakano. Nonlinear System Modeling and Predictive Control using the RBFNets-based Quasi-linear ARX Model. Control Engineering Practice.2009, 17(1):50-66.
[72]H. Peng, K. Nakano, H. Shioya. Nonlinear predictive control using neural nets-based local linearization ARX model Stability and industrial application. IEEE Trans, on Control Systems Technology.2007,15(1):130-143.
[73]H. Peng, Z. J. Yang, W. Gui, M. Wu, H. Shioya, K. Nakano, K. Nonlinear system modeling and robust predictive control based on RBF-ARX model. (IFAC) Engineering Applications of Artificial Intelligence.2007,20(1):1-9.
[74]K. Lin, Q. Lin, C. Zhou, and J. Yao. Time Series Prediction Based on Linear Regression and SVR. Proc. of the Third International Conference on Natural Computation.2007,688-691.
[75]M. Boznar, M. Lesjak, and P. Mlakar. A Neural Network-Based Method for Short-Term Predictions of Ambient SO2 Concentrations in Highly Polluted Industrial Areas of Complex Terrain. Atmospheric Environment.1993,27(2):221-230.
[76]S. M. Chen, and N. Y. Chung. Forecasting Enrollments using High-Order Fuzzy Time Series and Genetic Algorithms. International Journal of Intelligent Systems.2006,21(5):485-501.
[77]E. Montafles, J. R. Quevedo, M. M. Prieto, and C. O. Menendez. Forecasting Time Series Combining Machine Learning and Box-Jenkins Time Series. Advances in Artificial Intelligence IBERAMIA.2002.2527:491-499.
[78]V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, NY,1995.
[79]A. J. Smola and B. Sch" olkopf. A tutorial on support vector regression. Statistics and Computing, 14(3):199-222,2004.
[80]H. Drucker, C. J. C. Burges, L. Kaufman, A. J. Smola, and V. Vapnik Support vector regression machines. In M. C. Mozer, M. I Jordan, T. Petsche. Advances in Neural Information Processing Systems. Cambridge, MA. MIT Press 1997,9:155-161.
[81]M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems. Cambridge, MA. MIT Press 1997,9:155-161.
[82]H. Peng, T. Ozaki, V. Haggan-Ozaki and Y. Toyoda. Structured parameter optimization method for the radial basis function-based state-dependent autoregressive model. International Journal of Systems Science.2002,33:1087-1098.
[83]H. Akaike, A new look at the statistical model identification, IEEE Transactions on Automatic Control.1974,19:716-723.
[84]K.B. Cho, B.H. Wang. Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction. Fuzzy Sets and Systems.1996,83:325-329.
[85]D. Kim, C. Kim. Forecasting time series with genetic fuzzy predictor ensembles. IEEE Transactions on Fuzzy Systems.1997,5:523-535.
[86]T. Ozaki, J.C. Jimenez, H. Peng, V.H. Ozaki. The innovation approach to the identification of nonlinear causal models in time series analysis, in:D.R. Brillinger, E.A. Robinson, F.P. Schoenberg (Eds.), Time Series Analysis and Applications to Geophysical Systems, Springer, New York.2004,195-226
[87]R.L. Hardy. Multiquadric equations of topography and other irregular surfaces. J. Geophys. Res. 1971,761905-1915.
[88]M.J.D. Powell. Radial basis functions for multivariable interpolation:a review. In:J.C. Mason and M.G. Cox, Editors, Algorithms for approximation, Clarendon Press, Oxford, UK.1987, 143-167.
[89]D.S. Broomhead and D. Lowe, Multivariable functional interpolation and adaptive networks. In: Complex Syst.1988,2:321-355.
[90]E.J. Kansa, Muliqudrics A scattered data approximation scheme with applications to computational fluid dynamics:II. Solutions toparabolic, hyperbolic, and elliptic partial differential equations. Comput. Math. Appl.1990, (6-8):147-161.
[91]X. Zhou, Y.C. Hon, J. Li, Overlapping domain decomposition method by radial basis functions, Appl. Numer. Math.2003,44:241-255.
[92]C. Franke, R. Schaback. Solving partial differential equations by collocation using radial basis functions, Appl. Math. Comput.1988,93:73-83.
[93]E.J. Kansa, Y.C. Hon. Circumventing the ill-conditioning problem with multiquadric radial basis functions:applications to elliptic partial differential equations. Comput. Math. Appl.2000,39: 123-137.
[94]Y.C. Hon, K.F. Cheung, X.Z.Mao, E.J. Kansa. A multiquadric solution for the shallow water equations. J. Hydrology.1999,125 (5):524-533.
[95]E.J. Kansa. Exact explicit time integration of hyperbolic partial differential equations with mesh free radial basis functions, Eng. Anal. Bound. Elem.2007,31:577-585.
[96]Y.C. Hon, X.Z. Mao. An efficient numerical scheme for Burgers equation. Appl. Math. Comput. 1998,95:37-50.
[97]A.E. Tarwater, A parameter study of Hardy_s multiquadric method for scattered data interpolation, UCRL-54670,1985.
[98]R.E. Carlson, T.A. Foley. The parameter R2 in multiquadric interpolations. Comput. Math. Appl.1991.21:29-42.
[99]J. C. Trahan, R. E. Wyatt. Radial basis function interpolation in the quantum trajectory method: optimization of the multi-quadric shape parameter, Journal of Computational Physics.2003,185: 27-^9.
[100]M. J. D Powell. Radial basis function for multivariable interpolation:a review, IMA Conference on Algorithms for the Approximation of Functions and Data, RMCS, Shrivenham.1985,43-167.
[101]G Inoussa, H. Peng Hui, L. Ren. RBF-ARX Modeling and Predictive Control Strategy Applied to a Liquid Level System Proceedings of the 26th Chinese Control Conference.2007,342-346.
[102]E. S. Cnng, S. Chen, and B. Mulgrew. Gradient Radial Basis Function Networks for Nonlinear and Nonstationary Time Series Prediction. IEEE TRANSACTIONS ON NEURAL NETWORKS. 1996,7(1):pp 190-194.
[103]M. J. D. Powell. Radial basis function approximations to polynomials, Proceedings of 12th Biennial Numerical Analysis Conference, Dundee.1987,223-241.
[104]M. Casdagli. Nonlinear prediction of chaotic time-series. Physica D,1989,35:335-356.
[105].S. Chen, C. F. N. Cowan, and P. M. Grant. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans.Neural Network.1991,2:302-309.
[106]H. Peng, T. Ozaki, V. Haggan-Ozaki and Y. Toyoda. Structured parameter optimization method for the radial basis function-based state-dependent autoregressive model. International Journal of Systems Science.2002,33:1087-1098.
[107]R.J.S. Jang, N. Gulley, Fuzzy Logic Toolbox User's Guide, The MathWorks, Inc., Natick, MA, 2000.
[108]H. Kantz, T. Schreiber. Nonlinear Time Series Analysis.Cambridge. UK:Cambridge University Press.1997.
[109]P.D. McNelis. Neural Networks in Finance:Gaining Predictive Edge in the Market. Elsevier Academic Press, Burlington, MA.200, chapters 3-4.
[110]M.S. Kim, C.H. Kim, J.J. Lee. Evolving compact and interpretable Takagi-Sugeno fuzzy models with a new encoding scheme. IEEE Trans. Syst. Man Cybern.2006,36 (5):1006-1023.
[111]M. Sugeno, T. Yasukawa. A fuzzy-logic-based approach to qualitative modeling, IEEE Trans. Fuzzy Syst.1993,1 (1):7-31.
[112]Y. Lin, G.A. Cunningham. A new approach to fuzzy-neural system modeling, IEEE Trans. Fuzzy Syst.1995,3 (2):190-197.
[113]J. Nie, Constructing fuzzy model by self-organizing counterpropagation network, IEEE Trans. Syst. Man Cybernet.1995 25 (6):963-970.
[114]T. Ozaki, P. V. Sosa, and V. Haggan-Ozaki, "Reconstructing the nonlinear dynamics of epilepsy data using nonlinear time series analysis," Journal of Signal Processing, Vol.3,1999, pp.153-162.
[115]D.W. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics.1963,11:431-441.
[116]D. Gorinevsky. An approach to parametric nonlinear least square optimization and application to task-level learning control. IEEE Transactions on Automatic Control.1997,42:912-927.
[117]D. Gorinevsky, D. E. Torfs, and A. A. Goldenberg. Learning approximation of feedforward control dependence on the task parameters with application to direct-drive manipulator tracking. IEEE Transactions on Robotics and Automation.1997,13:567-581.
[118]G. Prasad, E. Swidenbank, and B. W. Hogg. A Local Model Networks Based Multivariable Long-range Predictive Control Strategy for Thermal Power Plants. Automatica.1998,34(10):1185-1204.
[119]G. Armano, M. Marchesi, A. Murru. A hybrid genetic-neural architecture for stock indexes forecasting," Information Sciences.2005,170 (1):3-33.
[120]J. A. Leonard and M. A. Kramer. Radial basis function networks for classifying process faults. IEEE Control Systems Magazine.1991,11(3):31-38.
[121]A. Grossman and J. Morlet. Decomposition of Hardy functions into square integrable wavelets of constant shape. SIAM Journal of Mathematical Analysis1984,15:723-736.
[122]Y. Meyer. Principe d'incertitude, bases hilbertiennes et algebres d'op6rateurs. Seminaire Bourbaki. 1986,662:209-223.
[123]I. Daubechies. Orthonormal basis of compactly supported wavelets. Communications on Pure and Applied Mathematics.1988,41(7):909-996.
[124]S. G. Mallat. A theory for multiresolution signal decomposition:the wavelet representation. IEEE Transactions on Pattern Analysis and Machine. Intelligence.1989,11(7):674-693.
[125]H. H. Szu, B. A. Telfer, and S. L. Kadambe. Neural network adaptive wavelets for signal representation and classification. Optical Engineering.1992,31(9):1907-1915.
[126]Y. C. Pati, and P. S. Krishnaprasad. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks.1993,4:73-85.
[127]A. A. Safavi. Wavelet neural networks and multiresolution analysis with applications to process systems engineering. Ph.D. thesis, The University of Sydney, Australia.1996.
[128]A. A. Safavi and J. A. Romagnoli. Application of Wavelet-based Neural Networks to the Modelling and Optimisation of an Experimental Distillation Column. Engineering Applications of Artificial Intelligence.1997,10(3):301-313.
[129]Q, Zhang, and A Benveniste. A Wavelet networks. IEEE Transactions on Neural Networks.1992, 3(6),889-898.
[130]B. R, Bakshi. and G, Stephanopoulos. Wave-net a multiresolution. hierarchical neural network with localised learning. AIChE. Journal 1993,39(1):57-81.
[131]M. Meyer and G. Pfeiffer. Multilayer perceptron based decision feedback equalizers for channels with intersymbol interference. In IEE Proceedings-I.1993,140:420-424.
[132]C. J. Lin. Wavelet Neural Networks with a Hybrid Learning Approach. Journal of Information Science and Engineering.2006,1367-1387.
[133]B. Fischer, O. Nelles, R. Isermann. Adaptive predictive control of a heat exchanger based on a fuzzy model, Control Eng. Practice.1998,6:259-269.
[134]B. Foss, T.A. Johansen. On local and fuzzy modelling, Proceedings of 3rd International Industrial Fuzzy Control and Intelligent Systems.1993,134-139.
[135]G. H. Golub and C. F. Van Loan, Matrix Computations,3rd ed. Baltimore, MD:Johns Hopkins Univ. Press,1996.
[136]T. Coleman, M.A. Branch, A. Grace, Optimization Toolbox User's Guide, The MathWorks, Inc., Natick,MA,1999.
[137]C. J. Lin and C. T. Lin. An ART-based fozzy adaptive learning control network. IEEE Transactions on Fuzzy Systems, Vol.5,1997, pp.477-496.
[138]C. F. Juang and C. T. Lin. An on-line self-constructing neural fuzzy inference network and its applications. IEEE Transactions on Fuzzy Systems.1998,6:.12-31.
[139]F. J. Lin, C. H. Lin, and P. H. Shen. Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive. IEEE Transactions on Fuzzy Systems.2001,9:751-759.
[140]Wikipedia.2011.
[141]K. Teo, L. Wang, Z. Lin. Wavelet packet multi-layer perceptron for chaotic time series prediction: effects of weight initialization. Computational Science.2001,2074:310-317.
[142].G. McNish, J.V. Lincoln. Prediction of sunspot numbers. Transactions American Geophysical Union.1949,30(5):673-685.
[143]S. Sello, Solar cycle forecasting:a nonlinear dynamics approach. Astronomy & Astrophysics. 2001,377:312-320.
[144]K. Denkmayr, P. Cugnon. About sunspot number medium-terra predictions. In:G. Heckman et al. (Ed.), Solar-Terrestrial Prediction Workshop V, Hiraiso Solar Terrestrial Research Center, Japanl997, p.103.
[145]T. Koskela, M. Lehtokangas, J. Saarinen, K. Kaski. Time series prediction with multilayer perceptron, FIR and Elman neural networks. In Proceedings of the World Congress on Neural Networks.1996,491-496.
[146]SIDC (World Data Center for the Sunspot Index), [147]Q. Ma, Q. Zheng, H. Peng, T. Zhong, L. Xu. Chaotic time series prediction based on evolving recurrent neural networks. In:Proceedings of the Sixth International Conference on Machine Learning and Cybernetics. Hong Kong,2007.
[148]R. M. Tong. The evaluation of fuzzy model derived from experimental data. Fuzzy Sets and Systems.1980,4:1-12.
[149]W. Pedrycz. An identification algorithm in fuzzy relational systems. Fuzzy Sets Syst.1984 13(2): 153-167.
[150]C. Xu and Y. Lu. Fuzzy model identification and self-learning for dynamic systems. IEEE Trans. Syst, Man, Cybern.1987,17(4); 683-689.
[151]J. Kim*, N. Kasabov. FIS:adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems Neural Networks. PERGAMON.1999,12:1301-1319.
[152]Szu, H. H., Teller, B. and Kadambe, S. Neural network adaptive wavelets for signal representation and classification. Optical Engineering.1992,31(9):1907-1915.
[153]Y. C. Pati, P. S. Krishnaprasad. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks.1993,4 (1):1993 73-85.
[154]J. H. Schwarz. Estimating the dimension of a model. The Annals of Statistics.1978,6:461-464.
[155]M.J.L. Orr. Introduction to radial basis function networks,.1996.
[156]C. I. Mosier, Problems and designs of cross-validation. Educational and Psychological Measurement.1951,11:5-11.
[157]H. kantz and T. Schreiber, Nonlinear Time Series Analysis, Cambridge, UK:Cambridge University press.1998.
[158]H. Peng, k. Ohtsu, G. Kitagawa and H. Oda. A statistical modeling and tracking control approach to marine vehicle. IEEE International Conference on Control Applications.2010,640-645.
[159]K. D. Do, Z. P. Jiang, and J. Pan. Universal controllers for stabilization and tracking of underactuated ships. Systems & Control Letters,2002,47:299-317.
[160]Z. P. Jiang, "Global tracking control of underactuated ships by Lyapunov's direct method," Automatica,2002,38:301-309.
[161]K. Y. Pettersen; H. Nijmeijer, "Underactuated ship tracking control:theory and experiments," Int. J. of Control,2001,74:.1435-1446.
[162]E. Lefeber, K. Y. Pettersen, and H. Nijmeijer, "Tracking control of an underactuated ship," IEEE Trans. Control System Technology,2003,11:52-61.
[163]S. Miyoshi, Y. Hara, K. Ohtsu, "Study on Optimum Tracking Control with Linearized Model for Vessel [in Japanese]," Japan Institute of Navigation,2007,11:183-189.
[164]E. Lopez, F. J. Velasco, E. Moyano and T. M. Rueda full-scale manoeuvering trials Simulation Journal of Maritime Research,1(3); 37-50.
[165]J. S. Park, K. Ohtsu, and G. Kitagawa, Batch-adaptive ship's autopilotst, Int. J. Adapt. Control Signal Process,,2000,14:427-439.
[166]K. Ohtsu and G. Kitagawa, Statistical Analysis of the AR Type Ship's Autopilot System, Journal of Dynamic System, Measurement and Control.1984,106; 193-202.
[167]K. Ohtsu, M. Horigome and G. Kitagawa. A new ship's auto pilot'design through a stochastic model, Automatica.1979,15:255-268.
[168]T. Iseki, and K. Ohtsu, "Bayesian estimation of directional wave spectra based on ship motions," Control Engineering Practice.2000,8:215-219.
[169]H. Oda, K. Ohtsu and T. Hotta, "Statistical analysis and design of a rudder roll stabilization system," Control Engineering Practice.1996,4:351-358.
[2]C.W.J. Granger, A.P. Andersen. An Introduction to Bilinear Time Series Models. Vandenhoek and Ruprecht, Gottingen.1978
[3]H. Tong. Threshold Models in Nonlinear Time Series Analysis. Springer Verlag, New York.1983.
[4]V. Haggan, T. Ozaki. Modeling nonlinear random vibrations using an amplitude-dependent autoregressive time series model. Biometrika.1981,68:189-196.
[5]T. Ozaki, H. Oda. Nonlinear time series model identification by Akaike's information criterion, in: B. Dubuisson (Ed.), Information and Systems,1978,83-91.
[6]K.S. Chan, H. Tong. On estimating thresholds in autoregressive models, Journal of Time Series Analysis,1986,7:179-190.
[7]M.B. Priestley. State dependent models:A general approach to nonlinear time series analysis, Journal of Time Series Analysis,1980,1:57-71.
[8]R. Chen, R.S. Tsay. Functional-coefficient autoregressive models, Journal of the American Statistical Association 1993,88:298-308.
[9]G.P. Zhang, A neural network ensemble method with jittered training data for time series forecasting, Information Sciences.2007,177:5329-5346.
[10]T.G. Barbounis, J.B. Teocharis, Locally recurrent neural networks for wind speed prediction using spatial correlation, Information Sciences.2007,177:5775-5797.
[11]Y. Chen, B. Yang, J. Dong. A. Abraham, Time series forecasting using flexible neural tree model, Information Sciences.2005,174 (3-4):219-235.
[12]A. Gholipour, B.N. Araabi, C. Lucas. Predicting chaotic time series using neural and neurofuzzy models:a comparative study, Neural Processing Letter.2006,24:217-239.
[13]C. Hamzacebi. Improving artificial neural networks'performance in seasonal time series forecasting. Information Sciences.2008,178 (23):4550-4559.
[14]F. Giordano, M.L. Rocca, C. Perna. Forecasting nonlinear time series with neural network sieve bootstrap. Computational Statistics and Data Analysis.2007,51:3871-3884
[15]V.M. Rivas, J.J. Merelo, P.A. Castillo, et al. Evolving RBF neural networks for time-series forecasting with EvRBF. Information Sciences.2005,165:207-220.
[16]H. Leung, T. Lo, S. Wang. Prediction of noisy chaotic time series using an optimal radial basis function neural network. IEEE Transactions on Neural Networks,2001,12 (5):1163-1172.
[17]I. Rojas, H. Pomares, J.L. Bernier, J. Ortega, et al. Time series analysis using normalized PG-RBF network with regression weights. Neurocomputing.2002,42:267-285.
[18]K.B. Cho, B.H. Wang. Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction:Fuzzy Sets and Systems.1996,83:325-329.
[19]A.F. Sheta, K.D. Jong. Time-series forecasting using GA-tuned radial basis functions, Information Sciences.2001,133:221-228.
[20]H. Du, N. Zhang. Time series prediction using evolving radial basis function networks with new encoding scheme. Neurocomputing.2008,71:1388-1400.
[21]C. Harpham, C.W. Dawson. The effect of different basis functions on a radial basis function network for time series prediction:a comparative study. Neurocomputing.2006,69:2161-2170.
[22]M. A. S. Potts and D. S. Broomhead, "Time series prediction with a radial basis function neural network," SPEE Adaptive Signal
[23]R. Langari, L. Wang and J. Yen. Radial Basis Function Networks, Regression Weights, and the Expectation-Maximization Algorithm. IEEE Trans on systems, man, and cybernetics—part a: systems and humans.1997,27(5):613-623.
[24]I. Rojas, J. Gonzalez, A Caflas, A. F. Diaz, F. J. Rojas and M. Rodriguez. Short-Term Prediction of Chaotic Time Series by Using RBF Network with Regression Weights. International Journal of Neural Systems.2000,10(5):353-64.
[25]I. Rojas, H. Pomares, J.L. Bernier, J. Ortega, et al. Time series analysis using normalized PG-RBF network with regression weights, Neurocomputing.2002,42:267-285.
[26]C.A. da Silva. Time series forecasting with a non-linear model and the scatter search meta-heuristic. Information Sciences.2008,178:3288-3299.
[27]O. Valenzuela, I. Rojas, F. Rojas, H. Pomares, et al. Hybridization of intelligent techniques and ARJMA models for time series prediction. Fuzzy Sets and Systems.2008,159:821-845.
[28]G. P. Zhang. Time series forecasting using a hybrid ARJMA and neural network model, Neurocomputing.2003,50:159-175.
[29]M. Ardalani-Farsa, S. Zolfaghari. Chaotic time series prediction with residual analysis method using hybrid Elman-NARX neural networks. Neurocomputing.2010,73:2540-2553
[30]J. Vesin. An amplitude-dependent autoregressive signal model based on a radial basis function expansion. In:Proceedings of International Conference on Acoustics, Speech, Signal Processing. 1993,3:129-132.
[31]Z. Shi, Y. Tamura, T. Ozaki. Nonlinear time series modeling with the radial basis function-based state-dependent autoregressive model. International Journal of System Science.1999,30 (7): 717-727.
[32]H. Peng, T. Ozaki, V. Haggan-Ozaki, Y. Toyoda. A parameter optimization method for radial basis function type models. IEEE Transactions on Neural Networks.2003,14:432-438.
[33]G. Min, P. Hui, P. Xiaoyang, C. Xiaohong, G. Inoussa. A locally linear RBF network-based state-dependent AR model for nonlinear time series modeling. Information Sciences.2010,180: 4370-4383.
[34]Y. Chen, B. Yang, J. Dong. Time series prediction using a local linear wavelet neural network. Neurocomputing.2006,69:449-465.
[35]J. Cao, Z. Lin and G.-B. Huang. Composite function wavelet neural networks with extreme learning machine. Neurocomputing,2010,73(7-9):1405-1416.
[36]J. Cao, Z. Lin and G.-B. Huang. Composite Function Wavelet Neural Networks with Differential Evolution and Extreme Learning Machine. Neural Process Lett.2011,33:251-265.
[37]S.A. Billings and H.L. Wei. The Wavelet-NARMAX Representation:A Hybrid Model Structure Combining Polynomial Models with Multiresolution Wavelet Decompositions. International Journal of Systems Science.2005,36(3):137-152.
[38]Choi, T.M., Y. Yu, K.F. Au. A Hybrid SARIMA Wavelet Transform Method for Sales Forecasting. Decision Support Systems.2011,51:130-140.
[39]J. Needham. Science and Civilization in China. Mechanical Engineering. Taipei:Caves Books, Ltd.1986,4(2).
[40]R. T. Howard. Science in Medieval Islam:An Illustrated Introduction, p.181, University of Texas Press. ISBN 0-292-78149-0.1997,181
[41]H. Nyquist Regeneration theory. Bell System Technical.1932
[42]H.W Bode. Network Analysis and Feedback Amplifier Design, D. Van Nostrand, Princeton.1945, 39:321-335.
[43]W. R. Evans.:Control-System Dynamics. McGraw-Hill.1954.
[44]R. E Kalman,.A New Approach to Linear Filtering and Prediction Problems. Transaction of the ASME--Journal of Basic Engineering.1960,35-45.
[45]D. Landau, R Lozano, and M. M'Saad. Adaptive Control. London:Springer.1998.
[46]G. Zames. Feedback and optimal sensitivity:Model reference transformation, multiplicative seminonns and approximate inverses. IEEE Trans. Automat. Contr.1981,23:.301-320,
[47]K. Zhou, J. C. Doyle, K. Glover. Robust and Optimal Control. Prentice Hall, NJ,1996
[48]R. Marino and P. Tomei. Nonlinear Control Design:Geometric, Adaptive and Robust London: Prentice Hall.1995
[49]M. Krstic and H. Deng. Stabilization of Nonlinear Uncertain Systems. Springer.1998
[50]R A. Freeman, P. V. Kokotovic. Robust Nonlinear Control Design State-Space and Lyapunov Techniques Birkhauser Boston · Basel · Berlin 1996.
[51]H.Nakamura, H Akaike. Statistical Identification for Optimal Control for Supercritical Thermal Power Plants", Automatica.1981,17(1):143-155.
[52]H. Nakamura,Y. Toyoda. Statistical identification and optimal control of thermal power plants. Ann Inst Stat. Math.1988,40:1-28.
[53]G. Prasad, G. W. Irwin, E. Swidenbank, B.W. HoggBW. A local model networks. predictive control strategy for thermal power plants. Automatica.1998,34:1185-1204.
[54]H. H. J. Bloemen, T.J.J. van den Boom, H.B. Verbruggen. Model-based Predictive Control for Hammerstein systems. International Journal of Control.2001,74:482-485
[55]D. Q. Mayne, J. B. Rawlings, C. V. Rao, P. O. M. Scokaert. Constrained model predictive control: stability and optimality. Automatica.2000.36 (6):789-814.
[56]C. E. Garcia; M. Morari. Internal Model Control 1 A Unifying Review and Some New Results. Industrial & Engineering Chemistry Process Design and Development.1982,21 (2):308-323.
[57]D. W. Clarke, C. Mohtadi, and P. S. Tuffs. Generalized Predictive Control-Part Ⅰ and Ⅱ. Automatica.1987,.23(2):137-160.
[58]J. Richalet, A Rault, J. L. Testud, J. Papon. Model Predictive Heuristic Control:Application to Industrial Processes. Automatika.1978,14:413-428.
[59]R Rouhani, R.K Mehra, Model Algorithmic control (MAC), Basic Theoretical Properties, Automatics.1982,18(4); 401-414.
[60]C.R. Cutler, B.L. Ramaker, "Dynamic Matrix Control - A Computer Control Algorithm," in Proc. Joint Automatic Control Conference, San Francisco.1980.
[61]F. Allgower, A. Zheng (Eds.). Nonlinear Model Predictive Control Progress in system and control theory. Birkhauser Verlag,. Basel,2000,26
[62]S.Qin and T. Badgewell, A survey of industrial model predictive Control technology, Control Engineering Practice.2003,11:733-764.
[63]H. Chen, F. Allgower. A Quasi-infinite Horizon Non-linear Model Predictive Control Scheme with Guaranteed Stability. Automatica.1998,34:1205-1217.
[64]D. Q Mayne, H. Michalska. Receding Horizon Control of Non-Linear Systems, IEEE Transactions on Automatic Control.1990 35(5):814-824.
[65]T. Parisini, R. Zoppoli. A Receding-horizon Regulator for Nonlinear Systems and a Neural Approximation", Automatica, vol.31, n.10, pp.1443-1451,1995.
[66]G.De Nicolao, L.Magni, R.Scattolini:Stabilizing Predictive Control of Nonlinear ARX Models", Automatica.1997,.33(9); 1691-1697.
[67]G.De Nicolao, L.Magni, R.Scattolini:Stabilizing Receding-Horizon Control of Nonlinear Time-Varying Systems. IEEE Trans, on Automatic Control 1998,43:,1030-1036.
[68]L. Magni, G. De Nicolao, R. Scattolini. Output Feedback and Tracking of Nonlinear Systems with Model Predictive Control", Automatica.2001,37:1601-1607.
[69]L. Magni, G. De Nicolao, L. Magnani, and R. Scattolini. A Stabilizing Model-Based Predictive Controller for Nonlinear Systems. Automatica.2001,37:11351-1362.
[70]D.W.Clarke, R.Scattolini. Constrained Receding-Horizon Predictive Control. Proc. IEE Part D, 138(4):347-354.
[71]H. Peng, J. Wu, G. Inoussa, Q. Deng, K. Nakano. Nonlinear System Modeling and Predictive Control using the RBFNets-based Quasi-linear ARX Model. Control Engineering Practice.2009, 17(1):50-66.
[72]H. Peng, K. Nakano, H. Shioya. Nonlinear predictive control using neural nets-based local linearization ARX model Stability and industrial application. IEEE Trans, on Control Systems Technology.2007,15(1):130-143.
[73]H. Peng, Z. J. Yang, W. Gui, M. Wu, H. Shioya, K. Nakano, K. Nonlinear system modeling and robust predictive control based on RBF-ARX model. (IFAC) Engineering Applications of Artificial Intelligence.2007,20(1):1-9.
[74]K. Lin, Q. Lin, C. Zhou, and J. Yao. Time Series Prediction Based on Linear Regression and SVR. Proc. of the Third International Conference on Natural Computation.2007,688-691.
[75]M. Boznar, M. Lesjak, and P. Mlakar. A Neural Network-Based Method for Short-Term Predictions of Ambient SO2 Concentrations in Highly Polluted Industrial Areas of Complex Terrain. Atmospheric Environment.1993,27(2):221-230.
[76]S. M. Chen, and N. Y. Chung. Forecasting Enrollments using High-Order Fuzzy Time Series and Genetic Algorithms. International Journal of Intelligent Systems.2006,21(5):485-501.
[77]E. Montafles, J. R. Quevedo, M. M. Prieto, and C. O. Menendez. Forecasting Time Series Combining Machine Learning and Box-Jenkins Time Series. Advances in Artificial Intelligence IBERAMIA.2002.2527:491-499.
[78]V. Vapnik. The Nature of Statistical Learning Theory. Springer-Verlag, New York, NY,1995.
[79]A. J. Smola and B. Sch" olkopf. A tutorial on support vector regression. Statistics and Computing, 14(3):199-222,2004.
[80]H. Drucker, C. J. C. Burges, L. Kaufman, A. J. Smola, and V. Vapnik Support vector regression machines. In M. C. Mozer, M. I Jordan, T. Petsche. Advances in Neural Information Processing Systems. Cambridge, MA. MIT Press 1997,9:155-161.
[81]M. C. Mozer, M. I. Jordan, and T. Petsche, editors, Advances in Neural Information Processing Systems. Cambridge, MA. MIT Press 1997,9:155-161.
[82]H. Peng, T. Ozaki, V. Haggan-Ozaki and Y. Toyoda. Structured parameter optimization method for the radial basis function-based state-dependent autoregressive model. International Journal of Systems Science.2002,33:1087-1098.
[83]H. Akaike, A new look at the statistical model identification, IEEE Transactions on Automatic Control.1974,19:716-723.
[84]K.B. Cho, B.H. Wang. Radial basis function based adaptive fuzzy systems and their applications to system identification and prediction. Fuzzy Sets and Systems.1996,83:325-329.
[85]D. Kim, C. Kim. Forecasting time series with genetic fuzzy predictor ensembles. IEEE Transactions on Fuzzy Systems.1997,5:523-535.
[86]T. Ozaki, J.C. Jimenez, H. Peng, V.H. Ozaki. The innovation approach to the identification of nonlinear causal models in time series analysis, in:D.R. Brillinger, E.A. Robinson, F.P. Schoenberg (Eds.), Time Series Analysis and Applications to Geophysical Systems, Springer, New York.2004,195-226
[87]R.L. Hardy. Multiquadric equations of topography and other irregular surfaces. J. Geophys. Res. 1971,761905-1915.
[88]M.J.D. Powell. Radial basis functions for multivariable interpolation:a review. In:J.C. Mason and M.G. Cox, Editors, Algorithms for approximation, Clarendon Press, Oxford, UK.1987, 143-167.
[89]D.S. Broomhead and D. Lowe, Multivariable functional interpolation and adaptive networks. In: Complex Syst.1988,2:321-355.
[90]E.J. Kansa, Muliqudrics A scattered data approximation scheme with applications to computational fluid dynamics:II. Solutions toparabolic, hyperbolic, and elliptic partial differential equations. Comput. Math. Appl.1990, (6-8):147-161.
[91]X. Zhou, Y.C. Hon, J. Li, Overlapping domain decomposition method by radial basis functions, Appl. Numer. Math.2003,44:241-255.
[92]C. Franke, R. Schaback. Solving partial differential equations by collocation using radial basis functions, Appl. Math. Comput.1988,93:73-83.
[93]E.J. Kansa, Y.C. Hon. Circumventing the ill-conditioning problem with multiquadric radial basis functions:applications to elliptic partial differential equations. Comput. Math. Appl.2000,39: 123-137.
[94]Y.C. Hon, K.F. Cheung, X.Z.Mao, E.J. Kansa. A multiquadric solution for the shallow water equations. J. Hydrology.1999,125 (5):524-533.
[95]E.J. Kansa. Exact explicit time integration of hyperbolic partial differential equations with mesh free radial basis functions, Eng. Anal. Bound. Elem.2007,31:577-585.
[96]Y.C. Hon, X.Z. Mao. An efficient numerical scheme for Burgers equation. Appl. Math. Comput. 1998,95:37-50.
[97]A.E. Tarwater, A parameter study of Hardy_s multiquadric method for scattered data interpolation, UCRL-54670,1985.
[98]R.E. Carlson, T.A. Foley. The parameter R2 in multiquadric interpolations. Comput. Math. Appl.1991.21:29-42.
[99]J. C. Trahan, R. E. Wyatt. Radial basis function interpolation in the quantum trajectory method: optimization of the multi-quadric shape parameter, Journal of Computational Physics.2003,185: 27-^9.
[100]M. J. D Powell. Radial basis function for multivariable interpolation:a review, IMA Conference on Algorithms for the Approximation of Functions and Data, RMCS, Shrivenham.1985,43-167.
[101]G Inoussa, H. Peng Hui, L. Ren. RBF-ARX Modeling and Predictive Control Strategy Applied to a Liquid Level System Proceedings of the 26th Chinese Control Conference.2007,342-346.
[102]E. S. Cnng, S. Chen, and B. Mulgrew. Gradient Radial Basis Function Networks for Nonlinear and Nonstationary Time Series Prediction. IEEE TRANSACTIONS ON NEURAL NETWORKS. 1996,7(1):pp 190-194.
[103]M. J. D. Powell. Radial basis function approximations to polynomials, Proceedings of 12th Biennial Numerical Analysis Conference, Dundee.1987,223-241.
[104]M. Casdagli. Nonlinear prediction of chaotic time-series. Physica D,1989,35:335-356.
[105].S. Chen, C. F. N. Cowan, and P. M. Grant. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans.Neural Network.1991,2:302-309.
[106]H. Peng, T. Ozaki, V. Haggan-Ozaki and Y. Toyoda. Structured parameter optimization method for the radial basis function-based state-dependent autoregressive model. International Journal of Systems Science.2002,33:1087-1098.
[107]R.J.S. Jang, N. Gulley, Fuzzy Logic Toolbox User's Guide, The MathWorks, Inc., Natick, MA, 2000.
[108]H. Kantz, T. Schreiber. Nonlinear Time Series Analysis.Cambridge. UK:Cambridge University Press.1997.
[109]P.D. McNelis. Neural Networks in Finance:Gaining Predictive Edge in the Market. Elsevier Academic Press, Burlington, MA.200, chapters 3-4.
[110]M.S. Kim, C.H. Kim, J.J. Lee. Evolving compact and interpretable Takagi-Sugeno fuzzy models with a new encoding scheme. IEEE Trans. Syst. Man Cybern.2006,36 (5):1006-1023.
[111]M. Sugeno, T. Yasukawa. A fuzzy-logic-based approach to qualitative modeling, IEEE Trans. Fuzzy Syst.1993,1 (1):7-31.
[112]Y. Lin, G.A. Cunningham. A new approach to fuzzy-neural system modeling, IEEE Trans. Fuzzy Syst.1995,3 (2):190-197.
[113]J. Nie, Constructing fuzzy model by self-organizing counterpropagation network, IEEE Trans. Syst. Man Cybernet.1995 25 (6):963-970.
[114]T. Ozaki, P. V. Sosa, and V. Haggan-Ozaki, "Reconstructing the nonlinear dynamics of epilepsy data using nonlinear time series analysis," Journal of Signal Processing, Vol.3,1999, pp.153-162.
[115]D.W. Marquardt. An algorithm for least-squares estimation of nonlinear parameters. Journal of the Society for Industrial and Applied Mathematics.1963,11:431-441.
[116]D. Gorinevsky. An approach to parametric nonlinear least square optimization and application to task-level learning control. IEEE Transactions on Automatic Control.1997,42:912-927.
[117]D. Gorinevsky, D. E. Torfs, and A. A. Goldenberg. Learning approximation of feedforward control dependence on the task parameters with application to direct-drive manipulator tracking. IEEE Transactions on Robotics and Automation.1997,13:567-581.
[118]G. Prasad, E. Swidenbank, and B. W. Hogg. A Local Model Networks Based Multivariable Long-range Predictive Control Strategy for Thermal Power Plants. Automatica.1998,34(10):1185-1204.
[119]G. Armano, M. Marchesi, A. Murru. A hybrid genetic-neural architecture for stock indexes forecasting," Information Sciences.2005,170 (1):3-33.
[120]J. A. Leonard and M. A. Kramer. Radial basis function networks for classifying process faults. IEEE Control Systems Magazine.1991,11(3):31-38.
[121]A. Grossman and J. Morlet. Decomposition of Hardy functions into square integrable wavelets of constant shape. SIAM Journal of Mathematical Analysis1984,15:723-736.
[122]Y. Meyer. Principe d'incertitude, bases hilbertiennes et algebres d'op6rateurs. Seminaire Bourbaki. 1986,662:209-223.
[123]I. Daubechies. Orthonormal basis of compactly supported wavelets. Communications on Pure and Applied Mathematics.1988,41(7):909-996.
[124]S. G. Mallat. A theory for multiresolution signal decomposition:the wavelet representation. IEEE Transactions on Pattern Analysis and Machine. Intelligence.1989,11(7):674-693.
[125]H. H. Szu, B. A. Telfer, and S. L. Kadambe. Neural network adaptive wavelets for signal representation and classification. Optical Engineering.1992,31(9):1907-1915.
[126]Y. C. Pati, and P. S. Krishnaprasad. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks.1993,4:73-85.
[127]A. A. Safavi. Wavelet neural networks and multiresolution analysis with applications to process systems engineering. Ph.D. thesis, The University of Sydney, Australia.1996.
[128]A. A. Safavi and J. A. Romagnoli. Application of Wavelet-based Neural Networks to the Modelling and Optimisation of an Experimental Distillation Column. Engineering Applications of Artificial Intelligence.1997,10(3):301-313.
[129]Q, Zhang, and A Benveniste. A Wavelet networks. IEEE Transactions on Neural Networks.1992, 3(6),889-898.
[130]B. R, Bakshi. and G, Stephanopoulos. Wave-net a multiresolution. hierarchical neural network with localised learning. AIChE. Journal 1993,39(1):57-81.
[131]M. Meyer and G. Pfeiffer. Multilayer perceptron based decision feedback equalizers for channels with intersymbol interference. In IEE Proceedings-I.1993,140:420-424.
[132]C. J. Lin. Wavelet Neural Networks with a Hybrid Learning Approach. Journal of Information Science and Engineering.2006,1367-1387.
[133]B. Fischer, O. Nelles, R. Isermann. Adaptive predictive control of a heat exchanger based on a fuzzy model, Control Eng. Practice.1998,6:259-269.
[134]B. Foss, T.A. Johansen. On local and fuzzy modelling, Proceedings of 3rd International Industrial Fuzzy Control and Intelligent Systems.1993,134-139.
[135]G. H. Golub and C. F. Van Loan, Matrix Computations,3rd ed. Baltimore, MD:Johns Hopkins Univ. Press,1996.
[136]T. Coleman, M.A. Branch, A. Grace, Optimization Toolbox User's Guide, The MathWorks, Inc., Natick,MA,1999.
[137]C. J. Lin and C. T. Lin. An ART-based fozzy adaptive learning control network. IEEE Transactions on Fuzzy Systems, Vol.5,1997, pp.477-496.
[138]C. F. Juang and C. T. Lin. An on-line self-constructing neural fuzzy inference network and its applications. IEEE Transactions on Fuzzy Systems.1998,6:.12-31.
[139]F. J. Lin, C. H. Lin, and P. H. Shen. Self-constructing fuzzy neural network speed controller for permanent-magnet synchronous motor drive. IEEE Transactions on Fuzzy Systems.2001,9:751-759.
[140]Wikipedia
[141]K. Teo, L. Wang, Z. Lin. Wavelet packet multi-layer perceptron for chaotic time series prediction: effects of weight initialization. Computational Science.2001,2074:310-317.
[142].G. McNish, J.V. Lincoln. Prediction of sunspot numbers. Transactions American Geophysical Union.1949,30(5):673-685.
[143]S. Sello, Solar cycle forecasting:a nonlinear dynamics approach. Astronomy & Astrophysics. 2001,377:312-320.
[144]K. Denkmayr, P. Cugnon. About sunspot number medium-terra predictions. In:G. Heckman et al. (Ed.), Solar-Terrestrial Prediction Workshop V, Hiraiso Solar Terrestrial Research Center, Japanl997, p.103.
[145]T. Koskela, M. Lehtokangas, J. Saarinen, K. Kaski. Time series prediction with multilayer perceptron, FIR and Elman neural networks. In Proceedings of the World Congress on Neural Networks.1996,491-496.
[146]SIDC (World Data Center for the Sunspot Index),
[148]R. M. Tong. The evaluation of fuzzy model derived from experimental data. Fuzzy Sets and Systems.1980,4:1-12.
[149]W. Pedrycz. An identification algorithm in fuzzy relational systems. Fuzzy Sets Syst.1984 13(2): 153-167.
[150]C. Xu and Y. Lu. Fuzzy model identification and self-learning for dynamic systems. IEEE Trans. Syst, Man, Cybern.1987,17(4); 683-689.
[151]J. Kim*, N. Kasabov. FIS:adaptive neuro-fuzzy inference systems and their application to nonlinear dynamical systems Neural Networks. PERGAMON.1999,12:1301-1319.
[152]Szu, H. H., Teller, B. and Kadambe, S. Neural network adaptive wavelets for signal representation and classification. Optical Engineering.1992,31(9):1907-1915.
[153]Y. C. Pati, P. S. Krishnaprasad. Analysis and synthesis of feedforward neural networks using discrete affine wavelet transformations. IEEE Transactions on Neural Networks.1993,4 (1):1993 73-85.
[154]J. H. Schwarz. Estimating the dimension of a model. The Annals of Statistics.1978,6:461-464.
[155]M.J.L. Orr. Introduction to radial basis function networks,
[156]C. I. Mosier, Problems and designs of cross-validation. Educational and Psychological Measurement.1951,11:5-11.
[157]H. kantz and T. Schreiber, Nonlinear Time Series Analysis, Cambridge, UK:Cambridge University press.1998.
[158]H. Peng, k. Ohtsu, G. Kitagawa and H. Oda. A statistical modeling and tracking control approach to marine vehicle. IEEE International Conference on Control Applications.2010,640-645.
[159]K. D. Do, Z. P. Jiang, and J. Pan. Universal controllers for stabilization and tracking of underactuated ships. Systems & Control Letters,2002,47:299-317.
[160]Z. P. Jiang, "Global tracking control of underactuated ships by Lyapunov's direct method," Automatica,2002,38:301-309.
[161]K. Y. Pettersen; H. Nijmeijer, "Underactuated ship tracking control:theory and experiments," Int. J. of Control,2001,74:.1435-1446.
[162]E. Lefeber, K. Y. Pettersen, and H. Nijmeijer, "Tracking control of an underactuated ship," IEEE Trans. Control System Technology,2003,11:52-61.
[163]S. Miyoshi, Y. Hara, K. Ohtsu, "Study on Optimum Tracking Control with Linearized Model for Vessel [in Japanese]," Japan Institute of Navigation,2007,11:183-189.
[164]E. Lopez, F. J. Velasco, E. Moyano and T. M. Rueda full-scale manoeuvering trials Simulation Journal of Maritime Research,1(3); 37-50.
[165]J. S. Park, K. Ohtsu, and G. Kitagawa, Batch-adaptive ship's autopilotst, Int. J. Adapt. Control Signal Process,,2000,14:427-439.
[166]K. Ohtsu and G. Kitagawa, Statistical Analysis of the AR Type Ship's Autopilot System, Journal of Dynamic System, Measurement and Control.1984,106; 193-202.
[167]K. Ohtsu, M. Horigome and G. Kitagawa. A new ship's auto pilot'design through a stochastic model, Automatica.1979,15:255-268.
[168]T. Iseki, and K. Ohtsu, "Bayesian estimation of directional wave spectra based on ship motions," Control Engineering Practice.2000,8:215-219.
[169]H. Oda, K. Ohtsu and T. Hotta, "Statistical analysis and design of a rudder roll stabilization system," Control Engineering Practice.1996,4:351-358.