作业管理系统的研究与实现
|
摘要
作业管理的概念非常重要，目的在于强化操作系统的批处理功能，提供对作业的提交、调度、执行及控制等机制，从而能够更加有效地利用系统资源、平衡网络负载，提高系统的整体性能。自1986年网络队列系统NQS面世以来，作业管理系统取得了长足的发展。国外一些大型的研究机构和公司都将作业管理系统作为提高生产效率和资源利用率的一项重要技术保证，并面向实际需求进行了大量的研究和开发工作。但国内该领域的研究尚未深入展开。
作者借参加西北工业大学计算机软件与理论系的国际合作项目之契机，对作业管理系统进行了系统的深入研究。本文总结了作者参加该项目的三年间的主要研究成果，包括如下内容：
1)根据作业管理的普遍原理，勾勒出作业管理系统MTZJMS的体系结构框架。在此基础上，重点研究了MTZJMS系统中作业的图形化描述方法及实现；强化了作业网络的自动提交(也称日程表驱动调度)机制，从而，尽可能的自动化并且更有效率的完成诸如收集每日的业务数据、创建日报、月报等周期性的任务，充分利用企业宝贵的人力资源；设计了多机器作业网络轨迹跟踪监视机制；实现了作业网络轨迹的控制和同步，从而实现分布式环境中对作业网络轨迹的集中管理。
2)对作业调度特别是请求调度算法及其实现进行了深入的研究，在对一些影响和制约负载平衡的因素进行了讨论和分析的基础上，设计了加权批量作业按比例调度算法，实现了作业管理系统MTJZJMS的负载平衡机制，该系统可以通过改变管道队列属性的方法，灵活的选取负载平衡算法。
3)探讨国际化作业管理系统中的多时区问题，提出并实现了作业管理系统MTZJMS的多时区模型，并对该模型进行简要的评价。这是国内对相关问题的首次尝试，MTZJMS系统成功的运行，验证了该模型的实用性。
The concept of Job Management is quite important, for the purpose of strengthening the batch processing of operating system, providing the mechanism of submitting, scheduling, executing and controlling jobs, and hence taking more effective use of the resources in the system, implementing load balance, improving the performance of the whole system. Since the Network Queuing System (NQS) came into being in 1986, the job management system has made great improvement. Many large research organizations and corporations have been using job management systems as an important warranty of improving productive efficiency and promoting ratio of resource usage and lots of researches and development have been carried out according to the actual need. However, the domestic research in this field hasn't been carried out extensively and deeply.
By the opportunity of taking part in the international project prompted by the Department of Software and Theory of NPU, I made a deep research on the job management system systematically. This thesis summarizes the main achievements of my researches in the last three years, it covers the following aspects:
1) According to the common theory of job management, the thesis outlines the architecture of the job management system, Mutiple Time Zone Job Management System (MTZJMS). And based on this, I put my emphasis on the methods of graphic description of job and its implement, improved the mechanism of submitting job network automatically (i.e. schedule driven submitting), thus, tasks that have to be performed regularly - such as collecting daily transaction data, creating daily reports, and generating monthly reports -can be automated and streamlined as much as possible so that enterprises can make better use of their valuable human resources; I designed the mechanism of monitoring job network trackers of multiple computers, implemented the mechanism of controlling and synchronizing the job network trackers, this turns the centralized management of job network trackers in a distributed environment into realities.
2) I also studied the algorithm and implements of scheduling jobs especially requests. Based on the discussion and analysis of the factors that influence the load balance, I designed the weighted proportional job scheduling algorithm, and implemented the mechanism of load balance in MTZJMS, it can choose different scheduling algorithm flexibly by changing the property of pipe queue.
3) I've probed into the main problems of multiple time zones in implementing
international job management system, brought forward and implemented the multiple time zones model of our job management system, MTZJMS, and then made a brief estimate of this model. This is the first attempt trying to solve the relative problem in our country. The successful operating of MTZJMS proved the practicability of this model.
作者借参加西北工业大学计算机软件与理论系的国际合作项目之契机，对作业管理系统进行了系统的深入研究。本文总结了作者参加该项目的三年间的主要研究成果，包括如下内容：
1)根据作业管理的普遍原理，勾勒出作业管理系统MTZJMS的体系结构框架。在此基础上，重点研究了MTZJMS系统中作业的图形化描述方法及实现；强化了作业网络的自动提交(也称日程表驱动调度)机制，从而，尽可能的自动化并且更有效率的完成诸如收集每日的业务数据、创建日报、月报等周期性的任务，充分利用企业宝贵的人力资源；设计了多机器作业网络轨迹跟踪监视机制；实现了作业网络轨迹的控制和同步，从而实现分布式环境中对作业网络轨迹的集中管理。
2)对作业调度特别是请求调度算法及其实现进行了深入的研究，在对一些影响和制约负载平衡的因素进行了讨论和分析的基础上，设计了加权批量作业按比例调度算法，实现了作业管理系统MTJZJMS的负载平衡机制，该系统可以通过改变管道队列属性的方法，灵活的选取负载平衡算法。
3)探讨国际化作业管理系统中的多时区问题，提出并实现了作业管理系统MTZJMS的多时区模型，并对该模型进行简要的评价。这是国内对相关问题的首次尝试，MTZJMS系统成功的运行，验证了该模型的实用性。
The concept of Job Management is quite important, for the purpose of strengthening the batch processing of operating system, providing the mechanism of submitting, scheduling, executing and controlling jobs, and hence taking more effective use of the resources in the system, implementing load balance, improving the performance of the whole system. Since the Network Queuing System (NQS) came into being in 1986, the job management system has made great improvement. Many large research organizations and corporations have been using job management systems as an important warranty of improving productive efficiency and promoting ratio of resource usage and lots of researches and development have been carried out according to the actual need. However, the domestic research in this field hasn't been carried out extensively and deeply.
By the opportunity of taking part in the international project prompted by the Department of Software and Theory of NPU, I made a deep research on the job management system systematically. This thesis summarizes the main achievements of my researches in the last three years, it covers the following aspects:
1) According to the common theory of job management, the thesis outlines the architecture of the job management system, Mutiple Time Zone Job Management System (MTZJMS). And based on this, I put my emphasis on the methods of graphic description of job and its implement, improved the mechanism of submitting job network automatically (i.e. schedule driven submitting), thus, tasks that have to be performed regularly - such as collecting daily transaction data, creating daily reports, and generating monthly reports -can be automated and streamlined as much as possible so that enterprises can make better use of their valuable human resources; I designed the mechanism of monitoring job network trackers of multiple computers, implemented the mechanism of controlling and synchronizing the job network trackers, this turns the centralized management of job network trackers in a distributed environment into realities.
2) I also studied the algorithm and implements of scheduling jobs especially requests. Based on the discussion and analysis of the factors that influence the load balance, I designed the weighted proportional job scheduling algorithm, and implemented the mechanism of load balance in MTZJMS, it can choose different scheduling algorithm flexibly by changing the property of pipe queue.
3) I've probed into the main problems of multiple time zones in implementing
international job management system, brought forward and implemented the multiple time zones model of our job management system, MTZJMS, and then made a brief estimate of this model. This is the first attempt trying to solve the relative problem in our country. The successful operating of MTZJMS proved the practicability of this model.
引文
[1]Netshepherd & SystemScope/JobCenter User's Guide, NEC Corporation, 1998
[2]陆松年，操作系统教程，电子工业出版杜，2000年10月1日．
[3]S. Zhou, LSF: Load Sharing and Batch Queueing Software, Platform Computing Corporation, North York, Canada, 1996.
[4]W. Saphire, L.A. Tanner, and B. Traversat, Job Management Requirements for NAS Parallel Systems and Clusters, Technical Report NAS-95-006, NASA Ames Research Center, 1995
[5]http://www.microsoft.com/
[6]严蔚敏，吴伟民，“数据结构”，清华大学出版社，1990．
[7]汤小春，李战怀，郑炜，一个基于偏序的定时投入关联网络作业调度算法，计算机研究与发展，1998．4
[8]http://www.sw.nec.co.jp/middle/SystemScope/Products/kousei.html.
[9]NEC Corporation.SystemScope／JobCenter User's Guide．东京都港区芝五丁目7番1号．1999．
[10]Brent A.Kingsbury, The Network Queueing System, Sterling Software, 1999
[11]Load Share Facility: User's and Administrator's Guide, Platform Computing Corporation,Toronto, Canada, Aug 2000.
[12]Batch Queueing Systems, Scott Presnell, August 1998
[13]Andrew S.Tanenbaum著，陆丽娜 伍卫国 刘隆国 等译校 “分布式操作系统”，电子工业出版社、PRENTICE HALL出版公司，1999年12月第一版
[14]Microsoft corporation MSDN Library July 2003.
[15]陆丽娜、伍卫国等译校，分布式操作系统，北京：电子工业出版社，1999
[16]Robert L. Henderson, Dave Tweten, Portable Batch System Requirements Specification,NAS Scientific Computing Branch, NAS Systems Division, NASA Ames Research Center,August 1998.
[17]汤子瀛、哲凤屏、汤小丹“计算机操作系统”，西安电子科技大学出版社，1996年12月第一版
[18]"Portable Batch System Administrator Guide", Veridian Systems, Inc., October 2000.
[19]Stuart Herbert, "Features Of Generic NQS", June 1996.
[20]David M. Carver, "Using the Network Queueing System (NQS)", 1998.
[21]Albeaus Bayucan, etc., "Portable Batch System Internal Design Specification", Numerical Aerospace Simulation Systems Division NASA Ames Research Center, October 1998
[22]Robert L. Henderson, Dave Tweten, "Portable Batch System Requirements Specification", NAS Scientific Computing Branch, NAS Systems Division, NASA Ames Research Center, August 1998
[23]Albeaus Bayucan, etc., "Portable Batch System External Reference Specification",Numerical Aerospace Simulation System Division, NASA Ames Research Center, August 1998
[24]Albeaus Bayucan, etc., "Portable Batch System Administrator Guide", Numerical Aerospace Simulation System Division, NASA Ames Research Center, August 1998
[25]"Batch Queueing Systems", Scott Presnell,August 1998
[26]Harsh Anand, "Batch Differences: NQE/NQS vs. LoadLeveler",http://hpcf. nersc.gov,August 2001.
[27]"Network Queuing System", http://www.reading.ac.uk/ITS/Topic/UnixOS/UnSQnqs_01/April 1998
[28]IEEE P1003.2 Draft 11.2, Institute of Electrical and Electronics Engineers, Inc, 1991.9.
[29]Joseph A. Kaplan, Micheal L. Nelson, "A Comparison of Queuing, Cluster and Distributed Computing Systems", NASA Langley Research Center, June 1994.
[30]Stuart Herbert, "Changes To Generic NQS v3.50.0",June 1996.
[31]Nicholas P. Cardo, Batch Scheduling: A Fresh Approach, Sterling Software, Inc.
[32]Vijay K. Naik, An APl for Dynamic Job schedulers, IBM T. J. Watson Research Center,October 1996
[33]辞海1979年版上海辞书出版社
[2]陆松年，操作系统教程，电子工业出版杜，2000年10月1日．
[3]S. Zhou, LSF: Load Sharing and Batch Queueing Software, Platform Computing Corporation, North York, Canada, 1996.
[4]W. Saphire, L.A. Tanner, and B. Traversat, Job Management Requirements for NAS Parallel Systems and Clusters, Technical Report NAS-95-006, NASA Ames Research Center, 1995
[5]http://www.microsoft.com/
[6]严蔚敏，吴伟民，“数据结构”，清华大学出版社，1990．
[7]汤小春，李战怀，郑炜，一个基于偏序的定时投入关联网络作业调度算法，计算机研究与发展，1998．4
[8]http://www.sw.nec.co.jp/middle/SystemScope/Products/kousei.html.
[9]NEC Corporation.SystemScope／JobCenter User's Guide．东京都港区芝五丁目7番1号．1999．
[10]Brent A.Kingsbury, The Network Queueing System, Sterling Software, 1999
[11]Load Share Facility: User's and Administrator's Guide, Platform Computing Corporation,Toronto, Canada, Aug 2000.
[12]Batch Queueing Systems, Scott Presnell, August 1998
[13]Andrew S.Tanenbaum著，陆丽娜 伍卫国 刘隆国 等译校 “分布式操作系统”，电子工业出版社、PRENTICE HALL出版公司，1999年12月第一版
[14]Microsoft corporation MSDN Library July 2003.
[15]陆丽娜、伍卫国等译校，分布式操作系统，北京：电子工业出版社，1999
[16]Robert L. Henderson, Dave Tweten, Portable Batch System Requirements Specification,NAS Scientific Computing Branch, NAS Systems Division, NASA Ames Research Center,August 1998.
[17]汤子瀛、哲凤屏、汤小丹“计算机操作系统”，西安电子科技大学出版社，1996年12月第一版
[18]"Portable Batch System Administrator Guide", Veridian Systems, Inc., October 2000.
[19]Stuart Herbert, "Features Of Generic NQS", June 1996.
[20]David M. Carver, "Using the Network Queueing System (NQS)", 1998.
[21]Albeaus Bayucan, etc., "Portable Batch System Internal Design Specification", Numerical Aerospace Simulation Systems Division NASA Ames Research Center, October 1998
[22]Robert L. Henderson, Dave Tweten, "Portable Batch System Requirements Specification", NAS Scientific Computing Branch, NAS Systems Division, NASA Ames Research Center, August 1998
[23]Albeaus Bayucan, etc., "Portable Batch System External Reference Specification",Numerical Aerospace Simulation System Division, NASA Ames Research Center, August 1998
[24]Albeaus Bayucan, etc., "Portable Batch System Administrator Guide", Numerical Aerospace Simulation System Division, NASA Ames Research Center, August 1998
[25]"Batch Queueing Systems", Scott Presnell,August 1998
[26]Harsh Anand, "Batch Differences: NQE/NQS vs. LoadLeveler",http://hpcf. nersc.gov,August 2001.
[27]"Network Queuing System", http://www.reading.ac.uk/ITS/Topic/UnixOS/UnSQnqs_01/April 1998
[28]IEEE P1003.2 Draft 11.2, Institute of Electrical and Electronics Engineers, Inc, 1991.9.
[29]Joseph A. Kaplan, Micheal L. Nelson, "A Comparison of Queuing, Cluster and Distributed Computing Systems", NASA Langley Research Center, June 1994.
[30]Stuart Herbert, "Changes To Generic NQS v3.50.0",June 1996.
[31]Nicholas P. Cardo, Batch Scheduling: A Fresh Approach, Sterling Software, Inc.
[32]Vijay K. Naik, An APl for Dynamic Job schedulers, IBM T. J. Watson Research Center,October 1996
[33]辞海1979年版上海辞书出版社