基于Hadoop平台的招聘数据分析

详细信息    查看全文 | 推荐本文 |

英文篇名：Recruitment Data Analysis Using Hadoop Platform 作者：武晓军 ; 陈怡丹 ; 赵青杉 英文作者：WU Xiao-jun;CHEN Yi-dan;ZHAO Qi-shan;Department of Computer,Xinzhou Teachers University;College of Information Engineering,Henan Radio & Television University; 关键词：Hadoop ; 奇异值分解 ; 关联规则 ; 分析 英文关键词：Hadoop;;Singular Value Decomposition;;Association Rules;;Analysis 中文刊名：XDJS 英文刊名：Modern Computer 机构：忻州师范学院计算机系;河南广播电视大学信息工程学院; 出版日期：2019-07-05 出版单位：现代计算机 年：2019 基金：智能信息处理山西省重点实验室开放基金项目(No.2016002) 语种：中文; 页：XDJS201919002 页数：5 CN：19 ISSN：44-1415/TP 分类号：5-8+14

摘要

结合Hadoop平台的高扩展性、高性能、与低成本的优点,设计基于Hadoop招聘数据分析的框架。对近200万条数据分词、去重、去噪、提取特征,构造特征矩阵与文本矩阵,利用奇异值分解法对文本矩阵降维,按相似度分类,对分类结果进行关联规则挖掘与数据统计分析。结果表明Hadoop平台数据分析效率明显提高,具有较高的加速比。实验结果(IT行业)呈现目前就业岗位、薪资、所需技能、工作地点的关联规则与统计结果分析,为行业的发展与就业提供一定的数据参考与支撑。
        Combining the advantages of Hadoop platform that has high scalability, high performance and low-cost, the framework of recruitment data analysis using Hadoop platform can be built already. The procedures are executing following operations on almost 2 million data: segmenting, duplication eliminating, denoising and extracting features at first. Then constructing characteristic matrix and text matrix. After that reducing the dimension of text matrix by using singular value decomposition method. Finally classifying them by similarity, and doing association rules mining and data statistical analysis on the classifications. The results show that Hadoop platform can provide more effective data analysis function, more significantly, it has higher speedup. Experiments results represent the association rules and statistical analysis of current jobs, salaries, skills required and workplaces. It surely could provide certain data reference and support to industry development as well as employment.

引文

[1]钟晓旭,胡学钢.基于数据挖掘的Web招聘信息相关性分析[J].安徽建筑工业学院学报(自然科学版),2010,18(04):93-96.
    [2]王静. Web对象的信息抽取的关键技术研究[D].西安:西安电子科技大学,2011.
    [3]张学新,贾园园,饶希,蔡黎.海量非结构化网络招聘数据的挖掘分析[J].长春师范大学学报,2017,36(10):28-36.
    [4]谌志华.基于大数据的网络舆情分析系统[J].现代电子技术,2017,40(24):15-17.
    [5]郝艳妮,田维丽.基于Hadoop的数据挖掘算法在葡萄酒信息数据分析系统中的应用[J].计算机应用,2017,37(S1):72-74+79.
    [6]张登耀.基于Hadoop分布式文件系统的商业银行大数据分析[J].山东农业大学学报(自然科版),2018,49(05):884-888.
    [7]Bendre M,Manthalkar R. Time Series Decomposition and Predictive Analytics Using MapReduce Framework[J]. Expert Systems with Applications,2019,116:108-120.
    [8]邬启为.基于向量空间的文本聚类方法与实现[D].北京:北京交通大学,2014.
    [9]廖飞.基于关联规则的试题生成与数据分析方案研究[D].广州:华南理工大学,2018.