A Secondary Index Scheme of Big Data in HBase Based on Solr

doi:10.3969/j.issn.1671-1122.2017.08.006

Abstract

Abstract:

For the problem that HBase cannot provide secondary indexes and Huawei hindex scheme is difficult to meet the project demand of retrieval speed, designed an HBase secondary index scheme named SIHBase (Solr Indexing HBase) based on Solr. The scheme uses HBase Coprocessor to achieve the related callback function for creating, changing, deleting operations of the data tables and inserting, updating, deleting recovering operations of data. Thus, it can create and manage secondary indexes in Solr for HBase automatically and ensure the consistency of the data and index. The scheme has favorable generality and can create index for multi-column data of multiple tables in the meantime. And then extended the client-side function of HBase, increased the direct query interface of Solr, using efficient, flexible and diversified retrieval functions which Solr provided to realize quick retrieval for the mass data of HBase. Finally, a contrast experiment about query performance with hindex show that SIHBase was turned out to be much faster than hindex in query speed.

Key words: HBase, secondary index, Solr, quick retrieval

CLC Number:

TP309

Wenxian WANG, Xingshu CHEN, Haizhou WANG, Xiaosong WU. A Secondary Index Scheme of Big Data in HBase Based on Solr[J]. Netinfo Security, 2017, 17(8): 39-44.

Figures/Tables 7

References 19

[1]	中国互联网信息中心. CNNIC 中国互联网络发展状况统计报告 [EB/OL]. , 2017-2-15.
[2]	江民彬. 非关系型与关系型空间数据库对比分析与协同应用研究[D]. 北京: 首都师范大学, 2013.
[3]	The Apache Software Foundation. Apache HBase [EB/OL]. , 2016-6-15.
[4]	WANG D, XIAO L.Storage and Query of Condition Monitoring Data in Smart Grid Based on Hadoop[C] // IEEE Computer Society. 2012 Fourth International Conference on Computational and Information Sciences (ICCIS), August 17-19, 2012, Chongqing, China. Washington, DC, USA: IEEE Computer Society, 2012: 377-380.
[5]	宋衍,周庆,张国双,等. 一种基于数据库服务的密文检索实现[J]. 信息网络安全,2015(9):10-14.
[6]	NISHIMURA S, DAS S, AGRAWAL D, et al.MD-HBase: a Scalable Multi-dimensional Data Infrastructure for Location Aware Services[C] //Luleå University of Technology. 12th IEEE International Conference on Mobile Data Management (MDM), June 6-9, 2011, Luleå, Norrbotten, Sweden. Washington, DC, USA: IEEE Computer Society, 2011, 1: 7-16.
[7]	刘浩阳. MS SQL数据库在线取证研究[J]. 信息网络安全,2016 (9):26-30.
[8]	CLINT M. Indexed Transactional HBase [EB/OL]. , 2016-6-15.
[9]	YORAM K. IHBase [EB/OL]. , 2016-6-15.
[10]	陈新鹏. 基于HBase的数据生成与索引方法的研究[D]. 北京: 北京邮电大学, 2014.
[11]	张榆, 马友忠, 孟小峰. 一种基于HBase的高效空间关键字查询策略[J]. 小型微型计算机系统, 2012, (10): 2141-2146.
[12]	GAO X, ROTH E, MCKELVEY K, et al.Cloud Computing for Data-Intensive Applications[M]. New York: Springer, 2014.
[13]	CHENG P, AN J.The Key as Dictionary Compression Method of Inverted Index Table under the Hbase Database[J]. Journal of Software, 2013, 8(5): 1086-1093.
[14]	MINGJIE L, EUGENE K, ANDREW P. Apache HBase [EB/OL]. , 2016-06-15.
[15]	丁飞, 陈长松, 张涛,等. 基于协处理器的 HBase区域级第二索引研究与实现[J]. 计算机应用, 2014(S1): 181-185.
[16]	卓海艺. 基于HBase的海量数据实时查询系统设计与实现[D]. 北京: 北京邮电大学, 2013.
[17]	邹敏昊. 基于Lucene的HBase全文检索功能的设计与实现[D]. 南京: 南京大学, 2013.
[18]	华为. HBase二级索引hindex [EB/OL]. , 2016-6-15.
[19]	RAJESHBABU C. hindex [EB/OL]. , 2016-6-15.