摘要：前言項(xiàng)目中的由升級(jí)至。已經(jīng)棄用，相應(yīng)功能由實(shí)現(xiàn)，直接替換即可。構(gòu)造報(bào)文調(diào)整調(diào)整成棄用，相關(guān)功能由實(shí)現(xiàn)。類(lèi)型表示精確查找的文本，不需要進(jìn)行分詞。查詢(xún)字段時(shí)，使用表示改版后，設(shè)置了的情況下，也要設(shè)置，否則會(huì)報(bào)。
前言
項(xiàng)目中的es由ver.1.4.5升級(jí)至ver.5.2.0。
安裝elasticSearch
#下載
wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-5.2.0.tar.gz
# 解壓
tar zxvf elasticsearch-5.5.0.tar.gz
修改elasticsearch.yml
$ES_HOME/config/elasticsearch.yml
在這里不詳細(xì)展開(kāi)elasticsearch.yml的各個(gè)配置項(xiàng)，附上鏈接。
配置es外部鏈接
安裝elasticsearch-head
lasticsearch-head是一個(gè)很好的可視化前端框架，方便用可視化界面對(duì)es進(jìn)行調(diào)用。elasticsearch-head在Github的地址如下：https://github.com/mobz/elast...，安裝也不復(fù)雜，由于它是一個(gè)前端的工具，因此需要我們預(yù)先安裝了node和npm，之后執(zhí)行下面的步驟：
git clone git://github.com/mobz/elasticsearch-head.git
cd elasticsearch-head
npm install
安裝完成后，運(yùn)行命令npm run start就行。
調(diào)整棄用api的兼容問(wèn)題
1.setting
1.4.5的org.elasticsearch.common.settings.ImmutableSettings已經(jīng)棄用，生成配置對(duì)象setting的方式改成：
Settings settings = Settings.builder().put("cluster.name", clusterName).put("client.transport.sniff", true).build();
2.InetSocketTransportAddress
org.elasticsearch.common.transport.InetSocketTransportAddress#InetSocketTransportAddress(java.lang.String, int)方法已經(jīng)棄用，注入集群地址的方式改成：
clusterNodeAddressList.add(new InetSocketTransportAddress(InetAddress.getByName(host), 9300));
3.TransportClient
org.elasticsearch.client.transport.TransportClient#TransportClient(org.elasticsearch.common.settings.Settings)，該構(gòu)造方法已經(jīng)棄用，生成TransportClient實(shí)例的方式改成：
transportClient = new PreBuiltTransportClient(settings);
4.ClusterHealthStatus
org.elasticsearch.action.admin.cluster.health.ClusterHealthStatus類(lèi)已經(jīng)棄用，相同功能由org.elasticsearch.cluster.health.ClusterHealthStatus繼承
5.ScriptSortBuilder調(diào)整
原版寫(xiě)法：
Map optionalSortMap = manualSortMapBuilder.put("other", sortIndex + 1).build();
String script = "paramsMap.containsKey(doc["%s"].value) ? paramsMap.get(doc["%s"].value) : paramsMap.get("other")";
script = String.format(script, sort.getFieldName(), sort.getFieldName());
sortBuilder = SortBuilders.scriptSort(script, "number").param("paramsMap", optionalSortMap).order(SortOrder.ASC).missing(optionalSortMap.get("other"));
調(diào)整為：
Map optionalSortMap = manualSortMapBuilder.put("other", sortIndex + 1).build();
String script = "paramsMap.containsKey(doc["%s"].value) ? paramsMap.get(doc["%s"].value) : paramsMap.get("other")";
script = String.format(script, sort.getFieldName(), sort.getFieldName());
Map params = Maps.newConcurrentMap();
params.put("paramsMap", optionalSortMap);
Script scriptObject = new Script(Script.DEFAULT_SCRIPT_TYPE, Script.DEFAULT_SCRIPT_LANG, script, params);
sortBuilder = SortBuilders.scriptSort(scriptObject, ScriptSortBuilder.ScriptSortType.fromString("number")).order(SortOrder.ASC);
6.FilterBuilder調(diào)整
org.elasticsearch.index.query.FilterBuilder類(lèi)已經(jīng)棄用，基本上從2.x版本開(kāi)始，F(xiàn)ilter就已經(jīng)棄用了（不包括bool查詢(xún)內(nèi)的filter），所有FilterBuilder全都要用QueryBuilder的各種子類(lèi)來(lái)調(diào)整：
1.org.elasticsearch.index.query.BoolFilterBuilder
BoolFilterBuilder boolFilterBuilder = FilterBuilders.boolFilter();
調(diào)整為：
BoolQueryBuilder boolFilterBuilder = new BoolQueryBuilder();
2.org.elasticsearch.index.query.NestedFilterBuilder
filterBuilder = FilterBuilders.nestedFilter(param.getPath(), boolFilterBuilder);
調(diào)整為：
filterBuilder = new NestedQueryBuilder(param.getPath(), boolFilterBuilder, ScoreMode.None);
3.org.elasticsearch.index.query.MissingFilterBuilder
5.x版本中，missing關(guān)鍵字已經(jīng)棄用，其功能由其逆運(yùn)算exist繼承。
MissingFilterBuilder missingFilterBuilder = FilterBuilders.missingFilter(paramName);
if (param.getNvlType() == QueryFieldType.EXISTS) {
filterBuilder = FilterBuilders.boolFilter().mustNot(missingFilterBuilder);
}
if (param.getNvlType() == QueryFieldType.MISSING) {
filterBuilder = FilterBuilders.boolFilter().must(missingFilterBuilder);
}
調(diào)整為：
ExistsQueryBuilder existsQueryBuilder = new ExistsQueryBuilder(paramName);
if (param.getNvlType() == QueryFieldType.EXISTS) {
filterBuilder = new BoolQueryBuilder().must(existsQueryBuilder);
}
if (param.getNvlType() == QueryFieldType.MISSING) {
filterBuilder = new BoolQueryBuilder().mustNot(existsQueryBuilder);
}
4.org.elasticsearch.index.query.TermFilterBuilder
filterBuilder = FilterBuilders.termFilter(paramName, param.getEqValue());
調(diào)整為：
filterBuilder = new TermQueryBuilder(paramName, param.getEqValue());
5.org.elasticsearch.index.query.TermsFilterBuilder
filterBuilder = FilterBuilders.inFilter(paramName, param.getInValues());
調(diào)整為：
filterBuilder = new TermsQueryBuilder(paramName, param.getInValues());
6.org.elasticsearch.index.query.RangeFilterBuilder
//gte
if (null != param.getGteValue()) {
filterBuilder = FilterBuilders.rangeFilter(paramName).gte(param.getGteValue());
}
//gt
if (null != param.getGtValue()) {
filterBuilder = FilterBuilders.rangeFilter(paramName).gt(param.getGtValue());
}
//lte
if (null != param.getLteValue()) {
filterBuilder = FilterBuilders.rangeFilter(paramName).lte(param.getLteValue());
}
//lt
if (null != param.getLtValue()) {
filterBuilder = FilterBuilders.rangeFilter(paramName).lt(param.getLtValue());
}
調(diào)整為：
//gte
if (null != param.getGteValue()) {
filterBuilder = new RangeQueryBuilder(paramName).gte(param.getGteValue());
}
//gt
if (null != param.getGtValue()) {
filterBuilder = new RangeQueryBuilder(paramName).gt(param.getGtValue());
}
//lte
if (null != param.getLteValue()) {
filterBuilder = new RangeQueryBuilder(paramName).lte(param.getLteValue());
}
//lt
if (null != param.getLtValue()) {
filterBuilder = new RangeQueryBuilder(paramName).lt(param.getLtValue());
}
7.search_type=count
原來(lái)我們想要計(jì)算文檔的需要用到search_type=count，現(xiàn)在5.0已經(jīng)將該API移除，取而代之你只需將size置于0即可：
GET /my_index/_search?search_type=count
{
"aggs": {
"my_terms": {
"terms": {
"field": "foo"
}
}
}
}
調(diào)整為：
#5.0以后
GET /my_index/_search
{
"size": 0,
"aggs": {
"my_terms": {
"terms": {
"field": "foo"
}
}
}
}
8.RangeBuilder
org.elasticsearch.search.aggregations.bucket.range.RangeBuilder已經(jīng)棄用，相應(yīng)功能由org.elasticsearch.search.aggregations.bucket.range.RangeAggregationBuilder實(shí)現(xiàn)，直接替換即可。
9.TopHitsAggregationBuilder
org.elasticsearch.search.aggregations.metrics.tophits.TopHitsBuilder已經(jīng)棄用，相應(yīng)功能由org.elasticsearch.search.aggregations.metrics.tophits.TopHitsAggregationBuilder實(shí)現(xiàn)，直接替換即可。
10.FiltersAggregationBuilder
org.elasticsearch.search.aggregations.bucket.filters.FiltersAggregationBuilder構(gòu)造報(bào)文調(diào)整
FiltersAggregationBuilder filtersAggregationBuilder = AggregationBuilders.filters(aggregationField.getAggName());
LufaxSearchConditionBuilder tmpConditionBuilder = new LufaxSearchConditionBuilder();
for (String key : aggregationField.getFiltersMap().keySet()) {
LufaxFilterCondition tmpLufaxFilterCondition = aggregationField.getFiltersMap().get(key);
FilterBuilder tmpFilterBuilder = tmpConditionBuilder.constructFilterBuilder(tmpLufaxFilterCondition.getAndParams(),tmpLufaxFilterCondition.getOrParams(),tmpLufaxFilterCondition.getNotParams());
filtersAggregationBuilder.filter(key, tmpFilterBuilder);
}
調(diào)整成：
List keyedFilters = new LinkedList();
LufaxSearchConditionBuilder tmpConditionBuilder = new LufaxSearchConditionBuilder();
for (String key : aggregationField.getFiltersMap().keySet()) {
LufaxFilterCondition tmpLufaxFilterCondition = aggregationField.getFiltersMap().get(key);
QueryBuilder tmpFilterBuilder = tmpConditionBuilder.constructFilterBuilder(tmpLufaxFilterCondition.getAndParams(),tmpLufaxFilterCondition.getOrParams(),tmpLufaxFilterCondition.getNotParams());
keyedFilters.add(new FiltersAggregator.KeyedFilter(key, tmpFilterBuilder));
}
FiltersAggregationBuilder filtersAggregationBuilder = AggregationBuilders.filters(aggregationField.getAggName(), keyedFilters.toArray(new FiltersAggregator.KeyedFilter[]{}));
11.HighlightBuilder;
org.elasticsearch.search.highlight.HighlightBuilder棄用，相關(guān)功能由org.elasticsearch.search.fetch.subphase.highlight.HighlightBuilder實(shí)現(xiàn)。
12.OptimizeRequestBuilder
org.elasticsearch.action.admin.indices.optimize.OptimizeRequestBuilder 已經(jīng)棄用，聚合索引的功能由org.elasticsearch.action.admin.indices.forcemerge.ForceMergeRequestBuilder來(lái)實(shí)現(xiàn)。
13.IndicesAliasesRequestBuilder
1.newAddAliasAction
舊版刪除了AliasAction類(lèi)的newAddAliasAction方法，故而IndicesAliasesRequestBuilder添加AliasActions應(yīng)該：
requestBuilder.addAliasAction(AliasAction.newAddAliasAction(toIndex, indexAlias));
調(diào)整成
requestBuilder.addAliasAction(IndicesAliasesRequest.AliasActions.add().index(toIndex).alias(indexAlias));
2.newRemoveAliasAction
舊版刪除了AliasAction類(lèi)的newRemoveAliasAction方法，故而IndicesAliasesRequestBuilder刪除AliasActions應(yīng)該：
requestBuilder.addAliasAction(AliasAction.newRemoveAliasAction(fromIdx, indexAlias));
調(diào)整成
requestBuilder.addAliasAction(IndicesAliasesRequest.AliasActions.remove().index(fromIdx).alias(indexAlias));
14.AbstractAggregationBuilder的子類(lèi)變更
1.org.elasticsearch.search.aggregations.bucket.terms.TermsBuilder
org.elasticsearch.search.aggregations.bucket.terms.TermsBuilder更名為
org.elasticsearch.search.aggregations.bucket.terms.TermsAggregationBuilder
2.org.elasticsearch.search.aggregations.bucket.range.date.DateRangeBuilder
org.elasticsearch.search.aggregations.bucket.range.date.DateRangeBuilder更名為
org.elasticsearch.search.aggregations.bucket.range.date.DateRangeAggregationBuilder
3.org.elasticsearch.search.aggregations.metrics.tophits.TopHitsBuilder
org.elasticsearch.search.aggregations.metrics.tophits.TopHitsBuilder更名為
org.elasticsearch.search.aggregations.metrics.tophits.TopHitsAggregationBuilder
15.SearchHit類(lèi)
org.elasticsearch.search.SearchHit#isSourceEmpty方法改為org.elasticsearch.search.SearchHit#hasSource方法，反向替換。
16.DeleteByQueryResponse
org.elasticsearch.action.deletebyquery.DeleteByQueryResponse已經(jīng)棄用，
調(diào)整關(guān)鍵字等結(jié)構(gòu)性問(wèn)題
1. String數(shù)據(jù)類(lèi)型棄用
在 ES2.x 版本字符串?dāng)?shù)據(jù)是沒(méi)有 keyword 和 text 類(lèi)型的，只有string類(lèi)型，ES更新到5版本后，取消了 string 數(shù)據(jù)類(lèi)型，代替它的是 keyword 和 text 數(shù)據(jù)類(lèi)型。區(qū)別在于：
text類(lèi)型定義的文本會(huì)被分析，在建立索引前會(huì)將這些文本進(jìn)行分詞，轉(zhuǎn)化為詞的組合，建立索引。允許 ES來(lái)檢索這些詞語(yǔ)。text 數(shù)據(jù)類(lèi)型不能用來(lái)排序和聚合。
keyWord類(lèi)型表示精確查找的文本，不需要進(jìn)行分詞?？梢员挥脕?lái)檢索過(guò)濾、排序和聚合。keyword 類(lèi)型字段只能用本身來(lái)進(jìn)行檢索。
在沒(méi)有顯性定義時(shí)，es默認(rèn)為“text”類(lèi)型。
2. multi_field關(guān)鍵字棄用
相關(guān)mapping方式改為：
#對(duì)需要設(shè)置的字段，在"type"屬性后增加"fields":
#其中的"raw"為自定義的名稱(chēng),想象它是city的一個(gè)分身。
PUT /my_index
{
"mappings": {
"my_type": {
"properties": {
"city": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
}
}
}
}
查詢(xún)r(jià)aw字段時(shí)，使用city.raw表示
3. analyzer
1.改版后，設(shè)置了search_analyzer的情況下，analyzer也要設(shè)置，否則會(huì)報(bào)：
analyzer on field [name] must be set when search_analyzer is set。
2.改版后，index_analyzer設(shè)置被棄用，如果設(shè)置，會(huì)報(bào)
MapperParsingException[Mapping definition for [fields] has unsupported parameters: [index_analyzer : ik_max_word]];
這里擴(kuò)展一下，在原來(lái)的版本中，index_analyzer負(fù)責(zé)建立索引時(shí)的分詞器定義，search_analyzer負(fù)責(zé)搜索時(shí)的分詞器定義。
索引期間查找解析器的完整順序是這樣的：
定義在字段映射中的index_analyzer
定義在字段映射中的analyzer
定義在文檔_analyzer字段中的解析器
type的默認(rèn)index_analyzer
type的默認(rèn)analyzer
索引設(shè)置中default_index對(duì)應(yīng)的解析器
索引設(shè)置中default對(duì)應(yīng)的解析器
節(jié)點(diǎn)上default_index對(duì)應(yīng)的解析器
節(jié)點(diǎn)上default對(duì)應(yīng)的解析器
standard解析器
而查詢(xún)期間的完整順序則是：
直接定義在查詢(xún)中的analyzer
定義在字段映射中的search_analyzer
定義在字段映射中的analyzer
type的默認(rèn)search_analyzer
type的默認(rèn)analyzer
索引設(shè)置中的default_search對(duì)應(yīng)的解析器
索引設(shè)置中的default對(duì)應(yīng)的解析器
節(jié)點(diǎn)上default_search對(duì)應(yīng)的解析器
節(jié)點(diǎn)上default對(duì)應(yīng)的解析器
standard解析器
現(xiàn)在新版刪除index_analyzer，具體功能由analyzer關(guān)鍵字承擔(dān)，analyzer關(guān)鍵字生效與index時(shí)和search時(shí)（除非search_analyzer已經(jīng)被顯性定義）。
3. _timestamp在2.0棄用
_timestamp官方建議自定義一個(gè)字段，自己賦值用來(lái)表示時(shí)間戳。
4. 嵌套字段排序時(shí)字段名稱(chēng)調(diào)整
對(duì)于如下的數(shù)據(jù)：
PUT /my_index/blogpost/2
{
"title": "Investment secrets",
"body": "What they don"t tell you ...",
"tags": [ "shares", "equities" ],
"comments": [
{
"name": "Mary Brown",
"comment": "Lies, lies, lies",
"age": 42,
"stars": 1,
"date": "2014-10-18"
},
{
"name": "John Smith",
"comment": "You"re making it up!",
"age": 28,
"stars": 2,
"date": "2014-10-16"
}
]
}
老版本中，對(duì)stars字段進(jìn)行排序時(shí)，直接可以
"sort" : [
{
"stars" : {
"order" : "desc",
"mode" : "min",
"nested_path" : "comments"
}
]
但在新版中，上述報(bào)文會(huì)報(bào)
No mapping found for [stars] in order to sort on
需要改成：
"sort" : [
{
"comments.stars" : {
"order" : "desc",
"mode" : "min"
}
]
5. _script腳本參數(shù)名變更
老版中，_script可以這樣定義
"sort" : [
{
"_script" : {
"script" : {
"inline" : "paramsMap.containsKey(doc["id"].value) ? params.paramsMap.get(doc["id"].value) : params.paramsMap.get("other")",
"lang" : "painless",
"params" : {
"paramsMap" : {
"1" : 1,
"2" : 1,
"3" : 2,
"other" : 3
}
}
},
"type" : "number",
"order" : "asc"
}
}
]
新版中，對(duì)于params的參數(shù)paramsMap必須用params.paramsMap
"sort" : [
{
"_script" : {
"script" : {
"inline" : "params.paramsMap.containsKey(doc["productCategory"].value) ? params.paramsMap.get(doc["productCategory"].value) : params.paramsMap.get("other")",
"lang" : "painless",
"params" : {
"paramsMap" : {
"901" : 1,
"902" : 1,
"701" : 2,
"other" : 3
}
}
},
"type" : "number",
"order" : "asc"
}
}
]
注意：es 5.2.0默認(rèn)禁用了動(dòng)態(tài)語(yǔ)言，所以lang為painless之外的語(yǔ)言，默認(rèn)情況下會(huì)報(bào)
ScriptException[scripts of type [inline], operation [update] and lang [groovy] are disabled];
需要在yml文件中添加配置（如groovy）：
script.engine.groovy.inline:true
script.engine.groovy.stored.search:true
script.engine.groovy.stored.aggs:true
6 .獲取特定字段返回
在舊版本中，獲取特定文檔特定字段返回，可以使用stored_fields：
{
"from" : 0,
"size" : 1,
"query" : {},
"stored_fields" : "timestamp",
"sort" : [
{
"timestamp" : {
"order" : "desc"
}
}
]
}
新版本中，引入了更為強(qiáng)大的_source過(guò)濾器
{
"from" : 0,
"size" : 1,
"query" : {},
"_source" : "timestamp",
"sort" : [
{
"timestamp" : {
"order" : "desc"
}
}
]
}
或者
{
"from" : 0,
"size" : 1,
"query" : {},
"_source" : {
"includes" : [ "timestamp" ],
"excludes" : [ "" ]
},
"sort" : [
{
"timestamp" : {
"order" : "desc"
}
}
]
}
java的api主要調(diào)用SearchRequestBuilder的setFetchSource方法
7. date字段的format定義
改版后，date字段最好再mapping時(shí)定義好format信息，以防止在請(qǐng)求前后因?yàn)楦袷睫D(zhuǎn)換問(wèn)題報(bào)錯(cuò)：
ElasticsearchParseException[failed to parse date field [Thu Jun 18 00:00:00 CST 2015] with format [strict_date_optional_time||epoch_millis]]; nested: IllegalArgumentException[Parse failure at index [0] of [Thu Jun 18 00:00:00 CST 2015]]; }
[strict_date_optional_time||epoch_millis]是es默認(rèn)的date字段解析格式
8. UncategorizedExecutionException
改版前，transport client發(fā)送數(shù)據(jù)之前將java代碼中的字段序列化成了json然后進(jìn)行傳輸和請(qǐng)求，而在5.x以后，es改用使用的內(nèi)部的transport protocol，這時(shí)候，如果定義一個(gè)比如bigDecimal類(lèi)型，es不支持bigDecimal，數(shù)據(jù)類(lèi)型不匹配會(huì)拋錯(cuò)誤。
UncategorizedExecutionException[Failed execution]; nested: IOException[can not write type [class java.math.BigDecimal]];
es支持的格式如下
static {
Map, Writer> writers = new HashMap<>();
writers.put(String.class, (o, v) -> {
o.writeByte((byte) 0);
o.writeString((String) v);
});
writers.put(Integer.class, (o, v) -> {
o.writeByte((byte) 1);
o.writeInt((Integer) v);
});
writers.put(Long.class, (o, v) -> {
o.writeByte((byte) 2);
o.writeLong((Long) v);
});
writers.put(Float.class, (o, v) -> {
o.writeByte((byte) 3);
o.writeFloat((float) v);
});
writers.put(Double.class, (o, v) -> {
o.writeByte((byte) 4);
o.writeDouble((double) v);
});
writers.put(Boolean.class, (o, v) -> {
o.writeByte((byte) 5);
o.writeBoolean((boolean) v);
});
writers.put(byte[].class, (o, v) -> {
o.writeByte((byte) 6);
final byte[] bytes = (byte[]) v;
o.writeVInt(bytes.length);
o.writeBytes(bytes);
});
writers.put(List.class, (o, v) -> {
o.writeByte((byte) 7);
final List list = (List) v;
o.writeVInt(list.size());
for (Object item : list) {
o.writeGenericValue(item);
}
});
writers.put(Object[].class, (o, v) -> {
o.writeByte((byte) 8);
final Object[] list = (Object[]) v;
o.writeVInt(list.length);
for (Object item : list) {
o.writeGenericValue(item);
}
});
writers.put(Map.class, (o, v) -> {
if (v instanceof LinkedHashMap) {
o.writeByte((byte) 9);
} else {
o.writeByte((byte) 10);
}
@SuppressWarnings("unchecked")
final Map map = (Map) v;
o.writeVInt(map.size());
for (Map.Entry entry : map.entrySet()) {
o.writeString(entry.getKey());
o.writeGenericValue(entry.getValue());
}
});
writers.put(Byte.class, (o, v) -> {
o.writeByte((byte) 11);
o.writeByte((Byte) v);
});
writers.put(Date.class, (o, v) -> {
o.writeByte((byte) 12);
o.writeLong(((Date) v).getTime());
});
writers.put(ReadableInstant.class, (o, v) -> {
o.writeByte((byte) 13);
final ReadableInstant instant = (ReadableInstant) v;
o.writeString(instant.getZone().getID());
o.writeLong(instant.getMillis());
});
writers.put(BytesReference.class, (o, v) -> {
o.writeByte((byte) 14);
o.writeBytesReference((BytesReference) v);
});
writers.put(Text.class, (o, v) -> {
o.writeByte((byte) 15);
o.writeText((Text) v);
});
writers.put(Short.class, (o, v) -> {
o.writeByte((byte) 16);
o.writeShort((Short) v);
});
writers.put(int[].class, (o, v) -> {
o.writeByte((byte) 17);
o.writeIntArray((int[]) v);
});
writers.put(long[].class, (o, v) -> {
o.writeByte((byte) 18);
o.writeLongArray((long[]) v);
});
writers.put(float[].class, (o, v) -> {
o.writeByte((byte) 19);
o.writeFloatArray((float[]) v);
});
writers.put(double[].class, (o, v) -> {
o.writeByte((byte) 20);
o.writeDoubleArray((double[]) v);
});
writers.put(BytesRef.class, (o, v) -> {
o.writeByte((byte) 21);
o.writeBytesRef((BytesRef) v);
});
writers.put(GeoPoint.class, (o, v) -> {
o.writeByte((byte) 22);
o.writeGeoPoint((GeoPoint) v);
});
WRITERS = Collections.unmodifiableMap(writers);
}
