cluster
配置中同时包含在线部分 (ha3) 配置和离线 (build_service) 两部分配置;离线配置中包含了对builder 和merger 的各种配置。
customized_merge_config
customized_merge_config:用户自定义配置。用户可以对不同场景定制不同的merge策略。比如增量阶段使用和缺省策略不同的merge策略;可以配置多个,多个merge策略会同时生效,相同时间如果触发了多个merge策略会按照配置中的顺序先后触发merge plan。 用户除了指定merge_config,还可以配置下面两个参数:
period: 索引的merge间隔时间,单位为s。或配置在某个固定时刻merge,比如在下午13pm触发merge,格式是"period": "daytime=13:00"
need_wait_alter_field:如果当前有动态加字段流程,是否要阻塞merge,默认是true。如果配成false,当前merge 策略将自动换成align_version merge策略生效。
merge_parallel_num: 每个partition参与merge的并发进程数量。
由于新增动态加字段功能。禁止用户自定义的merge策略命名为alter_field。
customized_merge_config默认值如下:
{
"full":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"optimize",
"merge_strategy_param":"after-merge-max-segment-count=20",
"merge_thread_count":4
}
},
"large_segment_reclaim":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=20480",
"output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
"strategy_conditions":"priority-feature=delete-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=8"
},
"merge_thread_count":4
},
"period":"period=1800"
},
"segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=12288",
"output_limits":"max-merged-segment-size=13312;max-total-merged-size=15360",
"strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=10"
},
"merge_thread_count":4
},
"period":"period=900"
},
"large_segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=12288",
"output_limits":"max-merged-segment-size=15360;max-total-merged-size=46080",
"strategy_conditions":"priority-feature=valid-doc-count#desc;conflict-segment-count=2;conflict-delete-percent=10"
},
"merge_thread_count":4
},
"period":"period=3600"
},
"small_segment_merge":{
"merge_config":{
"keep_version_count":40,
"merge_strategy":"priority_queue",
"merge_strategy_params":{
"input_limits":"max-segment-size=1536",
"output_limits":"max-merged-segment-size=1536;max-total-merged-size=3072",
"strategy_conditions":"priority-feature=valid-doc-count#asc;conflict-segment-count=2;conflict-delete-percent=20"
},
"merge_thread_count":4
}
}
}
segment_customize_metrics_updater
●segment_customize_metrics_updater:用于生成各个segment的segment metric,可由用户自定义生成。indexlib内置存在lifecycle updater,配置该updater后,每个segment会生成配置字段的min,max值,并根据定义的lifecycle_param配置的打标方式,为每个sgement打标(merge阶段,build阶段按照默认tag打标)。打标可用于load策略和time_series_merge策略。默认值:[]。