阿里云Elasticsearch(简称ES)新增8.13版本,支持硬件加速并提供开源向量检索等场景的新特性和优化点,较上一个版本有明显的性能优势。本文通过实际测试数据,展示ES 8.13版本和8.9版本的性能对比。
压测配置
ES集群配置
ES实例版本:8.13版本、8.9版本
数据节点规格:16核32 GiB
数据节点存储类型:ESSD云盘
数据节点存储性能级别:PL1
数据单节点存储空间:500 GiB
数据节点数量:1个
说明本压测场景因需要控制变量并突出单节点性能表现,特提供单节点实例的性能对比数据。阿里云ES暂不支持创建单节点实例,集群性能表现和节点个数正相关。
测试集索引的主分片数为1,副本分片数为0。
测试集索引的配置(mapping和setting)如下:
{ "vectors_so": { "aliases": {}, "mappings": { "properties": { "acceptedAnswerId": { "type": "keyword" }, "body": { "type": "text" }, "creationDate": { "type": "date" }, "questionId": { "type": "keyword" }, "tags": { "type": "keyword" }, "title": { "type": "text" }, "titleVector": { "type": "dense_vector", "dims": 768, "index": true, "similarity": "dot_product" }, "type": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "user": { "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }, "userId": { "type": "keyword" } } }, "settings": { "index": { "number_of_shards": "1", "number_of_replicas": "0" } } } }
测试数据集
本测试使用StackOverflow 200w 768维的向量数据。获取测集,请参见Elastic_vector。
测试集的数据示例:
{"user":"209706","tags":["java","svn","git"],"questionId":"3329041","creationDate":"2010-07-25T11:43:57.960","title":"Best practice to store .jar files in VCS (SVN, Git, ...)","titleVector":[-0.022568320855498314,0.016966139897704124,-0.006804537493735552,-0.025535643100738525,-0.022463209927082062,-0.0544169619679451,0.007135968655347824,0.02151673100888729,0.035291559994220734,0.02094927430152893,0.010055724531412125,0.0018165978835895658,-0.021382154896855354,0.02836012840270996,-0.026590753346681595,-0.039688147604465485,0.014944477006793022,0.0037696415092796087,0.024937929585576057,-0.013032712042331696,0.01987944357097149,-0.010971742682158947,0.028384437784552574,0.020739110186696053,-0.0012843584408983588,-0.019710227847099304,-0.019073335453867912,0.0036203600466251373,0.0190863236784935,0.0006095399148762226,0.007584390230476856,-0.010574880987405777,0.03173621743917465,0.05325346812605858,-0.0009263881365768611,0.037904348224401474,0.00554504431784153,0.013168608769774437,-0.0002838771906681359,0.04091921076178551,0.046684663742780685,-0.047092169523239136,-0.03201943635940552,0.01748107746243477,0.007151423953473568,-0.029639896005392075,0.009412004612386227,-0.08248382061719894,0.06483617424964905,-0.039679888635873795,-0.008396460674703121,-0.04700171947479248,-0.0024395363871008158,0.05125616118311882,0.059288349002599716,0.031760018318891525,-0.010636314749717712,0.053758665919303894,-0.038049813359975815,0.012179340235888958,0.01970532163977623,0.04069080948829651,0.07527115195989609,0.01719692163169384,0.05825617164373398,0.0161513090133667,-0.08610313385725021,-0.007587376981973648,-0.00686598801985383,-0.041036009788513184,0.04273809865117073,-0.01618768461048603,-0.00863699708133936,0.011954491958022118,-0.017342722043395042,-0.006477853283286095,-0.002123388461768627,-0.011649735271930695,0.05267751216888428,-0.007842873223125935,-0.011504382826387882,-0.007063502911478281,-0.034192878752946854,0.012672032229602337,0.03551720455288887,-0.09547989070415497,0.0006479862495325506,-0.0036617410369217396,-0.007667073048651218,0.0168107058852911,-0.0935259684920311,-0.06169058009982109,-0.0056406185030937195,-0.021994099020957947,0.05909443274140358,-0.0047522070817649364,-0.03239529952406883,-0.03731035441160202,0.026133738458156586,-0.05005648359656334,-0.08483706414699554,0.03291381150484085,-0.02870767191052437,-0.02267201989889145,-0.05542087182402611,0.06586036086082458,0.011262048967182636,0.02898000366985798,0.062353916466236115,-0.027525940909981728,-0.04769891873002052,-0.005141156259924173,-0.0504637137055397,0.005994201172143221,0.05173376202583313,0.0020877523347735405,-0.02355918660759926,-0.0025070980191230774,0.04757542163133621,0.014315967448055744,-0.025437284260988235,0.03984103724360466,0.005889470223337412,0.016815686598420143,0.01058722659945488,-0.00891159474849701,-0.013322465121746063,0.007238275371491909,0.05897805839776993,-0.020219894126057625,-0.009327727369964123,0.014212900772690773,-0.009871017187833786,-0.02634282223880291,0.008907344192266464,-0.02646763063967228,0.05261416733264923,0.005156352184712887,-0.013447979465126991,0.010898955166339874,0.03229687735438347,0.028374144807457924,0.0005994946695864201,0.029077177867293358,0.015455844812095165,-0.021326756104826927,-0.03915349021553993,0.02826860174536705,0.015046368353068829,-0.014139593578875065,0.03172646835446358,-0.05647587031126022,0.07042743265628815,0.009905800223350525,0.01943121664226055,0.02103494480252266,-0.03282477706670761,-0.01774805225431919,0.04843657836318016,0.025349726900458336,-0.06711459904909134,-0.024558287113904953,0.0018043402815237641,-0.044150419533252716,-0.03127370402216911,-0.01684996299445629,-0.07407478988170624,0.06862112134695053,0.026920832693576813,0.0457388237118721,-0.026039492338895798,0.033038828521966934,-0.02689289301633835,-0.044180139899253845,0.018091989681124687,-0.007654160261154175,0.02214517630636692,0.017185229808092117,0.005919412709772587,0.0355246476829052,0.04592865705490112,-0.0035185874439775944,0.01189420185983181,0.07549969851970673,0.016762737184762955,-0.007039632182568312,-0.059498634189367294,-0.004957953002303839,0.033884577453136444,0.016413042321801186,-0.04906884953379631,-0.01371896080672741,-0.02687361277639866,0.007799462415277958,-0.010260170325636864,-0.02136010117828846,0.024969611316919327,0.01621391996741295,-0.02629096433520317,-0.004342242144048214,-0.021826475858688354,0.012518396601080894,-0.04031432047486305,0.011757384985685349,-0.02253311313688755,0.007648595608770847,-0.038293641060590744,0.012258331291377544,-0.05203482508659363,-0.10458920150995255,0.04995381087064743,-0.011123030446469784,0.05595514178276062,-0.03153397515416145,0.0078066131100058556,-0.004604082554578781,0.009236234240233898,0.041576337069272995,0.024902883917093277,0.043439097702503204,0.023275984451174736,-0.02935713157057762,0.03641841188073158,0.00435029249638319,0.06283614039421082,0.023789631202816963,-0.00968264788389206,-0.021677223965525627,-0.010009178891777992,-0.008318934589624405,0.010726073756814003,-0.03316894918680191,-0.0173848494887352,-0.011089922860264778,0.00316459103487432,-0.024657653644680977,0.047302693128585815,0.03668683022260666,0.009666360914707184,0.046169329434633255,0.06253506243228912,-0.05089620500802994,0.01377822645008564,0.013364230282604694,-0.056329358369112015,0.04475446045398712,-0.011291306465864182,-0.008311367593705654,-0.02495403215289116,-0.015229254961013794,0.0016104120295494795,0.019953439012169838,-0.020877743139863014,0.027237361297011375,-0.00449178833514452,0.03784524276852608,-0.047541309148073196,0.05861435830593109,0.012838158756494522,-0.008239383809268475,0.040583159774541855,-0.025667961686849594,0.00260987994261086,-0.05442212149500847,0.02081277221441269,-0.0049480474554002285,-0.047668345272541046,-0.027836641296744347,0.01943138614296913,-0.017467467114329338,0.021935874596238136,-0.08722472190856934,-0.03151370957493782,-0.02208506129682064,-0.0014428343856707215,0.009389142505824566,0.058151401579380035,0.03156457468867302,-0.03103310987353325,-0.01860646903514862,0.02192562259733677,0.040813103318214417,-0.0404493473470211,0.004206551238894463,0.030254555866122246,0.022282332181930542,-0.019713835790753365,-0.028253355994820595,0.007667528465390205,0.03367909789085388,-0.0039051698986440897,0.01755901239812374,0.04164242371916771,-0.012693769298493862,-0.02684374712407589,-0.008283626288175583,0.030895130708813667,0.055572375655174255,-0.0396922267973423,-0.028674783185124397,0.03953632339835167,-0.020687779411673546,-0.01933380402624607,0.009744361974298954,0.01996387541294098,-0.05533822253346443,0.01597154140472412,0.02863628976047039,-0.04273497313261032,-0.08222821354866028,0.0026089330203831196,-0.04134223610162735,0.006577998399734497,0.019175158813595772,-0.04154052957892418,-0.017960628494620323,-0.00412540091201663,0.006837442517280579,-0.018161650747060776,0.028152810409665108,0.02926110103726387,-0.0010820962488651276,0.024585502222180367,-0.026463216170668602,-0.009788371622562408,-0.00045153251267038286,0.0030222004279494286,-0.05157322809100151,0.034691549837589264,0.05631604418158531,-0.05051704868674278,-0.0021566429641097784,0.013112308457493782,-0.05082176998257637,0.06757031381130219,-0.014824115671217442,-0.0037184804677963257,-0.049066897481679916,0.033723656088113785,-0.03727893531322479,-0.02892637439072132,0.02092050015926361,-0.04100112244486809,-0.016329817473888397,0.007387165445834398,0.0011634480906650424,-0.048837810754776,0.05008264258503914,0.024229703471064568,0.07980538159608841,0.03026900440454483,-0.022093679755926132,0.0018667720723897219,-0.0007226031739264727,0.03274497017264366,0.007752065546810627,0.01612628810107708,-0.0015951483510434628,-0.011697298847138882,0.04933670163154602,-0.01608007401227951,0.04265035688877106,0.10154743492603302,-0.03620481118559837,0.0015862829750403762,-0.08087439090013504,0.0869990661740303,0.028814632445573807,-0.031040135771036148,0.05266718938946724,-0.00029543187702074647,-0.027241889387369156,0.010084390640258789,0.010500782169401646,0.021171029657125473,0.019291449338197708,-0.007761659100651741,0.01761694625020027,0.08652865141630173,0.02458319440484047,0.05171985924243927,-0.030445005744695663,0.00048042554408311844,-0.014511095359921455,-0.024271266534924507,0.027662770822644234,-0.006052704527974129,-0.0017832177691161633,0.051913876086473465,0.023082256317138672,0.02954365685582161,-0.021671734750270844,-0.030222980305552483,0.03143236041069031,0.01236815843731165,-0.03840154781937599,-0.04293489083647728,-0.05501846596598625,0.03423580899834633,0.004786348901689053,0.014820712618529797,-0.05694545432925224,-0.01438840851187706,0.00041487423004582524,-0.006501889321953058,-0.0029918348882347345,0.056806378066539764,-0.06631509214639664,0.0037918214220553637,-0.04391039162874222,-0.03205549716949463,-0.006208882667124271,0.04312444478273392,0.01851271279156208,-0.028821200132369995,-0.007434142753481865,-0.01707986369729042,0.07009696960449219,-0.03032786212861538,0.033938825130462646,-0.027377497404813766,0.03591281548142433,0.012881964445114136,-0.0490662083029747,-0.01800457015633583,0.0029962421394884586,-0.02983405441045761,-0.027971865609288216,-0.005753741133958101,-0.0042424858547747135,0.020415622740983963,0.052845895290374756,-0.005748809780925512,-0.00391183327883482,0.06060341000556946,0.010863612405955791,-0.09753439575433731,-0.031696777790784836,0.014809386804699898,-0.023946058005094528,-0.032845668494701385,0.005366857163608074,-0.1121651753783226,-0.010608629323542118,-0.03286733850836754,-0.01518184319138527,-0.0008939955732785165,0.03380736708641052,-0.005993296392261982,-0.0012251981534063816,-0.0043722158297896385,0.05782324820756912,0.016369502991437912,0.011886055581271648,-0.06807675957679749,-0.028851406648755074,-0.05816076695919037,-0.022793371230363846,-0.011525285430252552,0.03684655949473381,0.004060117993503809,-0.030044345185160637,0.01217928808182478,0.011539846658706665,-0.020583922043442726,0.030714813619852066,0.03655166178941727,-0.03666571155190468,0.023502880707383156,-0.03369182348251343,-0.040057990700006485,-0.000004013560555904405,-0.01480079535394907,-0.012144369073212147,-0.04563474655151367,-0.045321349054574966,-0.02025204710662365,0.009174671024084091,-0.0889308899641037,-0.022527499124407768,0.019474899396300316,0.044222041964530945,-0.003216656157746911,0.057539280503988266,0.0007779379957355559,0.034109603613615036,0.06279256939888,-0.005324139725416899,-0.023193074390292168,0.03208332881331444,0.03672686219215393,0.013614351861178875,-0.026610372588038445,0.03582369536161423,0.05533266440033913,-0.028707629069685936,0.026829993352293968,-0.011561914347112179,0.052959952503442764,0.05918995290994644,0.008763168007135391,0.05365585908293724,-0.01041406486183405,-0.02573433518409729,-0.07467862218618393,0.014847896993160248,-0.04952200502157211,0.029755793511867523,-0.03199444338679314,-0.015181966125965118,-0.0003408882475923747,-0.008800947107374668,0.00020574385416693985,-0.009139910340309143,-0.030881227925419807,0.04660243168473244,0.03465842455625534,-0.008915398269891739,-0.007468358147889376,-0.051615480333566666,-0.00018453389930073172,-0.00368774333037436,0.01375708170235157,0.03534002602100372,-0.03734860196709633,-0.05910753831267357,-0.0016254845540970564,-0.011543968692421913,-0.04119078069925308,-0.027235684916377068,0.005582587327808142,-0.0068008555099368095,0.0032956067007035017,-0.03828691318631172,0.011587239801883698,0.04457457736134529,-0.05859078839421272,0.04892950877547264,0.004259657580405474,-0.045298147946596146,-0.0477784126996994,-0.031117357313632965,-0.016119591891765594,0.07209751009941101,-0.0049171457067132,0.003412538906559348,0.020771227777004242,0.015838487073779106,0.039533376693725586,0.05333689600229263,0.049018338322639465,0.026931170374155045,-0.08213973045349121,-0.006878034211695194,-0.050083715468645096,0.0067598395980894566,-0.019466320052742958,0.04844585433602333,0.015705952420830727,0.0250754002481699,0.00687074102461338,-0.01568557694554329,-0.014323839917778969,-0.04254802688956261,-0.014691188000142574,0.010361634194850922,0.020978564396500587,0.009622364304959774,0.030941935256123543,-0.07706917077302933,0.04246467351913452,0.0005529716145247221,-0.04142564535140991,0.031159354373812675,0.007720559369772673,-0.003725485410541296,-0.0603741891682148,-0.004892861936241388,0.030623359605669975,-0.0457223542034626,-0.01531533058732748,0.008588816970586777,0.03602635860443115,0.028087006881833076,-0.05782897397875786,-0.0030375535134226084,0.0132557712495327,0.017065169289708138,-0.054937299340963364,0.017705602571368217,0.035395119339227676,-0.07757074385881424,0.05859997496008873,0.008987262845039368,-0.03041536919772625,0.02561228536069393,0.0183942299336195,-0.06502795219421387,0.011738051660358906,-0.006459866650402546,0.020579950883984566,0.0075156670063734055,0.008700506761670113,0.027386141940951347,-0.06798586994409561,-0.04130703583359718,0.026013823226094246,-0.010291499085724354,0.007729311473667622,-0.001674408558756113,0.05831485986709595,-0.0201500803232193,0.03411034867167473,0.007884996943175793,-0.013345245271921158,-0.020150301977992058,-0.032769039273262024,0.011256161145865917,0.03748656436800957,-0.10405875742435455,0.004104201681911945,0.037583958357572556,-0.02187950536608696,-0.04488478973507881,0.010267623700201511,-0.002289202529937029,0.014976054430007935,-0.03869527950882912,0.048902105540037155,-0.029617805033922195,0.11036717146635056,-0.0781548023223877,0.0171914491802454,0.004673470743000507,-0.057904984802007675,0.012959512881934643,-0.10143953561782837,0.022075794637203217,0.017105476930737495,0.0041616992093622684,-0.020934253931045532,-0.04317566007375717,-0.028254862874746323,-0.004780441056936979,0.005597088485956192,0.005253334064036608,0.03270745277404785,-0.06404801458120346,0.06373090296983719,0.010657697916030884,0.09668827056884766,0.08736786991357803,-0.051720310002565384,-0.005920285824686289,0.01983417384326458,0.05913180857896805,0.007240667473524809,0.008302704431116581,0.03503921627998352,-0.08105749636888504,-0.03457126393914223,-0.03002745658159256,0.03757239133119583,-0.010914829559624195,0.04185640066862106,0.027861781418323517,-0.02038309909403324,-0.0033769423607736826,-0.08202539384365082,-0.02990865334868431,-0.018030330538749695,0.021875880658626556,0.016777411103248596,-0.008198398165404797,0.023779047653079033,-0.0569751113653183,-0.007992339320480824,0.005154220387339592,0.059560660272836685,-0.03551492840051651,0.028958972543478012,0.056636516004800797,0.06335761398077011,0.03422478958964348,-0.03659925237298012,0.07427392154932022,0.012194718234241009,0.012754388153553009,-0.00004210855331621133,-0.009875668212771416,0.039737749844789505,0.022092211991548538,-0.05501311644911766,-0.0047541591338813305,0.036219820380210876,0.0073831574991345406,-0.01100097969174385,-0.04413069039583206,-0.029255781322717667,-0.012701094150543213,0.03444666415452957,-0.00881791952997446,0.029486406594514847,0.02177346497774124,-0.06924846768379211,-0.04663984104990959,-0.03368207439780235,-0.001259694225154817,0.005179648753255606,0.10770683735609055,0.039790477603673935,0.00048222101759165525,-0.030503906309604645,0.009989997372031212,0.023120656609535217,0.040883682668209076,0.09497974067926407,0.03620487451553345,0.01803920976817608,-0.09199116379022598,0.00561245484277606,-0.03510742262005806,-0.06692400574684143,-0.01690664514899254,-0.0493900366127491,-0.0007601031684316695,0.04529772698879242,0.021885912865400314,-0.014822583645582199,0.013379829935729504,0.0884644091129303,-0.06919929385185242,0.037087373435497284,0.02372562140226364,-0.025841565802693367,0.019196046516299248,-0.008552762679755688,0.023789865896105766,-0.0401601605117321,0.001249208697117865,0.012803432531654835,0.03387373685836792,-0.007558054756373167,-0.02264394611120224,-0.01229764986783266,-0.04061853885650635,0.029595181345939636,-0.004438564646989107,-0.06074979901313782,-0.01740187220275402,0.01198149099946022,0.00828469917178154,0.05918612703680992,0.045886486768722534,0.1451934576034546,-0.008648079819977283,0.018697980791330338,-0.005558650940656662,-0.03625230863690376,-0.018993757665157318,0.007326720282435417,-0.035033319145441055,0.02445784956216812,-0.038400325924158096,-0.005789259448647499,-0.05283428728580475,-0.00716761127114296,-0.034205518662929535],"acceptedAnswerId":"3329272","type":"question","body":"I know, in the time of Maven it is not recommended to store libraries in VCS, but sometimes it makes sense, though. My question is how to best store them - compressed or uncompressed? Uncompressed they are larger, but if they are replaced a couple of times with newer ones, then maybe the stored difference between two uncompressed .jar files might be much smaller than the difference of compressed ones. Did someone make some tests? "}
测试集的查询示例:
GET vectors_so/_search?size=100 { "knn": [ { "field": "titleVector", "k": 100, "num_candidates": 100, "query_vector": [ -0.035655074, 0.02915079, -0.009953431, 0.016862303, 0.007259798, 0.01005873, 0.0031890676, 0.013714448, -0.06062733, -0.025562758, -0.03002447, -0.026550097, -0.02107223, -0.020553242, 0.002474099, -0.021948095, 0.03757188, -0.009190732, -0.00780341, -0.017052658, -0.0065468056, 0.030787662, 0.02590319, -0.04636106, -0.036633756, -0.05811224, 0.027320085, 0.051317003, -0.002910953, 0.02423434, -0.033778537, -0.04715699, 0.055929773, 0.052612506, -0.00042414063, -0.032236893, -0.019210164, 0.015807526, -0.005356853, 0.10626104, -0.0013599976, -0.019840837, 0.033086028, 0.030549651, -0.021256946, 0.060329687, 0.0234322, -0.062026054, 0.010168108, 0.01975769, 0.022075966, -0.027184976, 0.053044077, 0.019912066, -0.058743283, -0.02237231, 0.010880714, 0.10801274, 0.005823678, 0.1030897, -0.0020786347, -0.078742035, 0.003924668, 0.038136523, -0.062892705, -0.014300704, 0.024043757, 0.057133753, -0.021905988, 0.013701377, -0.019309642, 0.002611628, -0.020216659, 0.052892663, -0.04801682, -0.0428704, 0.025949506, -0.03272141, 0.032309573, 0.039714467, 0.0324138, -0.082197346, 0.023043571, 0.022732243, -0.062969305, 0.020080663, -0.047752194, 0.037787184, -0.047711946, -0.029745966, -0.033333372, 0.05168253, 0.008533111, -0.0011690593, 0.015184578, 0.0044088042, -0.021822112, 0.017584646, 0.022316365, -0.021504264, 0.04036433, 0.04423992, -0.06553741, 0.019488886, -0.009774748, 0.053420402, -0.06104895, 0.0079910215, 0.021223309, -0.031525925, 0.069041915, -0.01666679, -0.028979762, -0.024939895, 0.0839611, -0.018203193, 0.02863906, -0.019337371, -0.0058257356, -0.022833783, 0.06203988, -0.03219217, 0.038293894, -0.036684208, -0.01997551, 0.07674495, -0.03523276, 0.00017863407, -0.0038612934, -0.052855596, -0.03689549, 0.0026357404, 0.020991854, 0.027612716, 0.030929066, 0.0015022622, 0.009322592, 0.05091024, -0.034481723, -0.019720895, -0.052288715, 0.078428976, 0.010158794, -0.029476989, -0.02264689, -0.056788817, -0.00089345226, -0.048745316, -0.009679014, 0.014717792, -0.025908494, 0.05123777, -0.10946392, 0.022627808, 0.0629637, -0.065279864, -0.05776511, 0.008419831, 0.05475125, 0.0014037908, 0.049576245, 0.022818686, -0.010850337, 0.020566903, -0.0059649325, -0.026529962, -0.07000316, -0.00059256784, 0.038155116, -0.0036679604, 0.043199755, 0.020610584, 0.016138757, -0.028218422, 0.0043825572, -0.042442035, -0.032531306, 0.029496193, 0.06442059, -0.059175115, -0.055385217, -0.06622881, -0.025149196, 0.009978219, 0.0390013, -0.003974938, 0.076899596, -0.029714555, -0.041060273, -0.038494043, 0.009285519, -0.020366281, -0.00552419, 0.010278463, -0.035915997, -0.026300233, -0.0041749636, 0.025668716, -0.0107331835, 0.0035612462, 0.0106827505, -0.04099344, -0.033057723, -0.011039526, -0.011763911, -0.038125996, 0.019757867, 0.0012890182, -0.025424754, -0.03283122, 0.023626361, -0.01513745, 0.0099618845, -0.0073508327, 0.03280086, -0.0031597156, 0.0064643617, -0.03522152, 0.0407715, 0.019031955, -0.009723273, -0.07353718, 0.079735145, 0.032803833, 0.06543241, -0.063597605, 0.042152885, -0.027418429, -0.037108187, 0.009744578, 0.013476579, 0.017894737, 0.0070203347, 0.007983844, -0.039294142, -0.023337316, 0.004379248, -0.004015899, 0.024705276, -0.0027054744, 0.0079105655, 0.0834098, 0.015991285, -0.0018527613, 0.00031564743, 0.005881187, -0.019030582, -0.04534438, -0.037096143, 0.041083086, 0.02233062, 0.02402726, -0.031039586, -0.007287719, -0.03873248, -0.028300174, -0.0015663864, -0.013528352, -0.045447487, -0.008431108, -0.004370459, 0.023656571, 0.03166844, -0.018949592, -0.045675647, 0.032122392, -0.009707521, -0.030808836, 0.023984192, -0.021232292, 0.042215433, 0.016728519, -0.0072546597, 0.0025122224, -0.037439644, -0.05043408, 0.017068667, -0.08743435, 0.0115130115, 0.016343782, 0.001404199, 0.09553798, 0.0088382745, -0.047452994, -0.02695906, -0.034419507, -0.0018368709, 0.07157888, -0.023965519, 0.09751397, 0.027433643, 0.026339473, 0.010309662, 0.013991426, 0.029424759, -0.045021586, 0.007889588, -0.037715882, -0.0017137943, 0.023238556, -0.0021091902, 0.05250276, -0.10714014, 0.0044707586, 0.023741925, -0.0025698182, 0.018200701, 0.01986693, -0.026571605, -0.029355634, -0.034468956, -0.053618316, 0.07387493, 0.0041634794, -0.017754495, 0.018576652, -0.009797161, 0.019154865, 0.02988726, -0.0071307644, 0.025345579, -0.015894543, -0.02013885, 0.0020117126, 0.0012763311, 0.04748289, 0.0027929547, 0.025585776, -0.013460711, -0.028746776, 0.00056184386, -0.110901006, -0.008552718, 0.019708958, 0.067802645, -0.0051138448, -0.023446703, 0.009334193, -0.009444671, -0.0054092463, 0.018911345, 0.0088326875, -0.0006177318, 0.0062277066, -0.036681827, 0.11252966, -0.028750796, -0.02347743, -0.062866315, -0.04093959, 0.03886839, 0.070377536, -0.0032989169, -0.010539748, 0.0013698963, 0.009377954, 0.05339145, 0.010542219, 0.03913126, 0.021044984, -0.006046042, 0.010886611, -0.0070799403, 0.025334986, -0.02432221, -0.049644295, -0.014856045, -0.02535605, 0.04669711, 0.035462357, -0.012174314, -0.019514626, 0.00023600005, 0.016223324, 0.024001217, -0.047753595, -0.017870769, -0.038906563, -0.05119256, -0.033654578, -0.03526986, -0.058749612, -0.04393085, -0.026779767, -0.0371748, 0.030165503, 0.0023738584, 0.014556493, -0.044893175, -0.0626295, 0.007774993, -0.015520785, 0.027643904, -0.046478096, 0.01571064, -0.003552223, 0.056684636, -0.007884554, 0.025027188, 0.022557288, -0.021369036, -0.046165556, 0.068396784, 0.010489947, -0.02151329, 0.01621435, -0.0016323613, 0.0018603137, -0.01829692, 0.05440541, -0.0326959, -0.019722348, -0.028954716, -0.006595495, -0.037501175, -0.015340997, 0.021535035, 0.04605574, 0.030197969, -0.018932486, 0.035491325, 0.043004114, 0.014098184, 0.022361267, 0.009802298, -0.026968166, -0.0028520182, 0.013460146, 0.042134482, -0.06632637, -0.0136404475, 0.01654931, -0.032440547, 0.019699087, -0.00047389616, -0.059039444, 0.004159158, 0.04429101, 0.003872586, 0.0031527174, 0.025607014, -0.00828111, -0.15848775, 0.04422316, 0.0015401487, 0.06380972, 0.013214884, -0.008954356, -0.021055836, -0.024166146, 0.055964295, 0.0069205984, 0.037837394, 0.048384067, 0.029090129, 0.014646432, 0.019111915, -0.02021666, -0.0032435965, -0.03501175, 0.049577728, -0.044072326, 0.016265117, -0.009797879, 0.026676828, -0.030644778, 0.016425433, 0.009930545, -0.019714642, 0.0034964408, 0.07091106, -0.0057789953, 0.026004402, 0.013516524, -0.011916894, -0.036831535, 0.04016465, -0.12716857, -0.018487286, 0.033379532, 0.021601843, -0.04384704, -0.035796423, 0.11792618, -0.0029324908, -0.010638778, -0.069842905, 0.0038305714, 0.007833412, 0.0134473685, -0.0023471024, 0.007095982, -0.035789516, -0.06019452, 0.031073935, -0.00717761, 0.043447845, -0.004873193, -0.009990269, 0.07387206, -0.044872478, -0.009674619, 0.0077314205, -0.0031587896, 0.04997939, -0.016994108, -0.0016591386, 0.01752003, -0.021097694, 0.010463709, -0.009139511, 0.016735358, 0.060147632, 0.030006263, 0.024423616, -0.020157745, -0.044746242, 0.047522143, -0.022302236, 0.007747004, -0.0021971597, 0.012966715, 0.037624627, -0.024573836, 0.015103555, 0.057455253, 0.019378247, -0.010667713, -0.006894515, -0.014436144, -0.04680005, 0.0036504995, 0.03705293, 0.017737025, 0.014738512, 0.0008316689, -0.0033023998, -0.033323683, 0.017181942, 0.023572782, -0.0045203282, 0.0041480022, -0.036532737, -0.013946603, 0.021031283, -0.03473241, 0.03156023, 0.022317477, -0.01972129, 0.00888653, 0.001471229, 0.0098431455, 0.068481304, -0.0039132666, -0.055903986, 0.0023257122, 0.035733093, -0.0073416727, -0.007972695, 0.036954876, -0.013041575, -0.053169243, 0.07956131, 0.0074083777, -0.019228488, -0.0009132683, -0.026916483, 0.013592049, -0.031439934, -0.04238666, -0.0025418468, 0.063602835, 0.025705885, -0.07138311, -0.012239707, -0.041142438, 0.0009787465, -0.0311421, 0.028999269, 0.004927006, 0.041701373, 0.0073983716, -0.023993015, 0.02406577, 0.0016199419, -0.031180613, 0.029806338, 0.026011748, 0.016487677, -0.04594786, 0.0067858696, 0.08104468, -0.048297077, -0.04585522, -0.0030510155, -0.01578794, 0.055499114, -0.015017301, 0.013355118, 0.031204464, 0.05623067, -0.040792026, -0.018829469, 0.035098184, 0.023707109, -0.0010423487, -0.043535836, -0.008083583, 0.06760681, -0.0094824685, 0.020736562, 0.0025044428, -0.025875352, -0.045864087, -0.0601535, 0.03816534, 0.03082878, 0.0024594136, -0.030086778, -0.048101217, -0.023348495, -0.028618135, 0.013447613, -0.0010045123, -0.044410218, -0.050343066, 0.068491854, -0.0065228115, 0.028555244, 0.03996873, 0.08503508, 0.013869825, -0.09197791, 0.06859694, 0.011526136, -0.03449693, -0.029026777, -0.0067315637, 0.028936552, 0.034188583, 0.013723241, 0.06560098, -0.028441153, -0.023732632, -0.02886431, -0.025746964, 0.04814513, 0.052562293, -0.026192842, -0.025821669, -0.04349, -0.009074469, 0.03630079, -0.05020939, 0.010808986, 0.008669304, 0.058075447, -0.03749445, -0.025824975, 0.067853235, 0.0070648985, 0.10195903, -0.01952308, 0.0073264926, -0.013574462, -0.0074129277, 0.03791226, -0.02721496, 0.013223816, 0.026358698, -0.023497494, -0.018641194, 0.056463577, -0.074627034, -0.0016907289, -0.08673571, -0.004169064, -0.010106506, -0.0017142773, 0.042165753, -0.03959576, -0.050706442, -0.023953822, -0.05736858, -0.03070926, 0.004805277, -0.025231283, 0.005794143, -0.0068706116, 0.014444927, -0.012345491, -0.065554306, -0.02410455, 0.0029828304, 0.0059827617, -0.005617493, 0.042651, 0.055898033, -0.03698243, 0.017037764, 0.036744248, -0.06586883, 0.026106438, 0.0401622, -0.033117056, 0.047303215, 0.000647312, 0.06758076, -0.007591931, 0.021174923, -0.0077855806, 0.0038723678, -0.013889884, 0.011402813, 0.00044741257, -0.026866538, -0.026398199, 0.004480899, -0.03714357, -0.038313277, -0.010613461, 0.010628024, -0.012937155, 0.02901083, 0.01276943, 0.039021954, 0.046415202, -0.029221786, 0.04299161, -0.025286043, 0.019674785, -0.03450036, -0.028478773, 0.023912875, 0.01695282, -0.018761951, 0.017033143, -0.0048811752, 0.05159028, -0.015350196, -0.033129863, 0.04386473, -0.010108468, 0.0010479704, 0.028348226, 0.031211125, 0.002335234, -0.013986706, 0.0049793166, -0.046117682, -0.030708686, 0.058864865, -0.0285044, -0.0238416, 0.012596645, -0.017274922, 0.024820734, 0.035056587, -0.051926214, 0.010844215, 0.024518587, -0.072921835, 0.023325954, -0.0029717826, -0.036330853, 0.008756138, 0.052033767, 0.00037948627 ] } ], "_source": false }
测试结果
测试结果仅供参考,无法代表实际生产中写入查询情况,建议您结合业务生产数据进行压测。
当前测试结果仅提供数据写入后直接查询的结果,如果通过forcemerge成一个segment的方式进行优化,实际查询效果还有数倍提升空间。
查询性能
KNN查询case说明:
knn-search-100-100:k=100,num_candidates=100。
knn-search-1000-1000:k=1000,num_candidates=1000。
20并发测试结果
ES版本 | 查询case | QPS(次/秒) | 50分位查询耗时(ms) | 99分位查询耗时(ms) | CPU使用率(%) |
8.13版本 | knn-search-100-100 | 813.16 | 22.2122 | 45.159 | 92% |
knn-search-1000-1000 | 157.58 | 123.515 | 205.025 | 100% | |
8.9版本 | knn-search-100-100 | 191.32 | 97.4562 | 159.974 | 99% |
knn-search-1000-1000 | 32.97 | 588.647 | 839.913 | 100% |
单并发测试结果
ES版本 | 查询case | QPS(次/秒) | 50分位查询耗时(ms) | 99分位查询耗时(ms) | CPU使用率(%) |
8.13版本 | knn-search-100-100 | 168.79 | 4.47948 | 6.00996 | 22% |
knn-search-1000-1000 | 69.18 | 13.2466 | 15.2825 | 42% | |
8.9版本 | knn-search-100-100 | 16.48 | 59.4866 | 65.296 | 7% |
knn-search-1000-1000 | 2.98 | 336.262 | 363.806 | 8% |
写入性能
ES版本 | 写入TPS(docs/s) | 写入耗时(ms) (bulk size=1000) | CPU使用率(%) |
8.13版本 | 8201.66 | 1105.91 | 100 |
8.9版本 | 4711.86 | 2263.13 | 100 |
存储表现
引擎版本 | 存储容量 |
8.13版本 | 30.8 GB |
8.9版本 | 30.8 GB |
测试结果总结
查询性能:ES 8.13版本的查询性能是ES 8.9版本的4-5倍,单并发可以到8.9版本的10倍。
写入性能:ES 8.13版本的写入性能是ES 8.9版本的1.74倍。