Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ci](perf) integration clickbench test into new performance pipeline #29429

Merged
merged 24 commits into from
Jan 5, 2024
Merged

[ci](perf) integration clickbench test into new performance pipeline #29429

merged 24 commits into from
Jan 5, 2024

Conversation

hello-stephen
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@hello-stephen
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit fc4b8b071561b054edbe22a516077fde6cec9a76, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5542	5102	5149	5102
q2	393	194	149	149
q3	1467	1239	1196	1196
q4	1110	811	804	804
q5	3184	3140	3143	3140
q6	227	138	140	138
q7	984	569	525	525
q8	2148	2294	2225	2225
q9	6687	6702	6669	6669
q10	3140	3131	3135	3131
q11	350	219	207	207
q12	384	238	234	234
q13	4400	3647	3627	3627
q14	242	227	215	215
q15	601	565	548	548
q16	455	411	415	411
q17	1051	558	515	515
q18	7065	6774	6859	6774
q19	1663	1538	1399	1399
q20	601	355	350	350
q21	2863	2443	2462	2443
q22	394	313	317	313
Total cold run time: 44951 ms
Total hot run time: 40115 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5171	5108	4979	4979
q2	330	239	248	239
q3	3379	3345	3300	3300
q4	2142	1978	1969	1969
q5	5953	5938	5938	5938
q6	231	125	123	123
q7	2402	1893	1957	1893
q8	3572	3632	3677	3632
q9	9093	9022	8953	8953
q10	3862	3900	3878	3878
q11	585	483	495	483
q12	814	635	658	635
q13	3898	3197	3176	3176
q14	298	265	266	265
q15	641	543	555	543
q16	557	505	503	503
q17	2043	1807	1800	1800
q18	8758	8297	8446	8297
q19	1751	1699	1663	1663
q20	2282	1988	1975	1975
q21	5775	5250	5315	5250
q22	550	514	456	456
Total cold run time: 64087 ms
Total hot run time: 59950 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.3 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.2 seconds inserted 10000000 Rows, about 354K ops/s
storage size: 17183501773 Bytes

@hello-stephen
Copy link
Contributor Author

run performance

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 293f56057d7f1e80aa2cd4fc66861146a3b16a62, data reload: false

------ Round 1 ----------------------------------
q1	17662	5079	5068	5068
q2	2022	151	143	143
q3	10528	1075	1069	1069
q4	10182	788	771	771
q5	7779	2926	2866	2866
q6	208	133	130	130
q7	917	554	504	504
q8	9262	2005	2011	2005
q9	6846	6396	6416	6396
q10	8281	3053	3000	3000
q11	434	221	211	211
q12	390	234	233	233
q13	18014	3632	3621	3621
q14	238	212	215	212
q15	576	537	522	522
q16	481	414	396	396
q17	961	440	517	440
q18	7219	6703	6694	6694
q19	1570	1330	1436	1330
q20	717	333	344	333
q21	2778	2291	2408	2291
q22	380	320	311	311
Total cold run time: 107445 ms
Total hot run time: 38546 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5121	5103	5095	5095
q2	335	248	263	248
q3	3308	3252	3233	3233
q4	2094	1999	1999	1999
q5	5773	5746	5748	5746
q6	211	125	123	123
q7	2280	1912	1921	1912
q8	3337	3443	3470	3443
q9	8821	8737	8717	8717
q10	3789	3834	3813	3813
q11	584	491	477	477
q12	783	627	638	627
q13	9753	3186	3208	3186
q14	293	265	266	265
q15	592	539	524	524
q16	543	507	527	507
q17	1960	1799	1736	1736
q18	8677	8276	8182	8182
q19	1608	1562	1580	1562
q20	2198	1981	1942	1942
q21	5554	5275	5148	5148
q22	585	472	521	472
Total cold run time: 68199 ms
Total hot run time: 58957 ms

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 293f56057d7f1e80aa2cd4fc66861146a3b16a62, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	931	345	332	332
query2	6425	1931	1924	1924
query3	6658	206	204	204
query4	26261	22573	22593	22573
query5	5271	520	518	518
query6	273	209	201	201
query7	4594	281	264	264
query8	224	203	196	196
query9	8241	2618	2684	2618
query10	450	238	239	238
query11	16087	15526	15506	15506
query12	139	77	77	77
query13	1633	323	337	323
query14	11531	7264	7088	7088
query15	245	191	188	188
query16	6492	273	272	272
query17	1857	511	499	499
query18	1932	267	268	267
query19	276	140	139	139
query20	82	77	80	77
query21	186	99	98	98
query22	5109	4872	4924	4872
query23	32422	31097	30963	30963
query24	12078	2811	2802	2802
query25	590	351	346	346
query26	1709	144	143	143
query27	2902	286	279	279
query28	6845	1987	1952	1952
query29	2047	385	396	385
query30	295	142	144	142
query31	962	772	783	772
query32	93	58	61	58
query33	728	274	273	273
query34	887	452	432	432
query35	936	765	710	710
query36	1409	1251	1281	1251
query37	193	73	68	68
query38	3414	3263	3295	3263
query39	1340	1303	1269	1269
query40	292	91	95	91
query41	37	38	34	34
query42	87	96	93	93
query43	549	474	492	474
query44	1057	710	722	710
query45	195	193	182	182
query46	1049	654	669	654
query47	1674	1614	1630	1614
query48	340	264	265	264
query49	1198	325	330	325
query50	796	322	350	322
query51	5481	5262	5361	5262
query52	89	90	86	86
query53	222	150	149	149
query54	1343	559	577	559
query55	98	90	88	88
query56	215	203	196	196
query57	999	956	914	914
query58	218	203	199	199
query59	2770	2578	2565	2565
query60	256	255	254	254
query61	88	85	86	85
query62	635	498	479	479
query63	167	148	147	147
query64	5839	1779	1726	1726
query65	3329	3266	3262	3262
query66	1312	334	341	334
query67	15800	15332	15307	15307
query68	12522	530	524	524
query69	512	258	243	243
query70	1731	1490	1513	1490
query71	508	225	235	225
query72	5672	3633	3609	3609
query73	2862	314	315	314
query74	7010	6500	6440	6440
query75	5138	2285	2283	2283
query76	6271	1179	1131	1131
query77	656	279	288	279
query78	9114	8543	8839	8543
query79	1057	512	510	510
query80	553	371	346	346
query81	467	210	211	210
query82	209	101	101	101
query83	163	134	138	134
query84	248	53	51	51
query85	940	285	288	285
query86	393	364	386	364
query87	3661	3431	3372	3372
query88	3053	2254	2260	2254
query89	335	263	269	263
query90	1865	205	205	205
query91	123	92	97	92
query92	62	52	52	52
query93	1520	434	503	434
query94	803	195	186	186
query95	450	417	404	404
query96	622	318	322	318
query97	4275	4171	4224	4171
query98	211	187	191	187
query99	1143	864	828	828
Total cold run time: 294031 ms
Total hot run time: 179805 ms

stephen added 2 commits January 3, 2024 11:27
@hello-stephen
Copy link
Contributor Author

run performance

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 367519263340cb2bb7998e62ca62004178fa3240, data reload: false

------ Round 1 ----------------------------------
q1	6600	5137	5035	5035
q2	1795	166	149	149
q3	1424	1142	1191	1142
q4	1381	781	846	781
q5	2971	2880	2949	2880
q6	229	161	150	150
q7	976	524	512	512
q8	9360	2007	2015	2007
q9	6844	6424	6409	6409
q10	8310	3043	3104	3043
q11	425	226	211	211
q12	392	231	237	231
q13	17999	3642	3643	3642
q14	242	217	205	205
q15	582	537	537	537
q16	469	416	416	416
q17	975	458	460	458
q18	7403	6696	6671	6671
q19	1594	1296	1398	1296
q20	653	340	358	340
q21	2748	2359	2456	2359
q22	384	297	334	297
Total cold run time: 73756 ms
Total hot run time: 38771 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5176	5121	5051	5051
q2	337	248	276	248
q3	3304	3278	3252	3252
q4	2101	2056	2013	2013
q5	5818	5783	5820	5783
q6	220	126	126	126
q7	2334	1946	1948	1946
q8	3397	3494	3487	3487
q9	8850	8858	8791	8791
q10	3820	3879	3834	3834
q11	586	485	496	485
q12	806	654	644	644
q13	8253	3237	3209	3209
q14	289	273	277	273
q15	598	534	532	532
q16	570	502	531	502
q17	1961	1764	1759	1759
q18	8707	8444	8248	8248
q19	1631	1598	1601	1598
q20	2203	1969	1951	1951
q21	5721	5273	5320	5273
q22	513	445	519	445
Total cold run time: 67195 ms
Total hot run time: 59450 ms

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit 367519263340cb2bb7998e62ca62004178fa3240, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	920	345	332	332
query2	6445	1911	1966	1911
query3	6647	214	208	208
query4	24121	22511	22403	22403
query5	4407	521	527	521
query6	283	191	201	191
query7	4519	298	274	274
query8	253	218	209	209
query9	8239	2750	2657	2657
query10	374	251	265	251
query11	16070	15474	15614	15474
query12	136	77	73	73
query13	1533	335	326	326
query14	11873	7388	7496	7388
query15	356	211	232	211
query16	6451	269	264	264
query17	1761	518	517	517
query18	1949	270	262	262
query19	278	155	159	155
query20	81	80	81	80
query21	185	101	97	97
query22	4862	4910	4675	4675
query23	32270	31142	31311	31142
query24	12367	3454	3452	3452
query25	585	371	369	369
query26	1501	151	157	151
query27	2666	293	291	291
query28	6381	1998	1960	1960
query29	1675	428	419	419
query30	308	147	158	147
query31	1125	773	795	773
query32	94	64	56	56
query33	692	281	280	280
query34	839	446	440	440
query35	984	846	756	756
query36	1272	1252	1244	1244
query37	188	72	67	67
query38	3461	3347	3301	3301
query39	1357	1314	1306	1306
query40	305	96	100	96
query41	44	35	34	34
query42	96	84	96	84
query43	512	476	503	476
query44	1056	695	720	695
query45	232	231	217	217
query46	1186	795	792	792
query47	1621	1628	1538	1538
query48	350	277	284	277
query49	1208	323	319	319
query50	1073	886	824	824
query51	5350	5329	5322	5322
query52	92	86	89	86
query53	211	180	184	180
query54	1405	634	654	634
query55	97	86	83	83
query56	213	200	197	197
query57	1018	975	925	925
query58	242	210	204	204
query59	2784	2449	2583	2449
query60	250	238	238	238
query61	85	82	84	82
query62	1375	492	506	492
query63	192	172	174	172
query64	6062	1818	1741	1741
query65	3345	3284	3288	3284
query66	1108	341	332	332
query67	15752	15759	15335	15335
query68	11857	528	528	528
query69	517	251	246	246
query70	1675	1535	1522	1522
query71	509	222	223	222
query72	6343	4026	4040	4026
query73	2661	317	311	311
query74	6988	6510	6531	6510
query75	5106	2271	2300	2271
query76	6325	1156	1148	1148
query77	660	287	303	287
query78	9109	9010	8511	8511
query79	1105	526	527	526
query80	539	365	371	365
query81	469	219	213	213
query82	187	101	100	100
query83	182	137	137	137
query84	242	55	53	53
query85	866	281	280	280
query86	380	372	427	372
query87	3646	3395	3409	3395
query88	2957	2248	2270	2248
query89	346	283	291	283
query90	1804	213	209	209
query91	116	93	100	93
query92	59	53	50	50
query93	1181	510	482	482
query94	745	184	184	184
query95	464	425	400	400
query96	625	319	318	318
query97	4324	4171	4203	4171
query98	213	198	191	191
query99	1671	938	912	912
Total cold run time: 290643 ms
Total hot run time: 182305 ms

@doris-robot
Copy link

ClickBench test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/clickbench-tools

ClickBench test result on commit 367519263340cb2bb7998e62ca62004178fa3240, data reload: true

query1	0.02	0.02	0.01
query2	0.09	0.02	0.02
query3	0.23	0.04	0.05
query4	1.86	0.06	0.07
query5	0.54	0.51	0.51
query6	1.46	0.65	0.64
query7	0.00	0.01	0.01
query8	0.04	0.03	0.03
query9	0.53	0.49	0.48
query10	0.55	0.54	0.54
query11	0.15	0.10	0.10
query12	0.15	0.09	0.10
query13	0.62	0.61	0.61
query14	0.81	0.82	0.78
query15	0.82	0.81	0.82
query16	0.34	0.34	0.34
query17	1.01	0.95	0.96
query18	0.23	0.23	0.22
query19	1.87	1.73	1.80
query20	0.02	0.01	0.01
query21	15.32	0.55	0.55
query22	1.18	0.86	1.26
query23	17.11	0.72	0.72
query24	12.64	0.43	0.43
query25	1.10	0.16	0.15
query26	0.14	0.14	0.14
query27	0.16	0.16	0.16
query28	6.75	0.93	0.92
query29	12.63	3.17	3.17
query30	0.54	0.47	0.57
query31	2.80	0.38	0.37
query32	3.35	0.49	0.50
query33	3.25	3.26	3.23
query34	16.28	4.32	4.28
query35	4.21	4.28	4.36
query36	1.09	1.05	1.05
query37	0.08	0.05	0.06
query38	0.05	0.03	0.03
query39	0.03	0.03	0.02
query40	0.16	0.13	0.13
query41	0.08	0.03	0.03
query42	0.03	0.03	0.02
query43	0.03	0.02	0.02
Total cold run time: 110.35 s
Total hot run time: 29.55 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 367519263340cb2bb7998e62ca62004178fa3240 with default conf and session variables
 stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
 stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
 stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
 insert into select:          18.8 seconds inserted 10000000 Rows, about 531K ops/s

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2024

sh-checker report

To get the full details, please check in the job output.

shellcheck errors 
'shellcheck ' found no issues.
shfmt errors 

'shfmt ' returned error 1 finding the following formatting issues:

----------
--- regression-test/pipeline/performance/run-tpch.sh.orig
+++ regression-test/pipeline/performance/run-tpch.sh
@@ -54,7 +54,6 @@
 if [[ -z "${commit_id}" ]]; then echo "ERROR: env commit_id not set" && exit 1; fi
 if [[ -z "${target_branch}" ]]; then echo "ERROR: env target_branch not set" && exit 1; fi
 
-
 # shellcheck source=/dev/null
 source "$(bash "${teamcity_build_checkoutDir}"/regression-test/pipeline/common/get-or-set-tmp-env.sh 'get')"
 if ${skip_pipeline:=false}; then echo "INFO: skip build pipline" && exit 0; else echo "INFO: no skip"; fi
----------

You can reformat the above files to meet shfmt's requirements by typing:

  shfmt  -w filename

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.2 seconds
stream load tsv: 569 seconds loaded 74807831229 Bytes, about 125 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 27.9 seconds inserted 10000000 Rows, about 358K ops/s
storage size: 17183915983 Bytes

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit ef89af1c71f08fd31466a9cdb46368bee2d3eca9, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5501	5211	5081	5081
q2	394	169	161	161
q3	1462	1126	1124	1124
q4	1078	752	825	752
q5	3160	3127	3153	3127
q6	218	132	134	132
q7	979	548	515	515
q8	2170	2253	2254	2253
q9	6707	6667	6670	6667
q10	3213	3213	3091	3091
q11	332	212	209	209
q12	381	232	236	232
q13	4387	3641	3612	3612
q14	254	223	218	218
q15	584	538	535	535
q16	451	414	398	398
q17	1044	540	590	540
q18	7057	6760	7038	6760
q19	1647	1476	1522	1476
q20	543	333	340	333
q21	2854	2437	2496	2437
q22	406	319	349	319
Total cold run time: 44822 ms
Total hot run time: 39972 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5196	5101	5154	5101
q2	339	234	253	234
q3	3348	3330	3326	3326
q4	2155	2032	2033	2032
q5	5940	5925	5913	5913
q6	223	123	125	123
q7	2379	1939	1921	1921
q8	3584	3642	3696	3642
q9	9033	8958	9026	8958
q10	3857	3922	3906	3906
q11	568	461	482	461
q12	803	640	632	632
q13	3871	3220	3190	3190
q14	320	277	279	277
q15	584	531	540	531
q16	556	501	540	501
q17	2052	1799	1816	1799
q18	8727	8408	8342	8342
q19	1765	1717	1684	1684
q20	2266	1982	1985	1982
q21	5722	5301	5454	5301
q22	569	489	493	489
Total cold run time: 63857 ms
Total hot run time: 60345 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' Total cold run time: 301464 ms Total hot run time: 239000 ms ``` query941251183177177 query95530488483483 query96657331311311 query976591640664786406 query98185175174174 query9936881017997997 ```

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' Total cold run time: 301464 ms Total hot run time: 239000 ms 
query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

Total cold run time: 301464 ms Total hot run time: 239000 ms

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

TPC-H test result: Total cold run time: 301464 ms Total hot run time: 239000 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

TPC-H SF100 test result: Total cold run time: 301464 ms Total hot run time: 239000 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

TPC-H SF100 test result: Total cold run time: 301464 ms Total hot run time: 239000 ms<�> 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

TPC-H SF100 test result: Total cold run time: 301464 ms Total hot run time: 239000 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@doris-robot
Copy link

TPC-H SF100 test result: Total hot run time: 239000 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

query941251183177177
query95530488483483
query96657331311311
query976591640664786406
query98185175174174
query9936881017997997

@hello-stephen
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Tpch sf100 test result on commit 0fe184e, data reload: false

run tpch-sf100 query with default conf and session variables
q1 5692 5393 5380 5380
q2 399 167 158 158
q3 1554 1282 1268 1268
q4 1159 900 874 874
q5 3220 3198 3243 3198
q6 247 144 139 139
q7 1044 509 502 502
q8 2316 2437 2330 2330
q9 6960 6928 6885 6885
q10 3252 3193 3179 3179
q11 352 218 218 218
q12 401 236 234 234
q13 4473 3650 3656 3650
q14 255 217 215 215
q15 580 539 539 539
q16 457 408 391 391
q17 1068 540 528 528
q18 7181 6728 6800 6728
q19 1790 1676 1743 1676
q20 590 345 346 345
q21 3032 2597 2578 2578
q22 416 334 345 334
Total cold run time: 46438 ms
Total hot run time: 41349 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1 5452 5377 5372 5372
q2 361 236 234 234
q3 3448 3414 3392 3392
q4 2239 2108 2121 2108
q5 6135 6101 6104 6101
q6 244 128 128 128
q7 2419 1896 1944 1896
q8 3724 3840 3889 3840
q9 9105 9094 9072 9072
q10 3868 3991 4023 3991
q11 611 477 473 473
q12 826 634 628 628
q13 4020 3188 3168 3168
q14 314 266 306 266
q15 596 517 533 517
q16 548 472 520 472
q17 2130 1893 1794 1794
q18 8939 8431 8560 8431
q19 1933 1869 1888 1869
q20 2315 1968 1991 1968
q21 5953 5426 5500 5426
q22 600 499 548 499
Total cold run time: 65780 ms
Total hot run time: 61645 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools

@doris-robot
Copy link

TPC-H: Total hot run time: 39338 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 0fe184e888813566561edfa0fdba134f1e04b0bc, data reload: false

------ Round 1 ----------------------------------
q1	17698	5024	5070	5024
q2	2015	150	139	139
q3	10598	1140	1173	1140
q4	10209	833	779	779
q5	7789	3046	2926	2926
q6	217	122	120	120
q7	939	509	487	487
q8	9342	2220	2207	2207
q9	6688	6606	6560	6560
q10	8318	3105	3073	3073
q11	445	208	218	208
q12	421	229	230	229
q13	17987	3384	3334	3334
q14	240	204	203	203
q15	569	514	488	488
q16	469	383	397	383
q17	992	757	640	640
q18	7369	6650	6701	6650
q19	1724	1673	1688	1673
q20	715	318	303	303
q21	2838	2509	2459	2459
q22	350	316	313	313
Total cold run time: 107932 ms
Total hot run time: 39338 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5138	5105	5043	5043
q2	345	224	237	224
q3	3332	3321	3343	3321
q4	2206	2106	2086	2086
q5	5904	5958	5918	5918
q6	219	116	115	115
q7	2347	1885	1883	1883
q8	3563	3616	3616	3616
q9	8822	8797	8738	8738
q10	3814	3896	3906	3896
q11	554	436	436	436
q12	813	622	630	622
q13	6060	3162	3204	3162
q14	303	254	247	247
q15	539	511	500	500
q16	557	463	509	463
q17	1978	1954	1944	1944
q18	8676	8493	8341	8341
q19	1741	1808	1802	1802
q20	2234	1960	1943	1943
q21	5824	5475	5419	5419
q22	515	461	451	451
Total cold run time: 65484 ms
Total hot run time: 60170 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.05 seconds
stream load tsv: 564 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 26.5 seconds inserted 10000000 Rows, about 377K ops/s
storage size: 17183620698 Bytes

@doris-robot
Copy link

TPC-DS: Total hot run time: 184070 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 0fe184e888813566561edfa0fdba134f1e04b0bc, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	938	344	337	337
query2	6564	2066	1915	1915
query3	6705	236	223	223
query4	27291	22556	22417	22417
query5	5771	625	582	582
query6	320	207	212	207
query7	4629	296	292	292
query8	277	235	226	226
query9	8779	2922	2929	2922
query10	557	323	316	316
query11	16229	15625	15603	15603
query12	154	99	99	99
query13	1678	339	341	339
query14	13326	7643	7577	7577
query15	273	240	219	219
query16	6388	344	335	335
query17	1702	530	515	515
query18	1965	302	294	294
query19	238	159	155	155
query20	105	99	96	96
query21	188	96	96	96
query22	5217	5142	4926	4926
query23	32137	31455	31273	31273
query24	12123	2933	2981	2933
query25	595	357	358	357
query26	1801	168	168	168
query27	2772	318	317	317
query28	6842	2157	2132	2132
query29	2038	427	415	415
query30	283	144	149	144
query31	1035	810	825	810
query32	120	85	71	71
query33	852	347	345	345
query34	854	470	484	470
query35	984	894	834	834
query36	1385	1295	1305	1295
query37	209	92	79	79
query38	3485	3404	3299	3299
query39	1362	1286	1300	1286
query40	292	89	87	87
query41	45	38	39	38
query42	108	105	102	102
query43	547	534	554	534
query44	1211	798	806	798
query45	207	213	200	200
query46	1113	714	712	712
query47	1687	1688	1629	1629
query48	361	287	286	286
query49	1208	306	311	306
query50	796	362	360	360
query51	5389	5237	5340	5237
query52	111	97	103	97
query53	234	167	168	167
query54	1453	675	651	651
query55	103	97	93	93
query56	264	267	279	267
query57	1014	1027	987	987
query58	316	274	273	273
query59	3042	2898	2897	2897
query60	298	282	298	282
query61	152	144	145	144
query62	605	501	499	499
query63	195	172	176	172
query64	5999	1695	1707	1695
query65	3408	3354	3365	3354
query66	1355	392	375	375
query67	15821	15602	15114	15114
query68	12931	551	552	551
query69	594	296	333	296
query70	1823	1564	1647	1564
query71	591	297	294	294
query72	5343	3429	3399	3399
query73	2969	341	340	340
query74	7400	6439	6558	6439
query75	5382	2291	2329	2291
query76	6287	1124	1208	1124
query77	751	358	338	338
query78	9182	8600	8673	8600
query79	1079	556	537	537
query80	543	353	336	336
query81	464	214	211	211
query82	229	105	109	105
query83	165	141	138	138
query84	254	57	56	56
query85	898	267	267	267
query86	454	451	435	435
query87	3695	3495	3480	3480
query88	3089	2501	2544	2501
query89	392	290	277	277
query90	1888	251	260	251
query91	135	96	96	96
query92	81	67	82	67
query93	1243	442	438	438
query94	821	250	245	245
query95	571	503	470	470
query96	640	336	332	332
query97	4418	4252	4269	4252
query98	238	212	213	212
query99	1135	853	858	853
Total cold run time: 301190 ms
Total hot run time: 184070 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.53 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 0fe184e888813566561edfa0fdba134f1e04b0bc, data reload: false

query1	0.03	0.07	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.68	0.08	0.07
query5	0.52	0.52	0.51
query6	1.47	0.65	0.65
query7	0.02	0.01	0.02
query8	0.04	0.03	0.03
query9	0.57	0.54	0.54
query10	0.59	0.59	0.59
query11	0.17	0.12	0.11
query12	0.15	0.12	0.11
query13	0.61	0.60	0.59
query14	0.78	0.79	0.79
query15	0.82	0.79	0.79
query16	0.34	0.34	0.35
query17	0.98	0.95	0.98
query18	0.25	0.24	0.23
query19	1.89	1.81	1.78
query20	0.01	0.02	0.02
query21	15.40	0.54	0.54
query22	2.24	4.09	1.19
query23	17.06	1.06	1.09
query24	8.28	0.84	1.31
query25	2.39	0.15	0.17
query26	0.15	0.15	0.15
query27	0.17	0.16	0.15
query28	5.50	0.75	0.77
query29	12.76	3.53	3.47
query30	0.51	0.50	0.45
query31	2.89	0.39	0.38
query32	3.23	0.49	0.48
query33	3.22	3.26	3.20
query34	16.67	4.54	4.58
query35	4.54	4.65	4.76
query36	1.15	1.09	1.07
query37	0.06	0.05	0.05
query38	0.03	0.02	0.02
query39	0.02	0.02	0.01
query40	0.16	0.13	0.13
query41	0.07	0.01	0.02
query42	0.02	0.01	0.02
query43	0.02	0.02	0.01
Total cold run time: 107.75 s
Total hot run time: 31.53 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 0fe184e888813566561edfa0fdba134f1e04b0bc with default session variables
 stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
 stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
 stream load parquet:      32 seconds loaded 861443392 Bytes, about 25 MB/s
 insert into select:       17.5 seconds inserted 10000000 Rows, about 571K ops/s

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 5, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 5, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 5, 2024

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 0ae449f into apache:master Jan 5, 2024
25 of 26 checks passed
@hello-stephen hello-stephen deleted the 0102-perf branch January 5, 2024 14:31
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
@hello-stephen @HappenLee
[ci](perf) integration clickbench test into new performance pipeline (a…
1546f9c 
…pache#29429)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@shuke987 shuke987 shuke987 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@hello-stephen @doris-robot @yiguolei @shuke987 @dataroaring