Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Opt](load) Optimize the error messages of -235 and -238 for loading #41048 #41645

Merged

Conversation

liaoxin01
Copy link
Contributor

cherry pick from #41048

@liaoxin01
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 37.82% (8149/21546)
Line Coverage: 29.55% (67059/226963)
Region Coverage: 29.03% (34586/119123)
Branch Coverage: 24.95% (17823/71440)
Coverage Report: http://coverage.selectdb-in.cc/coverage/41e93af2cff17875c7e0695aa34eadddc10939fa_41e93af2cff17875c7e0695aa34eadddc10939fa/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 48815 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 41e93af2cff17875c7e0695aa34eadddc10939fa, data reload: false

------ Round 1 ----------------------------------
q1	17991	4359	4308	4308
q2	2063	150	147	147
q3	10267	1902	1911	1902
q4	10361	1246	1329	1246
q5	8492	3922	3946	3922
q6	230	122	124	122
q7	2035	1613	1574	1574
q8	9278	2730	2696	2696
q9	9944	9849	9702	9702
q10	8647	3483	3521	3483
q11	413	244	249	244
q12	466	299	294	294
q13	18361	3959	4006	3959
q14	365	323	323	323
q15	510	468	459	459
q16	539	452	450	450
q17	1126	957	915	915
q18	7182	6763	6971	6763
q19	1661	1593	1521	1521
q20	553	316	297	297
q21	4412	4120	4095	4095
q22	504	404	393	393
Total cold run time: 115400 ms
Total hot run time: 48815 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4322	4294	4292	4292
q2	322	227	226	226
q3	4133	4145	4126	4126
q4	2768	2747	2757	2747
q5	7149	7096	7136	7096
q6	237	117	118	117
q7	3272	2833	2781	2781
q8	4375	4425	4454	4425
q9	13544	13635	13650	13635
q10	4220	4265	4249	4249
q11	785	700	722	700
q12	1011	841	849	841
q13	6647	3714	3716	3714
q14	461	421	426	421
q15	495	460	468	460
q16	630	588	590	588
q17	3822	3828	3800	3800
q18	8755	8740	8887	8740
q19	1725	1673	1644	1644
q20	2359	2159	2115	2115
q21	8421	8549	8435	8435
q22	1039	928	921	921
Total cold run time: 80492 ms
Total hot run time: 76073 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211929 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 41e93af2cff17875c7e0695aa34eadddc10939fa, data reload: false

query1	922	386	434	386
query2	6553	2100	2218	2100
query3	6915	204	199	199
query4	22846	21319	21647	21319
query5	19727	6514	6538	6514
query6	288	219	235	219
query7	4336	300	301	300
query8	267	254	240	240
query9	3035	2624	2567	2567
query10	467	308	313	308
query11	15842	14715	14844	14715
query12	124	77	74	74
query13	1039	438	434	434
query14	17273	13515	13825	13515
query15	435	221	229	221
query16	6904	288	263	263
query17	2069	934	927	927
query18	905	324	313	313
query19	212	149	150	149
query20	111	96	104	96
query21	193	93	95	93
query22	5301	4903	5080	4903
query23	34736	33710	34264	33710
query24	8085	6452	6486	6452
query25	605	467	459	459
query26	1273	173	186	173
query27	2454	336	298	298
query28	6875	2261	2248	2248
query29	2963	2716	2718	2716
query30	244	167	166	166
query31	924	763	761	761
query32	70	66	59	59
query33	602	266	268	266
query34	1144	491	477	477
query35	1144	909	983	909
query36	1320	1278	1234	1234
query37	172	59	60	59
query38	3087	2933	2897	2897
query39	1370	1312	1313	1312
query40	306	98	94	94
query41	39	44	36	36
query42	87	93	85	85
query43	662	594	565	565
query44	1134	707	707	707
query45	239	234	230	230
query46	1227	953	975	953
query47	1878	1755	1655	1655
query48	515	420	403	403
query49	649	370	368	368
query50	852	617	585	585
query51	4776	4657	4721	4657
query52	92	76	87	76
query53	232	193	192	192
query54	2658	2459	2462	2459
query55	90	74	91	74
query56	217	208	200	200
query57	1280	1119	1178	1119
query58	220	215	225	215
query59	3437	3212	3126	3126
query60	214	206	201	201
query61	101	95	94	94
query62	811	527	496	496
query63	205	171	170	170
query64	3544	1602	1480	1480
query65	3658	3539	3565	3539
query66	765	395	435	395
query67	16304	15306	16942	15306
query68	9561	656	642	642
query69	490	281	265	265
query70	1608	1252	1473	1252
query71	409	311	318	311
query72	6916	4838	4769	4769
query73	755	321	337	321
query74	6201	5734	5803	5734
query75	5223	3612	3635	3612
query76	5123	1142	1213	1142
query77	855	269	261	261
query78	12669	11520	12806	11520
query79	13502	614	635	614
query80	1603	399	396	396
query81	495	232	231	231
query82	1267	96	99	96
query83	187	139	131	131
query84	257	68	69	68
query85	1073	328	318	318
query86	352	311	280	280
query87	3265	3037	3039	3037
query88	5382	2298	2290	2290
query89	452	297	283	283
query90	2229	219	214	214
query91	164	123	125	123
query92	57	52	53	52
query93	6064	590	560	560
query94	1120	210	206	206
query95	1972	1916	1974	1916
query96	640	316	323	316
query97	6428	6394	6362	6362
query98	206	227	211	211
query99	2881	865	890	865
Total cold run time: 328430 ms
Total hot run time: 211929 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.89 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 41e93af2cff17875c7e0695aa34eadddc10939fa, data reload: false

query1	0.02	0.02	0.03
query2	0.06	0.03	0.02
query3	0.24	0.05	0.05
query4	1.80	0.11	0.08
query5	0.54	0.52	0.52
query6	1.23	0.62	0.63
query7	0.01	0.01	0.01
query8	0.04	0.03	0.03
query9	0.52	0.51	0.48
query10	0.53	0.54	0.54
query11	0.11	0.09	0.09
query12	0.12	0.09	0.08
query13	0.61	0.63	0.61
query14	0.79	0.78	0.78
query15	0.79	0.77	0.75
query16	0.38	0.39	0.36
query17	1.01	1.01	0.97
query18	0.21	0.25	0.23
query19	1.87	1.85	1.85
query20	0.01	0.01	0.00
query21	15.48	0.57	0.55
query22	2.01	2.00	1.45
query23	17.45	0.92	0.96
query24	5.09	1.20	1.50
query25	0.35	0.11	0.07
query26	0.63	0.16	0.15
query27	0.04	0.04	0.03
query28	7.36	0.78	0.74
query29	12.75	2.31	2.31
query30	0.60	0.52	0.53
query31	2.82	0.38	0.38
query32	3.37	0.51	0.50
query33	3.10	3.06	3.09
query34	15.24	4.80	4.80
query35	4.88	4.86	4.86
query36	1.05	1.01	1.01
query37	0.06	0.04	0.04
query38	0.03	0.02	0.02
query39	0.02	0.01	0.01
query40	0.16	0.15	0.14
query41	0.06	0.01	0.02
query42	0.02	0.01	0.01
query43	0.03	0.02	0.02
Total cold run time: 103.49 s
Total hot run time: 30.89 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit 41e93af2cff17875c7e0695aa34eadddc10939fa with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       22.2 seconds inserted 10000000 Rows, about 450K ops/s

@dataroaring dataroaring merged commit 49b1ada into apache:branch-2.0 Oct 10, 2024
20 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants