Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[style](properties)(log) Optimize invalid property alert words #47933

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

chuang-wang-pre
Copy link

@chuang-wang-pre chuang-wang-pre commented Feb 14, 2025

What problem does this PR solve?

Problem Summary:
When executing SQL statements with properties, if some properties are based on valid keys with spaces, the returned exception information may be misleading.

For example, this alter statement
alter table test_dynamic_partition set ( "dynamic_partition.start " = "-2");

[dynamic_partition.start] is a valid property, but the user may have typed an extra space

Doris returned the following result:

errCode = 2, detailMessage = Invalid dynamic partition properties: dynamic_partition.start

Users may find it difficult to identify the issue from the returned information and may consider it a bug

Optimized return information:

errCode = 2, detailMessage = Invalid dynamic partition properties: [dynamic_partition.start ]

If multiple properties are illegal, they will be returned:

errCode = 2, detailMessage = Invalid dynamic partition properties: [dynamic_partition.start ], [dynamic_partition.edn]

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Feb 14, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@chuang-wang-pre
Copy link
Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31495 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 07805d362b03c640c6c40027d93d2b43ff597529, data reload: false

------ Round 1 ----------------------------------
q1	17821	5231	5116	5116
q2	2218	330	174	174
q3	11005	1242	740	740
q4	10197	999	530	530
q5	7524	2342	2325	2325
q6	186	167	132	132
q7	899	750	599	599
q8	9307	1257	1057	1057
q9	4752	4650	4829	4650
q10	6834	2293	1876	1876
q11	463	275	254	254
q12	339	346	214	214
q13	17793	3694	3019	3019
q14	226	223	215	215
q15	503	456	457	456
q16	625	619	580	580
q17	591	870	346	346
q18	6551	6312	6207	6207
q19	1222	938	559	559
q20	307	322	201	201
q21	2808	2117	1941	1941
q22	367	339	304	304
Total cold run time: 102538 ms
Total hot run time: 31495 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5148	5112	5094	5094
q2	236	328	230	230
q3	2108	2658	2282	2282
q4	1450	1780	1362	1362
q5	4229	4134	4126	4126
q6	213	168	124	124
q7	1864	1800	1627	1627
q8	2564	2690	2626	2626
q9	7124	7134	7062	7062
q10	2980	3191	2749	2749
q11	572	504	507	504
q12	677	783	597	597
q13	3500	3809	3286	3286
q14	283	304	260	260
q15	512	478	470	470
q16	624	677	635	635
q17	1149	1557	1381	1381
q18	7681	7371	7248	7248
q19	818	858	921	858
q20	1945	2013	1866	1866
q21	5320	4955	4647	4647
q22	607	589	524	524
Total cold run time: 51604 ms
Total hot run time: 49558 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189713 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 07805d362b03c640c6c40027d93d2b43ff597529, data reload: false

query1	1294	974	951	951
query2	6206	1787	1800	1787
query3	10995	4562	4582	4562
query4	54379	24223	22976	22976
query5	4980	549	482	482
query6	337	190	184	184
query7	4898	513	293	293
query8	315	248	233	233
query9	5722	2532	2506	2506
query10	447	316	261	261
query11	15095	15010	14755	14755
query12	161	106	111	106
query13	1051	505	387	387
query14	10587	6602	6771	6602
query15	204	207	179	179
query16	7044	659	472	472
query17	1089	741	592	592
query18	1509	445	324	324
query19	200	198	178	178
query20	129	126	118	118
query21	215	135	108	108
query22	4578	4781	4483	4483
query23	33961	33363	33416	33363
query24	5644	2450	2398	2398
query25	446	464	385	385
query26	733	289	149	149
query27	1934	505	326	326
query28	2842	2441	2372	2372
query29	575	569	415	415
query30	209	185	155	155
query31	883	895	852	852
query32	71	65	79	65
query33	449	357	300	300
query34	789	861	490	490
query35	785	821	749	749
query36	963	1001	898	898
query37	126	98	79	79
query38	4378	4524	4133	4133
query39	1512	1415	1426	1415
query40	218	115	109	109
query41	51	51	49	49
query42	133	103	111	103
query43	504	517	479	479
query44	1277	784	790	784
query45	187	177	165	165
query46	894	1064	636	636
query47	1825	1897	1787	1787
query48	395	440	316	316
query49	702	545	425	425
query50	700	754	422	422
query51	4281	4280	4215	4215
query52	108	98	92	92
query53	234	257	191	191
query54	503	485	415	415
query55	85	80	76	76
query56	275	272	275	272
query57	1185	1159	1143	1143
query58	231	235	239	235
query59	2747	2911	2668	2668
query60	274	286	278	278
query61	119	127	112	112
query62	745	720	672	672
query63	230	187	183	183
query64	2163	1010	697	697
query65	3297	3163	3122	3122
query66	756	439	298	298
query67	15870	15702	15549	15549
query68	5382	784	505	505
query69	499	296	254	254
query70	1202	1147	1137	1137
query71	437	288	269	269
query72	6325	3783	3699	3699
query73	1285	755	337	337
query74	8946	9197	8910	8910
query75	3168	3137	2674	2674
query76	3809	1163	730	730
query77	523	361	270	270
query78	10109	10066	9315	9315
query79	2735	811	592	592
query80	677	550	464	464
query81	500	274	230	230
query82	504	126	93	93
query83	177	165	148	148
query84	294	86	77	77
query85	793	397	298	298
query86	389	303	277	277
query87	4509	4518	4444	4444
query88	3594	2158	2142	2142
query89	416	311	351	311
query90	1795	188	190	188
query91	136	140	114	114
query92	71	59	57	57
query93	2443	1026	572	572
query94	683	404	304	304
query95	349	267	260	260
query96	480	559	268	268
query97	2768	2869	2748	2748
query98	229	200	197	197
query99	1300	1372	1253	1253
Total cold run time: 294251 ms
Total hot run time: 189713 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.17 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 07805d362b03c640c6c40027d93d2b43ff597529, data reload: false

query1	0.03	0.03	0.03
query2	0.07	0.03	0.04
query3	0.23	0.07	0.06
query4	1.61	0.10	0.11
query5	0.42	0.44	0.41
query6	1.15	0.67	0.65
query7	0.02	0.02	0.01
query8	0.04	0.03	0.03
query9	0.57	0.52	0.52
query10	0.58	0.58	0.58
query11	0.15	0.10	0.10
query12	0.14	0.11	0.11
query13	0.62	0.60	0.60
query14	2.66	2.73	2.70
query15	0.92	0.84	0.85
query16	0.37	0.37	0.38
query17	1.02	1.04	1.07
query18	0.21	0.19	0.20
query19	1.95	1.84	1.99
query20	0.01	0.02	0.02
query21	15.36	0.91	0.55
query22	0.74	1.14	0.65
query23	14.98	1.39	0.60
query24	12.22	1.01	0.35
query25	0.39	0.13	0.16
query26	1.05	0.20	0.15
query27	0.05	0.06	0.05
query28	6.30	0.80	0.42
query29	12.62	4.01	3.27
query30	0.25	0.09	0.07
query31	2.83	0.58	0.41
query32	3.22	0.53	0.46
query33	3.02	2.96	3.04
query34	15.75	5.12	4.53
query35	4.51	4.52	4.54
query36	0.67	0.50	0.50
query37	0.09	0.06	0.06
query38	0.05	0.04	0.03
query39	0.03	0.03	0.03
query40	0.17	0.15	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.03	0.02
Total cold run time: 107.23 s
Total hot run time: 30.17 s

@chuang-wang-pre chuang-wang-pre force-pushed the feat/invalid_property_exception_msg branch from 07805d3 to e69ca71 Compare February 15, 2025 05:27
@chuang-wang-pre
Copy link
Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31359 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e69ca717bf418d922e7d6924a4774f6856293be2, data reload: false

------ Round 1 ----------------------------------
q1	17614	5241	5140	5140
q2	2044	306	161	161
q3	10418	1224	742	742
q4	10200	997	529	529
q5	7510	2378	2266	2266
q6	183	165	134	134
q7	896	743	590	590
q8	9292	1294	1054	1054
q9	4853	4842	4580	4580
q10	6848	2318	1882	1882
q11	466	269	251	251
q12	337	344	219	219
q13	17794	3698	3116	3116
q14	221	225	222	222
q15	507	484	455	455
q16	616	598	573	573
q17	581	865	340	340
q18	6735	6158	6165	6158
q19	1200	959	536	536
q20	310	319	185	185
q21	2800	2110	1911	1911
q22	357	330	315	315
Total cold run time: 101782 ms
Total hot run time: 31359 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5110	5077	5093	5077
q2	231	331	226	226
q3	2161	2701	2287	2287
q4	1407	1812	1362	1362
q5	4236	4148	4160	4148
q6	210	162	125	125
q7	1852	1836	1674	1674
q8	2560	2665	2578	2578
q9	7249	7187	7182	7182
q10	2990	3296	2752	2752
q11	574	519	507	507
q12	699	794	630	630
q13	3563	3844	3334	3334
q14	291	315	263	263
q15	519	459	457	457
q16	629	684	644	644
q17	1105	1598	1323	1323
q18	7625	7426	7244	7244
q19	803	830	1006	830
q20	1957	2017	1863	1863
q21	5360	4892	4791	4791
q22	608	584	534	534
Total cold run time: 51739 ms
Total hot run time: 49831 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189862 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e69ca717bf418d922e7d6924a4774f6856293be2, data reload: false

query1	1322	971	914	914
query2	6279	1840	1845	1840
query3	11079	4647	4633	4633
query4	53897	25574	22995	22995
query5	5004	601	475	475
query6	358	196	198	196
query7	5042	506	297	297
query8	318	243	242	242
query9	5650	2528	2527	2527
query10	432	306	264	264
query11	15213	15190	15311	15190
query12	160	109	106	106
query13	1046	520	372	372
query14	10517	6387	6580	6387
query15	203	199	188	188
query16	7108	663	506	506
query17	1067	729	563	563
query18	1532	434	320	320
query19	208	199	191	191
query20	130	131	128	128
query21	210	131	106	106
query22	4467	4414	4217	4217
query23	34687	33246	33537	33246
query24	5859	2459	2440	2440
query25	451	475	415	415
query26	706	278	148	148
query27	1975	484	343	343
query28	2929	2438	2404	2404
query29	585	560	431	431
query30	221	190	159	159
query31	925	897	790	790
query32	74	62	60	60
query33	436	369	303	303
query34	768	867	513	513
query35	795	825	759	759
query36	964	1018	923	923
query37	127	100	76	76
query38	4309	4415	4393	4393
query39	1469	1440	1439	1439
query40	204	121	105	105
query41	54	59	54	54
query42	124	109	114	109
query43	497	504	477	477
query44	1349	806	809	806
query45	183	177	163	163
query46	873	1064	668	668
query47	1863	1853	1758	1758
query48	402	405	302	302
query49	717	514	419	419
query50	710	847	454	454
query51	4224	4333	4228	4228
query52	112	108	103	103
query53	249	268	188	188
query54	488	489	426	426
query55	82	78	81	78
query56	298	276	254	254
query57	1157	1208	1158	1158
query58	241	252	240	240
query59	2774	2866	2629	2629
query60	286	283	273	273
query61	122	123	115	115
query62	730	726	719	719
query63	232	197	192	192
query64	1803	1040	656	656
query65	3211	3112	3146	3112
query66	769	391	287	287
query67	15849	15590	15256	15256
query68	5500	777	501	501
query69	512	298	271	271
query70	1190	1099	1113	1099
query71	438	322	291	291
query72	5901	3619	3760	3619
query73	1266	717	347	347
query74	8946	9155	8833	8833
query75	3197	3152	2661	2661
query76	3844	1164	734	734
query77	539	365	314	314
query78	9974	10053	9374	9374
query79	2600	789	584	584
query80	673	511	439	439
query81	495	273	246	246
query82	491	125	91	91
query83	172	175	149	149
query84	291	86	71	71
query85	765	344	299	299
query86	359	322	300	300
query87	4459	4541	4465	4465
query88	3386	2181	2157	2157
query89	412	324	277	277
query90	1682	190	189	189
query91	143	140	107	107
query92	71	58	56	56
query93	2000	997	569	569
query94	728	419	304	304
query95	348	267	254	254
query96	487	557	267	267
query97	2833	2824	2772	2772
query98	237	211	206	206
query99	1414	1398	1288	1288
Total cold run time: 293478 ms
Total hot run time: 189862 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.3 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e69ca717bf418d922e7d6924a4774f6856293be2, data reload: false

query1	0.04	0.03	0.02
query2	0.07	0.03	0.03
query3	0.23	0.07	0.07
query4	1.62	0.10	0.10
query5	0.42	0.40	0.40
query6	1.18	0.66	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.04
query9	0.60	0.51	0.53
query10	0.58	0.58	0.57
query11	0.16	0.10	0.10
query12	0.15	0.11	0.11
query13	0.61	0.60	0.59
query14	2.68	2.76	2.72
query15	0.91	0.86	0.84
query16	0.38	0.37	0.38
query17	1.08	1.03	1.01
query18	0.21	0.19	0.19
query19	1.84	1.84	2.00
query20	0.01	0.01	0.01
query21	15.36	0.92	0.54
query22	0.77	1.15	0.70
query23	14.96	1.39	0.66
query24	12.25	1.12	0.39
query25	0.32	0.10	0.14
query26	0.93	0.18	0.15
query27	0.05	0.05	0.06
query28	6.09	0.77	0.42
query29	12.56	3.91	3.31
query30	0.25	0.09	0.07
query31	2.83	0.60	0.39
query32	3.23	0.55	0.47
query33	2.99	2.98	3.08
query34	15.84	5.22	4.53
query35	4.58	4.55	4.58
query36	0.66	0.49	0.50
query37	0.09	0.07	0.06
query38	0.05	0.05	0.04
query39	0.04	0.03	0.02
query40	0.16	0.13	0.13
query41	0.07	0.02	0.02
query42	0.03	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 106.98 s
Total hot run time: 30.3 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants