Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](cloud-mow)Modify some mow log to help solve problem #47695

Merged
merged 2 commits into from
Feb 14, 2025

Conversation

hust-hhb
Copy link
Contributor

What problem does this PR solve?

Modify some mow log to help solve problem on getting lock slow or failed
Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hust-hhb
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31455 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 1f87175139c8bb82170619142fcf7384c41fa169, data reload: false

------ Round 1 ----------------------------------
q1	17580	5317	5111	5111
q2	2045	309	175	175
q3	10405	1358	699	699
q4	10210	1032	524	524
q5	7509	2346	2410	2346
q6	185	163	131	131
q7	909	760	600	600
q8	9326	1313	998	998
q9	4946	4735	4692	4692
q10	6814	2298	1886	1886
q11	470	274	266	266
q12	349	352	219	219
q13	17755	3720	3061	3061
q14	230	226	217	217
q15	516	475	463	463
q16	644	620	584	584
q17	557	870	344	344
q18	6697	6296	6213	6213
q19	1078	934	526	526
q20	302	331	195	195
q21	2747	2149	1910	1910
q22	353	321	295	295
Total cold run time: 101627 ms
Total hot run time: 31455 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5127	5159	5151	5151
q2	237	334	226	226
q3	2183	2699	2321	2321
q4	1467	1829	1363	1363
q5	4216	4271	4102	4102
q6	200	159	120	120
q7	1854	1814	1650	1650
q8	2624	2575	2505	2505
q9	7357	7087	7113	7087
q10	2996	3177	2744	2744
q11	583	515	481	481
q12	685	807	620	620
q13	3528	4026	3281	3281
q14	277	282	282	282
q15	518	456	452	452
q16	659	649	629	629
q17	1118	1628	1276	1276
q18	7662	7339	7434	7339
q19	813	815	943	815
q20	1965	2005	1849	1849
q21	5475	4955	4855	4855
q22	645	589	545	545
Total cold run time: 52189 ms
Total hot run time: 49693 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183652 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 1f87175139c8bb82170619142fcf7384c41fa169, data reload: false

query1	965	376	375	375
query2	6516	1898	1913	1898
query3	6794	223	206	206
query4	26392	23634	22984	22984
query5	4892	662	479	479
query6	302	195	191	191
query7	4614	500	312	312
query8	302	243	242	242
query9	8629	2469	2486	2469
query10	497	305	259	259
query11	15496	15461	15009	15009
query12	177	107	106	106
query13	1672	521	378	378
query14	11356	6813	6306	6306
query15	241	186	182	182
query16	7721	627	449	449
query17	1513	720	534	534
query18	1973	395	315	315
query19	239	185	150	150
query20	121	110	112	110
query21	206	122	101	101
query22	4423	4561	4655	4561
query23	34267	33287	32935	32935
query24	6793	2367	2341	2341
query25	502	437	387	387
query26	1205	267	147	147
query27	2157	494	322	322
query28	4072	2398	2379	2379
query29	688	532	407	407
query30	236	185	154	154
query31	928	848	791	791
query32	69	66	62	62
query33	558	351	290	290
query34	777	842	495	495
query35	762	831	730	730
query36	978	977	887	887
query37	118	98	79	79
query38	4215	4210	4147	4147
query39	1442	1426	1411	1411
query40	207	117	103	103
query41	54	49	49	49
query42	119	103	107	103
query43	495	514	476	476
query44	1266	781	774	774
query45	174	169	158	158
query46	855	1046	658	658
query47	1763	1774	1768	1768
query48	368	405	290	290
query49	769	517	415	415
query50	700	732	407	407
query51	4124	4194	4197	4194
query52	104	102	95	95
query53	229	248	179	179
query54	494	504	393	393
query55	79	73	76	73
query56	243	270	248	248
query57	1141	1150	1068	1068
query58	254	233	239	233
query59	2571	2943	2635	2635
query60	287	296	261	261
query61	119	117	123	117
query62	814	760	678	678
query63	222	191	184	184
query64	4237	989	781	781
query65	3224	3120	3147	3120
query66	1059	396	298	298
query67	15803	15750	15671	15671
query68	6540	765	498	498
query69	467	291	265	265
query70	1176	1099	1107	1099
query71	402	304	270	270
query72	5579	3550	3792	3550
query73	740	739	337	337
query74	9234	8859	8760	8760
query75	3170	3167	2658	2658
query76	3253	1186	733	733
query77	487	391	282	282
query78	9918	10200	9265	9265
query79	1623	836	570	570
query80	667	529	446	446
query81	572	272	248	248
query82	215	155	121	121
query83	180	167	161	161
query84	245	94	69	69
query85	735	335	297	297
query86	373	320	279	279
query87	4477	4374	4347	4347
query88	2774	2163	2253	2163
query89	389	314	286	286
query90	1776	191	191	191
query91	135	134	107	107
query92	69	62	52	52
query93	1213	1019	567	567
query94	624	419	298	298
query95	353	259	254	254
query96	478	562	275	275
query97	2775	2828	2728	2728
query98	221	213	201	201
query99	1311	1360	1266	1266
Total cold run time: 268753 ms
Total hot run time: 183652 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 1f87175139c8bb82170619142fcf7384c41fa169, data reload: false

query1	0.04	0.04	0.03
query2	0.08	0.03	0.03
query3	0.23	0.06	0.06
query4	1.63	0.11	0.11
query5	0.40	0.41	0.40
query6	1.17	0.68	0.66
query7	0.02	0.01	0.01
query8	0.04	0.03	0.03
query9	0.58	0.52	0.53
query10	0.58	0.58	0.58
query11	0.15	0.11	0.11
query12	0.15	0.11	0.11
query13	0.62	0.60	0.62
query14	2.70	2.71	2.72
query15	0.92	0.85	0.86
query16	0.37	0.38	0.37
query17	1.03	1.05	1.03
query18	0.21	0.19	0.19
query19	1.91	1.83	1.99
query20	0.02	0.01	0.02
query21	15.35	0.93	0.56
query22	0.75	1.17	0.66
query23	14.95	1.39	0.58
query24	6.79	1.71	0.82
query25	0.52	0.32	0.07
query26	0.54	0.17	0.12
query27	0.05	0.05	0.06
query28	10.13	0.87	0.42
query29	12.53	3.99	3.28
query30	0.25	0.09	0.06
query31	2.82	0.58	0.38
query32	3.23	0.55	0.45
query33	2.96	3.08	3.08
query34	15.78	5.17	4.49
query35	4.58	4.54	4.60
query36	0.65	0.49	0.50
query37	0.10	0.06	0.06
query38	0.05	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.08	0.03	0.03
query42	0.04	0.02	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.24 s
Total hot run time: 30.62 s

zhannngchen
zhannngchen previously approved these changes Feb 11, 2025
Copy link
Contributor

@zhannngchen zhannngchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added approved Indicates a PR has been approved by one committer. reviewed labels Feb 11, 2025
Copy link
Contributor

PR approved by anyone and no changes requested.

dataroaring
dataroaring previously approved these changes Feb 12, 2025
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hust-hhb hust-hhb dismissed stale reviews from dataroaring and zhannngchen via 4808d36 February 12, 2025 04:22
@github-actions github-actions bot removed the approved Indicates a PR has been approved by one committer. label Feb 12, 2025
@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 12, 2025
Copy link
Contributor

PR approved by at least one committer and no changes requested.

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhannngchen
Copy link
Contributor

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31573 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 4808d36592c4a6f79083866761667bd5fee9226a, data reload: false

------ Round 1 ----------------------------------
q1	17564	5198	5026	5026
q2	2046	302	168	168
q3	10405	1256	767	767
q4	10218	1024	538	538
q5	7559	2330	2353	2330
q6	181	167	134	134
q7	881	753	596	596
q8	9298	1288	1048	1048
q9	4814	4725	4609	4609
q10	6823	2313	1876	1876
q11	468	278	262	262
q12	344	343	220	220
q13	17761	3698	3074	3074
q14	229	220	213	213
q15	503	463	449	449
q16	627	632	591	591
q17	583	874	345	345
q18	6977	6328	6346	6328
q19	1429	954	553	553
q20	319	330	188	188
q21	2789	2146	1965	1965
q22	365	331	293	293
Total cold run time: 102183 ms
Total hot run time: 31573 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5072	5083	5084	5083
q2	242	321	230	230
q3	2129	2673	2253	2253
q4	1392	1807	1352	1352
q5	4249	4170	4201	4170
q6	204	170	124	124
q7	1859	1809	1640	1640
q8	2587	2731	2574	2574
q9	7188	7177	7132	7132
q10	3004	3192	2758	2758
q11	568	537	508	508
q12	684	784	610	610
q13	3521	3718	3324	3324
q14	281	301	264	264
q15	516	471	467	467
q16	667	666	630	630
q17	1119	1634	1313	1313
q18	7598	7298	7307	7298
q19	801	831	832	831
q20	1940	1983	1878	1878
q21	5470	4975	4910	4910
q22	640	600	553	553
Total cold run time: 51731 ms
Total hot run time: 49902 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 189989 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 4808d36592c4a6f79083866761667bd5fee9226a, data reload: false

query1	1327	973	933	933
query2	6254	1809	1804	1804
query3	10951	4514	4347	4347
query4	53214	25425	22926	22926
query5	5160	575	481	481
query6	337	211	183	183
query7	4879	505	291	291
query8	314	244	235	235
query9	5862	2591	2600	2591
query10	434	305	257	257
query11	15177	15054	14975	14975
query12	164	108	103	103
query13	1045	534	413	413
query14	10541	6864	6665	6665
query15	201	196	207	196
query16	6981	648	468	468
query17	1054	719	572	572
query18	1358	408	305	305
query19	188	195	165	165
query20	122	151	122	122
query21	207	123	108	108
query22	4318	4630	4431	4431
query23	33934	33466	33564	33466
query24	5788	2406	2405	2405
query25	447	465	400	400
query26	663	279	153	153
query27	1795	498	343	343
query28	2795	2502	2444	2444
query29	586	589	456	456
query30	215	196	155	155
query31	904	854	818	818
query32	75	64	62	62
query33	472	355	331	331
query34	794	858	507	507
query35	806	829	754	754
query36	946	1010	939	939
query37	128	104	76	76
query38	4372	4251	4179	4179
query39	1519	1444	1454	1444
query40	217	120	111	111
query41	55	61	53	53
query42	123	116	114	114
query43	497	497	504	497
query44	1343	819	828	819
query45	188	183	213	183
query46	909	1083	654	654
query47	1839	1847	1744	1744
query48	402	424	330	330
query49	709	527	437	437
query50	758	761	425	425
query51	4227	4270	4179	4179
query52	112	111	96	96
query53	250	269	188	188
query54	484	492	431	431
query55	82	88	80	80
query56	276	261	261	261
query57	1150	1202	1131	1131
query58	247	257	286	257
query59	2789	2702	2642	2642
query60	296	288	263	263
query61	120	123	122	122
query62	753	708	692	692
query63	233	199	193	193
query64	1496	1035	673	673
query65	3211	3153	3108	3108
query66	804	388	295	295
query67	15870	15744	15535	15535
query68	5582	777	519	519
query69	538	293	268	268
query70	1251	1137	1105	1105
query71	470	302	263	263
query72	6276	3659	3995	3659
query73	1270	775	349	349
query74	8929	9071	8795	8795
query75	3174	3179	2681	2681
query76	3925	1174	754	754
query77	603	368	276	276
query78	10066	10022	9330	9330
query79	2460	815	586	586
query80	650	551	432	432
query81	494	277	237	237
query82	237	127	92	92
query83	180	177	150	150
query84	304	97	78	78
query85	798	350	322	322
query86	358	277	282	277
query87	4550	4571	4386	4386
query88	3393	2221	2204	2204
query89	396	322	281	281
query90	1730	195	194	194
query91	135	131	116	116
query92	71	59	57	57
query93	2325	1020	575	575
query94	679	401	290	290
query95	353	277	257	257
query96	484	561	267	267
query97	2758	2825	2710	2710
query98	229	205	203	203
query99	1288	1377	1260	1260
Total cold run time: 291723 ms
Total hot run time: 189989 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.24 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4808d36592c4a6f79083866761667bd5fee9226a, data reload: false

query1	0.03	0.03	0.02
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.62	0.10	0.11
query5	0.41	0.41	0.40
query6	1.18	0.64	0.66
query7	0.02	0.02	0.01
query8	0.04	0.03	0.04
query9	0.59	0.52	0.52
query10	0.58	0.57	0.56
query11	0.15	0.10	0.10
query12	0.15	0.12	0.11
query13	0.62	0.60	0.59
query14	2.79	2.69	2.79
query15	0.92	0.86	0.86
query16	0.38	0.36	0.39
query17	1.04	1.02	1.05
query18	0.22	0.20	0.20
query19	1.89	1.89	1.97
query20	0.01	0.01	0.02
query21	15.36	0.90	0.55
query22	0.77	1.32	0.70
query23	14.74	1.40	0.65
query24	9.71	1.20	0.36
query25	0.30	0.15	0.13
query26	0.68	0.18	0.14
query27	0.05	0.04	0.06
query28	7.91	0.88	0.45
query29	12.53	3.96	3.26
query30	0.25	0.08	0.07
query31	2.84	0.60	0.40
query32	3.23	0.55	0.46
query33	2.97	3.11	3.02
query34	15.84	5.17	4.46
query35	4.52	4.53	4.55
query36	0.66	0.49	0.50
query37	0.09	0.07	0.06
query38	0.05	0.04	0.04
query39	0.03	0.03	0.03
query40	0.17	0.14	0.13
query41	0.08	0.02	0.02
query42	0.03	0.02	0.02
query43	0.04	0.03	0.03
Total cold run time: 105.8 s
Total hot run time: 30.24 s

@zhannngchen zhannngchen merged commit 4ee4ec4 into apache:master Feb 14, 2025
29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.x dev/3.0.x-conflict reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants