Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improvement](statistics)Skip auto analyze when table row count is not fully reported. (return -1) (#42209) #42405

Merged
merged 1 commit into from
Oct 24, 2024

Conversation

Jibing-Li
Copy link
Contributor

backport: #42209

…t fully reported. (return -1) (apache#42209)

Skip auto analyze when table row count is not fully reported. Not fully
reported means row count is -1.
@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

Since 2024-03-18, the Document has been moved to doris-website.
See Doris Document.

@Jibing-Li Jibing-Li marked this pull request as ready for review October 24, 2024 09:13
@Jibing-Li
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 49048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit d3595215b10594a27fef78e60b8e312ec3cc37d2, data reload: false

------ Round 1 ----------------------------------
q1	17783	4400	4384	4384
q2	2063	160	157	157
q3	10242	1907	1919	1907
q4	10104	1275	1323	1275
q5	8420	3893	3904	3893
q6	238	123	125	123
q7	2048	1634	1607	1607
q8	9513	2748	2730	2730
q9	11469	9859	9660	9660
q10	8631	3533	3527	3527
q11	425	260	239	239
q12	482	302	304	302
q13	18366	3949	4055	3949
q14	359	326	338	326
q15	508	465	456	456
q16	558	463	468	463
q17	1125	983	899	899
q18	7283	6827	6881	6827
q19	1703	1576	1531	1531
q20	546	300	297	297
q21	4446	4161	4101	4101
q22	494	395	399	395
Total cold run time: 116806 ms
Total hot run time: 49048 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4340	4292	4301	4292
q2	327	228	226	226
q3	4170	4148	4101	4101
q4	2769	2736	2746	2736
q5	7195	7159	7113	7113
q6	242	126	126	126
q7	3211	2867	2932	2867
q8	4407	4465	4475	4465
q9	13683	13655	13715	13655
q10	4198	4268	4253	4253
q11	749	688	689	688
q12	1038	861	859	859
q13	6866	3735	3741	3735
q14	454	434	422	422
q15	511	453	465	453
q16	623	606	598	598
q17	3812	3833	3871	3833
q18	8900	8723	8816	8723
q19	1712	1683	1659	1659
q20	2382	2128	2103	2103
q21	8461	8555	8435	8435
q22	1028	949	924	924
Total cold run time: 81078 ms
Total hot run time: 76266 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 211101 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit d3595215b10594a27fef78e60b8e312ec3cc37d2, data reload: false

query1	936	391	411	391
query2	6530	2216	2154	2154
query3	6948	203	209	203
query4	23019	21474	21566	21474
query5	19737	6532	6531	6531
query6	296	229	240	229
query7	4339	301	308	301
query8	248	235	236	235
query9	3083	2642	2576	2576
query10	469	303	297	297
query11	15587	15183	15035	15035
query12	132	77	78	77
query13	1036	462	423	423
query14	17062	13231	13931	13231
query15	373	223	230	223
query16	6475	285	263	263
query17	1789	947	890	890
query18	888	324	312	312
query19	218	155	150	150
query20	105	101	95	95
query21	188	96	93	93
query22	5347	4992	5074	4992
query23	34227	33397	33420	33397
query24	7864	6258	6353	6258
query25	535	430	421	421
query26	1267	174	166	166
query27	2441	297	293	293
query28	6081	2241	2218	2218
query29	3035	2782	2754	2754
query30	244	168	164	164
query31	963	747	758	747
query32	73	59	58	58
query33	463	251	266	251
query34	856	468	475	468
query35	1135	938	864	864
query36	1336	1236	1087	1087
query37	174	62	59	59
query38	3076	2909	2950	2909
query39	1382	1320	1319	1319
query40	312	98	94	94
query41	39	38	37	37
query42	91	88	81	81
query43	769	541	583	541
query44	1154	733	728	728
query45	241	233	227	227
query46	1248	976	966	966
query47	1700	1896	1727	1727
query48	506	410	398	398
query49	658	384	382	382
query50	860	673	609	609
query51	4785	4642	4656	4642
query52	93	83	78	78
query53	228	183	185	183
query54	2680	2504	2486	2486
query55	82	83	85	83
query56	214	209	214	209
query57	1284	1288	1261	1261
query58	216	213	196	196
query59	3360	3028	3181	3028
query60	219	204	198	198
query61	96	93	93	93
query62	751	469	425	425
query63	196	176	180	176
query64	3513	1558	1440	1440
query65	3643	3535	3551	3535
query66	776	418	411	411
query67	15791	14851	15443	14851
query68	11239	643	647	643
query69	519	282	270	270
query70	2043	1323	1320	1320
query71	409	298	313	298
query72	6912	4582	4757	4582
query73	778	317	314	314
query74	6397	5774	5885	5774
query75	5344	3694	3774	3694
query76	6963	1188	1201	1188
query77	1178	257	263	257
query78	12469	11758	11580	11580
query79	8133	659	634	634
query80	1327	410	401	401
query81	468	240	237	237
query82	746	100	100	100
query83	166	136	131	131
query84	265	72	73	72
query85	886	318	319	318
query86	340	291	285	285
query87	3195	3028	3025	3025
query88	3787	2285	2279	2279
query89	456	291	303	291
query90	1932	215	219	215
query91	161	135	123	123
query92	57	52	51	51
query93	6618	538	607	538
query94	742	209	208	208
query95	1977	1962	2024	1962
query96	665	327	324	324
query97	6511	6283	6365	6283
query98	219	207	213	207
query99	3102	948	804	804
Total cold run time: 321466 ms
Total hot run time: 211101 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.4 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit d3595215b10594a27fef78e60b8e312ec3cc37d2, data reload: false

query1	0.02	0.03	0.02
query2	0.07	0.02	0.02
query3	0.24	0.04	0.06
query4	1.78	0.07	0.08
query5	0.53	0.52	0.53
query6	1.25	0.61	0.60
query7	0.01	0.01	0.01
query8	0.03	0.02	0.03
query9	0.51	0.48	0.47
query10	0.53	0.53	0.52
query11	0.12	0.09	0.09
query12	0.12	0.09	0.09
query13	0.62	0.62	0.60
query14	0.76	0.80	0.81
query15	0.79	0.77	0.78
query16	0.38	0.36	0.36
query17	0.99	1.03	0.98
query18	0.23	0.23	0.25
query19	1.95	1.84	1.84
query20	0.01	0.01	0.01
query21	15.46	0.59	0.55
query22	2.15	2.51	1.18
query23	17.30	0.85	0.87
query24	5.04	1.26	1.49
query25	0.36	0.11	0.06
query26	0.63	0.16	0.15
query27	0.04	0.04	0.03
query28	7.33	0.76	0.70
query29	12.63	2.30	2.27
query30	0.56	0.50	0.51
query31	2.82	0.38	0.37
query32	3.40	0.50	0.49
query33	3.05	3.10	3.13
query34	15.25	4.80	4.77
query35	4.84	4.83	4.82
query36	1.05	1.00	1.03
query37	0.06	0.05	0.05
query38	0.03	0.03	0.02
query39	0.02	0.01	0.02
query40	0.15	0.14	0.13
query41	0.06	0.02	0.01
query42	0.02	0.02	0.01
query43	0.02	0.02	0.01
Total cold run time: 103.21 s
Total hot run time: 30.4 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit d3595215b10594a27fef78e60b8e312ec3cc37d2 with default session variables
Stream load json:         19 seconds loaded 2358488459 Bytes, about 118 MB/s
Stream load orc:          58 seconds loaded 1101869774 Bytes, about 18 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       21.4 seconds inserted 10000000 Rows, about 467K ops/s

@Jibing-Li Jibing-Li merged commit 851477e into apache:branch-2.0 Oct 24, 2024
22 of 24 checks passed
@Jibing-Li Jibing-Li deleted the nosample2.0 branch October 24, 2024 10:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants