Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve](move-memtable) add log on idle timeout #29438

Merged
merged 1 commit into from
Jan 3, 2024

Conversation

kaijchen
Copy link
Contributor

@kaijchen kaijchen commented Jan 2, 2024

Proposed changes

Add a log when LoadStream is closing stream on idle timeout.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@kaijchen
Copy link
Contributor Author

kaijchen commented Jan 2, 2024

run buildall

Copy link
Contributor

github-actions bot commented Jan 2, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit 0b6fb2b8cb3d57d28fdfa8430f5cc4c61159950c, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5552	5159	5167	5159
q2	393	162	168	162
q3	1491	1200	1135	1135
q4	1131	854	860	854
q5	3186	3112	3150	3112
q6	231	143	131	131
q7	1022	553	523	523
q8	2186	2224	2225	2224
q9	6734	6766	6688	6688
q10	3164	3139	3146	3139
q11	354	215	215	215
q12	388	238	244	238
q13	4461	3677	3644	3644
q14	255	219	229	219
q15	605	556	535	535
q16	469	403	417	403
q17	1060	521	484	484
q18	7141	6722	6804	6722
q19	1662	1493	1466	1466
q20	566	363	329	329
q21	2899	2574	2492	2492
q22	404	319	332	319
Total cold run time: 45354 ms
Total hot run time: 40193 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5173	5062	5113	5062
q2	342	264	245	245
q3	3402	3336	3309	3309
q4	2165	2064	1995	1995
q5	5968	5936	5945	5936
q6	231	125	123	123
q7	2414	1924	1947	1924
q8	3576	3690	3668	3668
q9	9036	9079	9074	9074
q10	3921	3939	3920	3920
q11	586	519	480	480
q12	803	667	657	657
q13	3908	3252	3217	3217
q14	304	272	282	272
q15	625	551	534	534
q16	557	490	512	490
q17	2009	1847	1841	1841
q18	8763	8437	8435	8435
q19	1766	1679	1696	1679
q20	2298	2045	1994	1994
q21	5845	5455	5515	5455
q22	580	513	490	490
Total cold run time: 64272 ms
Total hot run time: 60800 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.18 seconds
stream load tsv: 562 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 18 seconds loaded 2358488459 Bytes, about 124 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.1 seconds inserted 10000000 Rows, about 355K ops/s
storage size: 17183910852 Bytes

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.62% (8615/23524)
Line Coverage: 28.68% (70014/244142)
Region Coverage: 27.66% (36233/131016)
Branch Coverage: 24.36% (18519/76010)
Coverage Report: http://coverage.selectdb-in.cc/coverage/0b6fb2b8cb3d57d28fdfa8430f5cc4c61159950c_0b6fb2b8cb3d57d28fdfa8430f5cc4c61159950c/report/index.html

Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

github-actions bot commented Jan 3, 2024

PR approved by at least one committer and no changes requested.

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 3, 2024
Copy link
Contributor

github-actions bot commented Jan 3, 2024

PR approved by anyone and no changes requested.

Copy link
Contributor

@sollhui sollhui left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dataroaring dataroaring merged commit 1e8bb75 into apache:master Jan 3, 2024
27 of 29 checks passed
seawinde pushed a commit to seawinde/doris that referenced this pull request Jan 3, 2024
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants