-
Notifications
You must be signed in to change notification settings - Fork 4.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read journal entries from all boots #41244
Read journal entries from all boots #41244
Conversation
This pull request is now in conflicts. Could you fix it? 🙏
|
This pull request does not have a backport label.
To fixup this pull request, you need to add the backport labels for the needed
|
|
c6cee3e
to
2c39ac0
Compare
21360eb
to
30c8299
Compare
This pull request is now in conflicts. Could you fix it? 🙏
|
6012cb8
to
1a91677
Compare
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane) |
1a91677
to
e32c6d9
Compare
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments. This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag.
This commit updates the parse test to use ndjson parser instead of multiline because the multiline parser can have issues when journald input is reading from files. There is a corner case where the journalctl exits successfully and the reader goroutine gets an error, this makes Next to return early, making the multiline to also return early. So far I have only seen this happening when reading from file and at the very end of the file, hence it does not seem to be a critical bug.
TestJournaldInput assumed journalctl was run only once, however that has changed recently. It is updated to accommodate for that and rename to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which better reflects what it is actually testing.
2d61a33
to
45c65d0
Compare
This pull request is now in conflicts. Could you fix it? 🙏
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A few linter issues that I think we should fix to have an easy win.
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments. This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag. The parser test is upsted to use ndjson parser instead of multiline because the multiline parser can have issues when journald input is reading from files. There is a corner case where the journalctl exits successfully and the reader goroutine gets an error, this makes Next to return early, making the multiline to also return early. TestJournaldInput assumed journalctl was run only once, however that has changed recently. The test is updated to accommodate for that and rename to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which better reflects what it is actually testing. --------- Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit d21ed32)
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments. This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag. The parser test is upsted to use ndjson parser instead of multiline because the multiline parser can have issues when journald input is reading from files. There is a corner case where the journalctl exits successfully and the reader goroutine gets an error, this makes Next to return early, making the multiline to also return early. TestJournaldInput assumed journalctl was run only once, however that has changed recently. The test is updated to accommodate for that and rename to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which better reflects what it is actually testing. --------- Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit d21ed32)
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments. This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag. The parser test is upsted to use ndjson parser instead of multiline because the multiline parser can have issues when journald input is reading from files. There is a corner case where the journalctl exits successfully and the reader goroutine gets an error, this makes Next to return early, making the multiline to also return early. TestJournaldInput assumed journalctl was run only once, however that has changed recently. The test is updated to accommodate for that and rename to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which better reflects what it is actually testing. --------- Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit d21ed32) Co-authored-by: Tiago Queiroz <[email protected]>
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments. This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag. The parser test is upsted to use ndjson parser instead of multiline because the multiline parser can have issues when journald input is reading from files. There is a corner case where the journalctl exits successfully and the reader goroutine gets an error, this makes Next to return early, making the multiline to also return early. TestJournaldInput assumed journalctl was run only once, however that has changed recently. The test is updated to accommodate for that and rename to TestJournaldInputRunsAndRecoversFromJournalctlFailures, which better reflects what it is actually testing. --------- Co-authored-by: Pierre HILBERT <[email protected]> (cherry picked from commit d21ed32) Co-authored-by: Tiago Queiroz <[email protected]>
Proposed commit message
Some versions of journalctl will only return messages from the current boot when --follow is passed, it will even ignore the cursor or date arguments.
This commit reads messages from all boots by first calling journalctl without the --follow flag, reading all entries and once it successfully exits, then we restart journalctl with the cursor and the --follow flag.
Checklist
I have made corresponding changes to the documentationI have made corresponding change to the default configuration filesCHANGELOG.next.asciidoc
orCHANGELOG-developer.next.asciidoc
.## Disruptive User ImpactAuthor's Checklist
TestInputParsers
fromfilebeat/input/journald/input_parsers_test.go
is not flakyHow to test this PR locally
1. Run the tests
2. Run Filebeat reading
filebeat/input/journald/testdata/multiple-boots.journal
There must be 6 entries, you can see the plaintext entries by looking at
filebeat/input/journald/testdata/multiple-boots.export
or by running:3. Fully manual test
Related issues
## Use cases## Screenshots## Logs