-
Notifications
You must be signed in to change notification settings - Fork 3
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Luc Perkins <[email protected]>
- Loading branch information
Luc Perkins
committed
Oct 18, 2021
1 parent
6acdb5f
commit 8ae0764
Showing
1 changed file
with
12 additions
and
8 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,20 +1,24 @@ | ||
# CSV enrichment in Vector | ||
|
||
**Note:** This feature is unreleased and still under active development. However, if you want to try it out, we would | ||
love any feedback! | ||
> **Note:** This feature is unreleased and still under active development. However, if you want to try it out, we would | ||
> love any feedback! | ||
## How to start it | ||
|
||
`docker-compose up` | ||
```shell | ||
docker-compose up | ||
``` | ||
|
||
## What it does | ||
|
||
This demo shows an example of using CSV enrichment in Vector to enrich events with data from a CSV file. | ||
This demo shows an example of using CSV enrichment in Vector to enrich events with data from two CSV files. | ||
|
||
The CSV file ([`./data/users.csv`](./data/users.csv) contains a list of users with their phone numbers and addresses. | ||
The [`users.csv`](./data/users.csv) file contains a list of users with their phone numbers and addresses, while the | ||
[`codes.csv`](./data/codes.csv) file contains a list of common Unix exit codes (with code, tag, and message info). | ||
|
||
The `vector.toml` configuration contains: | ||
The [`vector.toml`](./vector.toml) configuration contains: | ||
|
||
* A `random` source in the `vector.toml` that simply generates some fake events using names that match users in the CSV file | ||
* A `remap` transform then parses these events and then looks up the corresponding record from the CSV file to enrich the event with more metadata | ||
* Two `generator` sources in the that generate fake events that can be enriched using the CSV files (user data and coded error messages | ||
respectively) | ||
* Two `remap` transforms that parse the events and then look up the corresponding records from the CSV files to enrich the events with more metadata | ||
* A `console` sink to print out the events |