Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sink): support geospatial for redis sink #20402

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Conversation

xxhZs
Copy link
Contributor

@xxhZs xxhZs commented Feb 6, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

in this pr, we support geospatial type for redis sink
https://redis.io/glossary/geospatial-indexing/

Checklist

  • I have written necessary rustdoc comments.
  • I have added necessary unit tests and integration tests.
  • I have added test labels as necessary.
  • I have added fuzzing tests or opened an issue to track them.
  • My PR contains breaking changes.
  • My PR changes performance-critical code, so I will run (micro) benchmarks and present the results.
  • My PR contains critical fixes that are necessary to be merged into the latest release.

Documentation

Release note

We can use geospatial like

create table t1(v1 int, v2 int ,v3 varchar, v4 varchar);
CREATE SINK s1
FROM
    t1 WITH (
    primary_key = 'v3',
    connector = 'redis',
    redis.url= 'redis://127.0.0.1:6379/',
)FORMAT UPSERT ENCODE TEMPLATE
(redis_value_type ='geospatial', 
longitude = 'v1', 
latitude = 'v2', 
member = 'v3',
key_format = 'abcd3:{v4}'
);
insert into t1 values(1.1, 1.1, 'test','test');

then we can get it in redis like

GEOPOS abcd3:test test
------
1) 1) "1.10000044298171997"
   2) "1.10000000482478555"

redis_value_type must be set to string and geospatial, default is string
if redis_value_type is set to string, we must set key_format and value_fomat(Consistent with previous behavior)
if redis_value_type is set to geospatial, we must set key_format, longitude, latitude and member
The meaning of key_format is consistent with redis_value_type = string
The longitude is the column name of rw, the datatype must be float, real or varchar
The latitude is the column name of rw, the datatype must be float, real or varchar
The member is the column name of rw, the datatype must be varchar, and must be primary_key

@xxhZs xxhZs requested a review from a team as a code owner February 6, 2025 08:01
@xxhZs xxhZs requested a review from fuyufjh February 6, 2025 08:01
Copy link
Member

@fuyufjh fuyufjh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

pub fn encode_cols(
&self,
row: impl Row,
_col_indices: impl Iterator<Item = usize>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps off-the-topic but the trait function is confusing to me: why col_indices need to be passed from outside, while it's stored in self? 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants