Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Java: Changed handling of large requests to transfer them as leaked pointers #404

Closed
wants to merge 72 commits into from

Conversation

jduo
Copy link

@jduo jduo commented Jun 28, 2024

No description provided.

aaron-congo and others added 30 commits June 21, 2024 21:47
* Python: add XREVRANGE command

* Update doc for xrevrange

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update transaction docs

Signed-off-by: Andrew Carbonetto <[email protected]>

---------

Signed-off-by: Andrew Carbonetto <[email protected]>
Co-authored-by: Andrew Carbonetto <[email protected]>
* Java: Add `FUNCTION DUMP` and `FUNCTION RESTORE`. (#370)

* Add `FUNCTION DUMP` and `FUNCTION RESTORE` implementations.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Address PR comments.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Add tests.

Signed-off-by: Yury-Fridlyand <[email protected]>

* Address PR comments.

Signed-off-by: Yury-Fridlyand <[email protected]>

---------

Signed-off-by: Yury-Fridlyand <[email protected]>

* Use GlideString

Signed-off-by: Andrew Carbonetto <[email protected]>

* Clean up FUNCTION DUMP & RESTORE

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update handlers

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update comments

Signed-off-by: Andrew Carbonetto <[email protected]>

* Add cluster IT test

Signed-off-by: Andrew Carbonetto <[email protected]>

* quick review comment

Signed-off-by: Andrew Carbonetto <[email protected]>

* SPOTLESS

Signed-off-by: Andrew Carbonetto <[email protected]>

---------

Signed-off-by: Yury-Fridlyand <[email protected]>
Signed-off-by: Andrew Carbonetto <[email protected]>
Co-authored-by: Yury-Fridlyand <[email protected]>
support bitcount with GlideString
support bitop, bitpos and xdel with GlideString
valkey-io#1630)

support hincrby, hincrbyfloat, incrby and incrbyfloat with GlideString
support llen, strlen, xlen and hstrlen with GlideString
valkey-io#1634)

support zintercard, zdiffstore, zremrangebyrank, lpush, lpushx, lrem, rpus and rpushx with GlideString
* Python: add XREAD command

* Minor doc fix

* PR suggestions
…valkey-io#1636)

* support geopos, zmscore, persist, rename and renamex with GlideString

* add to integration tests the use of the API with GlideString parameters

* remove calls to getBytes()
…s due to disconnect notifications. 2. Typos fixes
Pubsub fixups: 1. Logging instead of exceptions on the pending futures due to disconnect notifications. 2. Typos fixes
…lkey-io#1658)

* Python: add XGROUP CREATECONSUMER and XGROUP DELCONSUMER commands

* Rename consumer variable to consumer_name
* Python: Add LOLWUT command (#387)

Added Python LOLWUT command

* Updated CHANGELOG.md

* Fixed formated issues

---------

Co-authored-by: Andrew Carbonetto <[email protected]>
Java: Fix flaky IT. (#390)

Signed-off-by: Yury-Fridlyand <[email protected]>
valkey-io#1642)

* support objectEncoding, objectFreq, objectIdletime and objectRefcount with GlideString

* add to integration tests the use of the API with GlideString parameters

* add binary version integration tests

* nit: spotlessApply

* fix use of GlideString in objectRefcount_returns_null

---------

Co-authored-by: Ubuntu <[email protected]>
…Time, pexpireTime, exists (valkey-io#1637)

* GS version of ttl, pttl, expire, pexpire, expireAt, pexpireAt, expireTime, pexpireTime, exists

Co-authored-by: Ubuntu <[email protected]>
…1635)

* support geodist, getbit, setbit and xack with GlideString

* add to integration tests the use of the API with GlideString parameters

* nit: apply spotlessApply

* remove calls to getBytes()
…lkey-io#1638)

* support move, ltrim, sadd, srem and smove with GlideString

* add to integration tests the use of the API with GlideString parameters

* remove calls to getBytes()

* nit: apply spotlessApply

* add binary and not binary integration test for smove

* add integration tests with GlideString version

* nit: spotlessApply

* remove the use of gs() in ltrim_existing_non_existing_key_and_type_error

* add binary version integration tests
* Java: Add `LCS` command (with IDX option) (#386)

* Implemented LCS with IDX

* TODO: add docs and more integTests

* Added docs and remaining tests

* Addressed comments

* Fixed rust formatting

* Addressed comments

* Added WITHMATCHLEN apis

* Expanded on example

* Fixed rust ci failure

* Removed LcsOptions

* Improved examples in docs

* Examples with different match lengths

* Throw NPE if matches is not present

* Resolved conflicts
* Python: add XREADGROUP command

* Fix mypy error

* PR suggestions
* Java: Add XPENDING command (#389)

* Java: Add XPENDING command

Signed-off-by: Andrew Carbonetto <[email protected]>

* Add @see doc

Signed-off-by: Andrew Carbonetto <[email protected]>

* Add @see doc

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update constant name

Signed-off-by: Andrew Carbonetto <[email protected]>

* Make tests more robust

Signed-off-by: Andrew Carbonetto <[email protected]>

* Add UT tests

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update docs for comments

Signed-off-by: Andrew Carbonetto <[email protected]>

---------

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update for review coments

Signed-off-by: Andrew Carbonetto <[email protected]>

* Fix merge conflicts

Signed-off-by: Andrew Carbonetto <[email protected]>

* Remove file

Signed-off-by: Andrew Carbonetto <[email protected]>

* Update options to point to valkey

Signed-off-by: Andrew Carbonetto <[email protected]>

---------

Signed-off-by: Andrew Carbonetto <[email protected]>
* Python: add XACK command

* PR suggestions
@jduo jduo force-pushed the use_args_ptr_java branch from 9b90f69 to 08f7ca7 Compare June 28, 2024 22:31

return commandArgs.build();
/** Dummy function for taking a series of String parameters and returning an String array */
private static String[] buildArgs(String... args) {
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I replaced the existing buildArgs() function with this dummy one that takes it's inputs and returns a String[] to make it easier to mass-update all 200+ functions.

It would be better to just inline parameters but that turned out to be more manual and probably harder to merge.

@jduo
Copy link
Author

jduo commented Jun 28, 2024

I tested this by manually editing the automated tests to send 512MB keys and values and it worked correctly. Note that I needed to increase the test runner's JVM heap space and increase timeouts on both the tests and the test client configurations to get this to run correctly.

TJ Zhang and others added 24 commits June 28, 2024 18:09
…arch

Java: Add Command GeoSearch & GeoSearchStore
* Restructure Java FFI layer to handle errors properly

* Fix failing tests

* Address clippy lints

* Add tests for error and panic handling

* Add missing errors module

* Fix clippy lint

* Fix FFI tests

* Apply Spotless

* Fix some minor issue I forgot about

* Add some comments

* Apply Spotless

* Make handle_panics return Option<T> instead
* Java: Add `SSCAN` command (#394)

* Add ScanOptions base class for scan-family options.
* Expose the cursor as a String to support unsigned 64-bit cursor values.

Co-authored-by: James Duong <[email protected]>

* Java: Add `ZSCAN` command (#397)

---------

Co-authored-by: James Duong <[email protected]>

* WIP TODO: support transactions, docs, and more IT

* Added more tests

* Added tests and javadocs

* Improved examples and tests

* Correct use of SScanOptions instead of ScanOptions for SScan

* Remove plumbing for SCAN command

* Sleep after sadd() calls before sscan() calls

Due to eventual consistency

* Change sscan cursor to be a String

Also fix bug in SharedCommandTests

* WIP with todos

# Conflicts:
#	glide-core/src/protobuf/redis_request.proto
#	glide-core/src/request_type.rs
#	java/client/src/main/java/glide/api/commands/SortedSetBaseCommands.java

* Add ZScan to TransactionTestUtilities

* Spotless cleanup

* Test fixes

* Cleanup test code

* Apply IntelliJ suggestions
* Use String.valueOf() instead of concatenating empty string

* Added better error info for set comparison failures

* More logging for test failures

* Add sleeps after zadd() calls

To help make sure data is consistent without WAIT

* Longer sleeps

* Reduce wait time

* Experiment with unsigned 64-bit cursors

* Fix rebase error

* WIP TODO: support transactions, docs, and more IT

* Added more tests

* Added tests and javadocs

* Improved examples and tests

* Apply PR comments

* Fix method ordering in BaseTransaction
* Fix broken line breaks within code tags in ScanOptions
* More thoroughly test results in SharedCommandTests

* Add better logging for set comparisons

* Spotless

* Sleep after sadd() calls before sscan() calls

Due to eventual consistency

* Change sscan cursor to be a String

Also fix bug in SharedCommandTests

* Update java/integTest/src/test/java/glide/SharedCommandTests.java

Co-authored-by: Guian Gumpac <[email protected]>

* Update java/integTest/src/test/java/glide/SharedCommandTests.java

Co-authored-by: Guian Gumpac <[email protected]>

* Fix rebase conflicts

* Fix another rebase conflict

* Spotless

* Update java/client/src/main/java/glide/api/models/BaseTransaction.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/models/BaseTransaction.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/models/BaseTransaction.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/models/BaseTransaction.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Correctly use constants in TransactionTests

* Rename ScanOptions to BaseScanOptions

* Doc PR fixes

* Treat end of cursor as failure

* Spotless

* Fixes

* Update java/client/src/main/java/glide/api/commands/SortedSetBaseCommands.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/commands/SortedSetBaseCommands.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/commands/SortedSetBaseCommands.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Update java/client/src/main/java/glide/api/commands/SortedSetBaseCommands.java

Co-authored-by: Andrew Carbonetto <[email protected]>

* Minor doc changes

---------

Co-authored-by: Guian Gumpac <[email protected]>
Co-authored-by: Andrew Carbonetto <[email protected]>
Due to using a JDK17 function
@jduo jduo force-pushed the use_args_ptr_java branch from df37f40 to aa13ce2 Compare June 29, 2024 19:26
@GumpacG
Copy link

GumpacG commented Jun 29, 2024

Closing as valkey-io#1709 is already created.

@GumpacG GumpacG closed this Jun 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.