Skip to content

Commit

Permalink
AUTO: Sync ScalarDB docs in English to docs site repo
Browse files Browse the repository at this point in the history
  • Loading branch information
josh-wong committed Feb 10, 2025
1 parent 3a567b3 commit 81c8be6
Showing 1 changed file with 39 additions and 25 deletions.
64 changes: 39 additions & 25 deletions versioned_docs/version-3.12/roadmap.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -17,34 +17,22 @@ If you have a feature request or want to prioritize feature development, please

:::

### CY2024 Q4
### CY2025 Q1

#### New capabilities

- **Data virtualization for analytics**
- Users will be able to run read-only OLAP SQL queries on diverse data sources through ScalarDB Analytics. ScalarDB Analytics currently supports only ScalarDB-managed data stores, so this enhancement will virtually unify various data stores, like relational databases and NoSQL databases, and files in cloud object stores, like Amazon S3, without regard to whether the data sources are managed by ScalarDB transactions.
- **Vector store abstraction**
- Users will be able to store and search embeddings (vectors) in and from vector stores through a new vector store interface in ScalarDB. With this feature, users can simplify the process of realizing retrieval-augmented generation (RAG) with large language models (LLMs) by reading data from databases through the existing ScalarDB interface, creating embeddings from the data, and storing and searching the embeddings to and from a vector store through the new interface.

#### Security

- **Fine-grained access control**
- Users will be able to authorize accesses to the underlying databases in a finer-grained way. In addition to the current simple authorization where ScalarDB checks if users are authorized to issue particular operations, ScalarDB will check if users can access particular records.

#### Usability

- **Addition of time-related data types**
- Users will be able to use time-related data types, which will make their existing applications easier to migrate.
- **Removal of extra-write strategy**
- Users will no longer be able to use the extra-write strategy to make transactions serializable. Although ScalarDB currently provides two strategies, extra-read and extra-write strategies, to make transactions serializable, the extra-write strategy has several limitations. For example, users can't issue write and scan operations in the same transaction. Therefore, the strategy will be removed so that users don't need to worry about such limitations when creating applications.

#### Performance

- **One-phase commit optimization**
- Users will experience faster execution for simple transactions that write to a single partition. ScalarDB will omit the prepare-record and commit-state phases without sacrificing correctness if a transaction updates only one partition by exploiting the single-partition linearizable operations of the underlying databases.
- **Reduction of storage space needed for managing ScalarDB metadata**
- Users will likely use less storage space to run ScalarDB. ScalarDB will remove the before image of committed transactions after they are committed. However, whether or not those committed transactions will impact actual storage space depends on the underlying databases.
- **Removal of coordinator writes for read-only transactions**
- Users will experience faster execution for read-only transactions by removing coordinator writes for those transactions.

#### Cloud support

Expand All @@ -53,37 +41,63 @@ If you have a feature request or want to prioritize feature development, please
- **Google Cloud Platform (GCP) support**
- Users will be able to deploy ScalarDB Cluster in Google Kubernetes Engine (GKE) in GCP.

### CY2025 Q1
### CY2025 Q2

#### New capabilities

- **Native secondary index**
- Users will be able to define flexible secondary indexes. The existing secondary index is limited because it is implemented based on the common capabilities of the supported databases' secondary indexes. Therefore, for example, you cannot define a multi-column index. The new secondary index will be created at the ScalarDB layer so that you can create more flexible indexes, like a multi-column index.

#### Support for additional databases

- **Databricks**
- Users will be able to use Databricks as an underlying database through ScalarDB Cluster.
- **Snowflake**
- Users will be able to use Snowflake as an underlying database through ScalarDB Cluster.

#### Usability

- **Addition of SQL operations for aggregation**
- Users will be able to issue aggregation operations in ScalarDB SQL.
- **Addition of decimal data types**
- Users will be able to use decimal data types so that users can handle decimal numbers with high precision.
- **Removal of extra-write strategy**
- Users will no longer be able to use the extra-write strategy to make transactions serializable. Although ScalarDB currently provides two strategies (extra-read and extra-write strategies) to make transactions serializable, the extra-write strategy has several limitations. For example, users can't issue write and scan operations in the same transaction. Therefore, the strategy will be removed so that users don't need to worry about such limitations when creating applications.
- **Better governance in ScalarDB Analytics**
- Users will be able to be authenticated and authorized by using the ScalarDB Core features.

- **Elimination of out-of-memory errors due to large scans**
- Users will be able to issue large scans without experiencing out-of-memory errors.
- **Enabling of read operations during a paused duration**
- Users will be able to issue read operations even during a paused duration so that users can still read data while taking backups.
- **Addition of more data types**
- Users will be able to use more data types so that their existing applications will be easier to migrate.
#### Performance

### CY2025 Q2 -
- **Removal of WAL-interpreted views in ScalarDB Analytics**
- Users will be able to read committed data by using ScalarDB Core instead of WAL-interpreted views.

### CY2025 Q3

#### Usability

- **Views**
- Users will be able to define Views so that they can manage multiple different databases in an easier and simplified way.
- Users will be able to define views so that they can manage multiple different databases in an easier and simplified way.
- **Addition of SQL operations for aggregation**
- Users will be able to issue aggregation operations in ScalarDB SQL.
- **Elimination of out-of-memory errors due to large scans**
- Users will be able to issue large scans without experiencing out-of-memory errors.
- **Enabling of read operations during a paused duration**
- Users will be able to issue read operations even during a paused duration so that users can still read data while taking backups.

#### Scalability and availability

- **Semi-synchronous replication**
- Users will be able to provide ScalarDB-based applications in a disaster-recoverable manner. For example, assume you provide a primary service in Tokyo and a standby service in Osaka. In case of catastrophic failure in Tokyo, you can switch the primary service to Osaka so that you can continue to provide the service without data loss and extended downtime.

### CY2025 Q4

#### Performance

- **One-phase commit optimization**
- Users will experience faster execution for simple transactions that write to a single partition. ScalarDB will omit the prepare-record and commit-state phases without sacrificing correctness if a transaction updates only one partition by exploiting the single-partition linearizable operations of the underlying databases.
- **Reduction of storage space needed for managing ScalarDB metadata**
- Users will likely use less storage space to run ScalarDB. ScalarDB will remove the before image of committed transactions after they are committed. However, whether or not those committed transactions will impact actual storage space depends on the underlying databases.
- **Removal of coordinator writes for read-only transactions**
- Users will experience faster execution for read-only transactions by removing coordinator writes for those transactions.

#### Cloud support

- **Red Hat OpenShift support**
Expand Down

0 comments on commit 81c8be6

Please sign in to comment.