Skip to content

Commit

Permalink
Add migrations section (#8873) (#8889)
Browse files Browse the repository at this point in the history
* Add migrations section (#8873)

* Add migrations section.

Signed-off-by: Archer <naarcha@amazon.com>

* Add back upgrade pages. Make them less migration focused.

Signed-off-by: Archer <naarcha@amazon.com>

* Add back in upgrade pages.

Signed-off-by: Archer <naarcha@amazon.com>

* Remove upgrade section. Add redirects.

Signed-off-by: Archer <naarcha@amazon.com>

* Update migration-console-commands-references.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Implement doc review.

Signed-off-by: Archer <naarcha@amazon.com>

* Fix getting started headers.

Signed-off-by: Archer <naarcha@amazon.com>

* Fix parent child relationship for Deploying migration assistant

Signed-off-by: Archer <naarcha@amazon.com>

* Update _migration-assistant/migration-phases/assessing-your-cluster-for-migration.md

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

* Additional feedback

Signed-off-by: Archer <naarcha@amazon.com>

* Add permalink for migration console

Signed-off-by: Archer <naarcha@amazon.com>

* Add redirects for index pages.

Signed-off-by: Archer <naarcha@amazon.com>

* Final edits.

Signed-off-by: Archer <naarcha@amazon.com>

* Fix headings.

Signed-off-by: Archer <naarcha@amazon.com>

* Add copy buttons.

Signed-off-by: Archer <naarcha@amazon.com>

* Last copy button.

Signed-off-by: Archer <naarcha@amazon.com>

---------

Signed-off-by: Archer <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
(cherry picked from commit 8664051)

* Fix merge conflicts

Signed-off-by: Archer <naarcha@amazon.com>

* Delete _migrations directory

Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>

---------

Signed-off-by: Archer <naarcha@amazon.com>
Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com>
Naarcha-AWS authored Dec 5, 2024
1 parent ac76760 commit 5b7e5ff
Showing 29 changed files with 218 additions and 717 deletions.
23 changes: 15 additions & 8 deletions _config.yml
Original file line number Diff line number Diff line change
@@ -31,9 +31,6 @@ collections:
install-and-configure:
permalink: /:collection/:path/
output: true
upgrade-to:
permalink: /:collection/:path/
output: true
im-plugin:
permalink: /:collection/:path/
output: true
@@ -94,6 +91,9 @@ collections:
data-prepper:
permalink: /:collection/:path/
output: true
migration-assistant:
permalink: /:collection/:path/
output: true
tools:
permalink: /:collection/:path/
output: true
@@ -137,11 +137,6 @@ opensearch_collection:
install-and-configure:
name: Install and upgrade
nav_fold: true
upgrade-to:
name: Migrate to OpenSearch
# nav_exclude: true
nav_fold: true
# search_exclude: true
im-plugin:
name: Managing Indexes
nav_fold: true
@@ -213,6 +208,12 @@ clients_collection:
name: Clients
nav_fold: true

migration_assistant_collection:
collections:
migration-assistant:
name: Migration Assistant
nav_fold: true

benchmark_collection:
collections:
benchmark:
@@ -252,6 +253,12 @@ defaults:
values:
section: "benchmark"
section-name: "Benchmark"
-
scope:
path: "_migration-assistant"
values:
section: "migration-assistant"
section-name: "Migration Assistant"

# Enable or disable the site search
# By default, just-the-docs enables its JSON file-based search. We also have an OpenSearch-driven search functionality.
2 changes: 2 additions & 0 deletions _data/top_nav.yml
Original file line number Diff line number Diff line change
@@ -63,6 +63,8 @@ items:
url: /docs/latest/clients/
- label: Benchmark
url: /docs/latest/benchmark/
- label: Migration Assistant
url: /docs/latest/migration-assistant/
- label: Platform
url: /platform/index.html
children:
8 changes: 7 additions & 1 deletion _includes/cards.html
Original file line number Diff line number Diff line change
@@ -30,8 +30,14 @@
<p class="description">Measure performance metrics for your OpenSearch cluster</p>
<p class="last-link">Documentation &#x2192;</p>
</div>

<div class="card">
<a href="{{site.url}}/docs/latest/migration-assistant/" class='card-link'></a>
<p class="heading">Migration Assistant</p>
<p class="description">Migrate to OpenSearch from other platforms</p>
<p class="last-link">Documentation &#x2192;</p>
</div>
</div>

</div>


2 changes: 2 additions & 0 deletions _layouts/default.html
Original file line number Diff line number Diff line change
@@ -87,6 +87,8 @@
{% assign section = site.clients_collection.collections %}
{% elsif page.section == "benchmark" %}
{% assign section = site.benchmark_collection.collections %}
{% elsif page.section == "migration-assistant" %}
{% assign section = site.migration_assistant_collection.collections %}
{% endif %}

{% if section %}
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
layout: default
title: Configuration options
nav_order: 15
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# Configuration options
@@ -61,6 +61,7 @@ The following CDK performs a backfill migrations using RFS:
}
}
```
{% include copy.html %}

Performing an RFS backfill migration requires an existing snapshot.

@@ -104,6 +105,7 @@ The following sample CDK performs a live capture migration with C&R:
}
}
```
{% include copy.html %}

Performing a live capture migration requires that a Capture Proxy be configured to capture incoming traffic and send it to the target cluster using the Traffic Replayer service. For arguments available in `captureProxyExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/CaptureProxy.java). For `trafficReplayerExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java). At a minimum, no extra arguments may be needed.

@@ -125,17 +127,18 @@ Both the source and target cluster can use no authentication, authentication lim

### No authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
"auth": {"type": "none"}
}
```
{% include copy.html %}

### Basic authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
@@ -146,10 +149,11 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

### Signature Version 4 authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
@@ -160,11 +164,12 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

The `serviceSigningName` can be `es` for an Elasticsearch or OpenSearch domain, or `aoss` for an OpenSearch Serverless collection.

All of these authentication options apply to both source and target clusters.

## Network configuration

The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
Original file line number Diff line number Diff line change
@@ -2,7 +2,7 @@
layout: default
title: IAM and security groups for existing clusters
nav_order: 20
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# IAM and security groups for existing clusters
@@ -33,7 +33,7 @@ For an OpenSearch Serverless Collection, you will need to configure both network
The Collection should have a network policy that uses the `VPC` access type. This requires creating a VPC endpoint on the VPC used for the solution. The VPC endpoint should be configured for the private subnets of the VPC and should attach the `osClusterAccessSG` security group.

2. **Data Access Policy Configuration**:
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, Migration Console, Reindex-from-Snapshot) should be used as the principals for this data access policy.
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, migration console, `Reindex-from-Snapshot`) should be used as the principals for this data access policy.

## Capture Proxy on Coordinator Nodes of Source Cluster

@@ -68,7 +68,3 @@ Before [setting up Capture Proxy instances](https://github.com/opensearch-projec
]
}
```

## Related Links

- [OpenSearch traffic capture setup]
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
---
layout: default
title: Deploying Migration Assistant
nav_order: 10
nav_order: 15
has_children: true
permalink: /deploying-migration-assistant/
redirect-from:
- /deploying-migration-assistant/index/
---

# Deploying Migration Assistant
Loading

0 comments on commit 5b7e5ff

Please sign in to comment.