From 07ecd61a03587e70d2f1a2b3f1041f1f3ae68e91 Mon Sep 17 00:00:00 2001 From: Manzarul Date: Fri, 20 Jul 2018 11:20:03 +0530 Subject: [PATCH 1/6] Issues #SB-5473 feat: doc for user external table data migration --- .../developer-docs/installation/migration.md | 55 +++++++++++++++++-- 1 file changed, 49 insertions(+), 6 deletions(-) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index 8a3f06ea0..e92ede18e 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -10,19 +10,62 @@ allowSearch: true ## Overview - +In Release-1.8 sunbird decide to support for multiple externalIds of a user.TO support multiple externalIds ,we design new table usr_external_identity.Now old table user_external_identity data need to be moved to new table.This activity need to be done after server setup. ## Prerequisites - +To configure domains to host external links, ensure the following: +- Caller should have the access of cassandra db , to read the records. +- get all unique provider value stored under user_external_identity table +- Run this cql: select provider from user_external_identity; +- create a csv file as follow +- provider,idType + ka,kaIdType + ap,apIdType +- no need to repeat same provider again inside file. +- Save this as {someName}.csv ## Configuration Parameters - +- encryption_key (used to encrypt the data) +- keyspace_name (cassandra keyspace name) +- cassandra_port (cassandra port) +- cassandra_server_ip (cassandra ip) +- provider_idtype_mapping_file_path (created .csv file path) ## Migrating User Data - +Step 1: Download https://github.com/project-sunbird/sunbird-utils/tree/data-migration-etl/cassandra-migration-etl/r1.8 - UserExternalIdentityMigrationBin.zip -## Table Details +Step2: unzip UserExternalIdentityMigrationBin.zip +step3: run below command + +### For windows : - +ExternalIdentityTableMigrationExtension_run.bat --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + +## For Linux : + +ExternalIdentityTableMigrationExtension_run.sh --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + + +## Table Details +Old Table (user_external_identity) structure: + - id text PRIMARY KEY, + - createdby text, + - createdon timestamp, + - externalid text, + - lastupdatedby text, + - lastupdatedon timestamp, + - provider text, + - userid text + + New Table (usr_external_identity) structure: + - provider text, + - idtype text, + - externalid text, + - createdby text, + - createdon timestamp, + - lastupdatedby text, + - lastupdatedon timestamp, + - userid text, + - PRIMARY KEY (provider, idtype, externalid) \ No newline at end of file From 32f3370b6a6cb0e1cdd0fec145518ec86a889b8d Mon Sep 17 00:00:00 2001 From: Manzarul Date: Fri, 20 Jul 2018 16:32:57 +0530 Subject: [PATCH 2/6] Issues #SB-5218 feat: adding for user-external_id migration --- .../developer-docs/installation/migration.md | 70 ++++++++++++------- 1 file changed, 45 insertions(+), 25 deletions(-) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index e92ede18e..cac0b07fd 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -10,46 +10,64 @@ allowSearch: true ## Overview -In Release-1.8 sunbird decide to support for multiple externalIds of a user.TO support multiple externalIds ,we design new table usr_external_identity.Now old table user_external_identity data need to be moved to new table.This activity need to be done after server setup. +From release-1.8, Sunbird provides support for multiple external IDs for a user. To add multiple external IDs, an administrator of any sunbird instance needs to design a new table with name "usr_external_identity" +This table must be seeded with the data from the existing table "user_external_identity. + +**Note:** This is a post server setup task. ## Prerequisites -To configure domains to host external links, ensure the following: -- Caller should have the access of cassandra db , to read the records. -- get all unique provider value stored under user_external_identity table -- Run this cql: select provider from user_external_identity; -- create a csv file as follow -- provider,idType - ka,kaIdType - ap,apIdType -- no need to repeat same provider again inside file. -- Save this as {someName}.csv +To configure your domain to host external links, ensure that the following prerequisites are met: + +1. Administrator access to read the record sets from Cassandra DB +2. Ensure to store all unique provider values under user_external_identity table, to do so, execute the following command: + + A. Run this cql: select provider from user_external_identity; + + B. create a csv file as follow + - provider,idType + - ka,kaIdType + - ap,apIdType + + Note: You do not need to repeat same provider again inside the file + + C. Save this as file as {typeyouename}.csv ## Configuration Parameters -- encryption_key (used to encrypt the data) -- keyspace_name (cassandra keyspace name) -- cassandra_port (cassandra port) -- cassandra_server_ip (cassandra ip) -- provider_idtype_mapping_file_path (created .csv file path) +The following are the configuration parameters: + +|Sno.|Configuration parameter| Usage| +| 1 | encryption_key | It is used to encrypt the data | +| 2 | keyspace_name | It is the cassandra keyspace name | +| 3 | cassandra_port | It is the port number for Cassandra | +| 4 | cassandra_server_ip | It is the IP address for Cassandra | +| 5 |provider_idtype_mapping_file_path | It is the .csv file path (which is created) | ## Migrating User Data -Step 1: Download https://github.com/project-sunbird/sunbird-utils/tree/data-migration-etl/cassandra-migration-etl/r1.8 - UserExternalIdentityMigrationBin.zip -Step2: unzip UserExternalIdentityMigrationBin.zip -step3: run below command +In order to migrate the user data, follow these steps: + +1. Download the code from [here] https://github.com/project-sunbird/sunbird-utils/tree/data-migration-etl/cassandra-migration-etl/r1.8 - UserExternalIdentityMigrationBin.zip + +2. Unzip the UserExternalIdentityMigrationBin.zip file + +3. Run the following commands based on the operating systems you are using: -### For windows : + - For Windows, run the following command: -ExternalIdentityTableMigrationExtension_run.bat --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + `ExternalIdentityTableMigrationExtension_run.bat --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}"` -## For Linux : + - For Linux, run the following command: -ExternalIdentityTableMigrationExtension_run.sh --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + `ExternalIdentityTableMigrationExtension_run.sh --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}"` ## Table Details -Old Table (user_external_identity) structure: + +The details of the existing table are as follows: + +user_external_identity structure: - id text PRIMARY KEY, - createdby text, - createdon timestamp, @@ -59,7 +77,9 @@ Old Table (user_external_identity) structure: - provider text, - userid text - New Table (usr_external_identity) structure: +The details of the new table are as follows: + +usr_external_identity structure: - provider text, - idtype text, - externalid text, From 3ec7122807ece0fc497d005c97525c849dabae41 Mon Sep 17 00:00:00 2001 From: manzarul Haque Date: Mon, 23 Jul 2018 16:51:59 +0530 Subject: [PATCH 3/6] Update migration.md --- .../developer-docs/installation/migration.md | 121 ++++++++++-------- 1 file changed, 66 insertions(+), 55 deletions(-) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index cac0b07fd..7dd8d720f 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -10,82 +10,93 @@ allowSearch: true ## Overview -From release-1.8, Sunbird provides support for multiple external IDs for a user. To add multiple external IDs, an administrator of any sunbird instance needs to design a new table with name "usr_external_identity" -This table must be seeded with the data from the existing table "user_external_identity. +Sunbird provides support for multiple external IDs for a user. In release 1.8, the table name and schema used to store user external IDs has changed. -**Note:** This is a post server setup task. +The purpose of this document is to describe the steps for migration of user external IDs from old "user_external_identity" table to new "usr_external_identity" table format. -## Prerequisites +Following are the changes: +* The table name changed from "user_external_identity" to "usr_external_identity". +* A new column "idtype" is introduced in "usr_external_identity" table for storing external ID type. e.g. UIDAI is provider of Aadhaar (ID type). +* In release 1.7, the external ID was unencrypted. In release 1.8, encryption of external ID is also performed to ensure privacy of user external ID information. -To configure your domain to host external links, ensure that the following prerequisites are met: +**Note:** This migration of data needs to be performed after deploying Sunbird learner service release version 1.8. -1. Administrator access to read the record sets from Cassandra DB -2. Ensure to store all unique provider values under user_external_identity table, to do so, execute the following command: +## Prerequisites - A. Run this cql: select provider from user_external_identity; +Following are the pre-requisites before running the migration script: - B. create a csv file as follow - - provider,idType - - ka,kaIdType - - ap,apIdType +1. Define a mapping file which specifies ID type for each unique external ID provider available in "user_external_identity" table. + A. Get list of all providers using CQL query: select provider from user_external_identity; - Note: You do not need to repeat same provider again inside the file + B. Create a CSV file (e.g. mapping.csv) which maps each **unique** external ID to desired ID type. + e.g. + ``` + provider,idType + ka,kaIdType + ap,apIdType + ``` - C. Save this as file as {typeyouename}.csv +2. Before performing any migration take a backup of data in sunbird keyspace within Cassandra DB. ## Configuration Parameters The following are the configuration parameters: -|Sno.|Configuration parameter| Usage| -| 1 | encryption_key | It is used to encrypt the data | -| 2 | keyspace_name | It is the cassandra keyspace name | -| 3 | cassandra_port | It is the port number for Cassandra | -| 4 | cassandra_server_ip | It is the IP address for Cassandra | -| 5 |provider_idtype_mapping_file_path | It is the .csv file path (which is created) | - -## Migrating User Data +| S.No. | Configuration parameter | Description | +| 1 | cassandra_server | Cassandra DB IP Address | +| 2 | cassandra_port | Cassandra DB Port Number | +| 3 | keyspace_name | Cassandra DB Keyspace Name | +| 4 | encryption_key | Key used in Sunbird for encryption of private user information | +| 5 | provider_idtype_mapping_file_path | Path to CSV file which defines mapping between external ID provider and type | -In order to migrate the user data, follow these steps: +## Running Migration Script -1. Download the code from [here] https://github.com/project-sunbird/sunbird-utils/tree/data-migration-etl/cassandra-migration-etl/r1.8 - UserExternalIdentityMigrationBin.zip +In order to migrate the user external ID data, follow below mentioned steps: -2. Unzip the UserExternalIdentityMigrationBin.zip file +1. Extract archive file (sunbird-utils/cassandra-migration-etl/r1.8/UserExternalIdentityMigrationBin.zip) which contains script for user external identity table migration. -3. Run the following commands based on the operating systems you are using: +2. Based on OS, run the corresponding script for performing the migration. - - For Windows, run the following command: - - `ExternalIdentityTableMigrationExtension_run.bat --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}"` - - For Linux, run the following command: + ``` + UserExternalIdentityMigration_run.sh --context_param cassandra_server="{cassandra_server}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_keyspace="{keyspace_name}" --context_param sunbird_encryption_key="{encryption_key}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + ``` - `ExternalIdentityTableMigrationExtension_run.sh --context_param sunbird_encryption_key="{encryption_key}" --context_param cassandra_keyspace="{keyspace_name}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_server="{cassandra_server_ip}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}"` +- For Windows, run the following command: + ``` + UserExternalIdentityMigration_run.bat --context_param cassandra_server="{cassandra_server}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_keyspace="{keyspace_name}" --context_param sunbird_encryption_key="{encryption_key}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" + ``` ## Table Details -The details of the existing table are as follows: - -user_external_identity structure: - - id text PRIMARY KEY, - - createdby text, - - createdon timestamp, - - externalid text, - - lastupdatedby text, - - lastupdatedon timestamp, - - provider text, - - userid text - -The details of the new table are as follows: - -usr_external_identity structure: - - provider text, - - idtype text, - - externalid text, - - createdby text, - - createdon timestamp, - - lastupdatedby text, - - lastupdatedon timestamp, - - userid text, - - PRIMARY KEY (provider, idtype, externalid) \ No newline at end of file +The schema of the existing table is as follows: + +``` +user_external_identity ( + id text PRIMARY KEY, + createdby text, + createdon timestamp, + externalid text, + lastupdatedby text, + lastupdatedon timestamp, + provider text, + userid text +) +``` + +The schema of the new table is as follows: + +``` +usr_external_identity ( + provider text, + idtype text, + externalid text, + createdby text, + createdon timestamp, + lastupdatedby text, + lastupdatedon timestamp, + userid text, + PRIMARY KEY (provider, idtype, externalid) +) +``` From 0ac369c207ff24c57ec09bc2f57b25af6f61b703 Mon Sep 17 00:00:00 2001 From: manzarul Haque Date: Mon, 23 Jul 2018 17:00:39 +0530 Subject: [PATCH 4/6] Update migration.md --- pages/developer-docs/installation/migration.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index 7dd8d720f..585841c4c 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -39,13 +39,12 @@ Following are the pre-requisites before running the migration script: 2. Before performing any migration take a backup of data in sunbird keyspace within Cassandra DB. ## Configuration Parameters - -The following are the configuration parameters: + The following are the configuration parameters: | S.No. | Configuration parameter | Description | -| 1 | cassandra_server | Cassandra DB IP Address | +| 1 | cassandra_server | Cassandra DB IP Address| | 2 | cassandra_port | Cassandra DB Port Number | -| 3 | keyspace_name | Cassandra DB Keyspace Name | +| 3 | keyspace_name | Cassandra DB Keyspace Name| | 4 | encryption_key | Key used in Sunbird for encryption of private user information | | 5 | provider_idtype_mapping_file_path | Path to CSV file which defines mapping between external ID provider and type | @@ -68,6 +67,9 @@ In order to migrate the user external ID data, follow below mentioned steps: UserExternalIdentityMigration_run.bat --context_param cassandra_server="{cassandra_server}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_keyspace="{keyspace_name}" --context_param sunbird_encryption_key="{encryption_key}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" ``` +3. Verify that external IDs are updated in usr_external_identity table with desired ID type and provider using CQL query: select * from usr_external_identity; + + ## Table Details The schema of the existing table is as follows: From fbf2cedeeaaa4b09f29a1c4b8e1ed29ad16d5714 Mon Sep 17 00:00:00 2001 From: Manzarul Date: Tue, 24 Jul 2018 10:00:01 +0530 Subject: [PATCH 5/6] Issues #SB-5473 fix: chnages in documentation based on review comment --- .../developer-docs/installation/migration.md | 26 +++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index 585841c4c..318ac6e05 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -11,6 +11,10 @@ allowSearch: true ## Overview Sunbird provides support for multiple external IDs for a user. In release 1.8, the table name and schema used to store user external IDs has changed. +* external id is the identity of user out-side of sunbird system. + Example: John belongs to organisation **X** and in **X** his identity is **jo_123** and idType is **employeeId**, +now he is trying to register with sunbird, then in sunbird he can use externalId as **jo_123** and provider as **X** and idType as **employeeId**. +* Similarly John can be part of multiple organisations and each organisations can have there own identity to recognise John, in that case during sunbird registration (create/update user api), he can pass all the externalIds and provider and idType. The purpose of this document is to describe the steps for migration of user external IDs from old "user_external_identity" table to new "usr_external_identity" table format. @@ -71,6 +75,28 @@ In order to migrate the user external ID data, follow below mentioned steps: ## Table Details +### Details description of old table user_external_identity +| S.No. | Field | Description | +| 1 | id | identity of the recoreds.Primary key| +| 2 | createdby | User id , By whom recoreds is created | +| 3 | createdon | On which date and time records inserted into database| +| 4 | externalid | Identity of user apart from sunbird system, so that when other system fetch records from sunbird, they can co-relate user within there system using externalid and provider combination | +| 5 | provider | provider can be any organisation| +| 6 | lastupdatedon | When recoreds was last updated| +| 7 | userid | User identity within sunbird| + + +### Details description of new table usr_external_identity +| S.No. | Field | Description | +| 1 | provider | provider can be any organisation| +| 2 | idtype | idtype can be employeeId, studentid etc | +| 3 | externalid | Identity of user apart from sunbird system, so that when other system fetch records from sunbird, they can co-relate user within there system using externalid and provider combination| +| 4 | createdby | User id , By whom recoreds is created | +| 5 | createdon | On which date and time records inserted into database| +| 6 | lastupdatedon | When recoreds was last updated| +| 7 | userid | User identity within sunbird| + +##Note : Combination of provider,idtype,externalid must be unique. System is using these combination as primaryKey. The schema of the existing table is as follows: From 01b06d682c11c2a63f22e01af941097065d5035e Mon Sep 17 00:00:00 2001 From: Sunitashelar <17581452+Sunitashelar@users.noreply.github.com> Date: Wed, 25 Jul 2018 20:38:31 +0530 Subject: [PATCH 6/6] Update migration.md --- .../developer-docs/installation/migration.md | 106 +++++++++--------- 1 file changed, 53 insertions(+), 53 deletions(-) diff --git a/pages/developer-docs/installation/migration.md b/pages/developer-docs/installation/migration.md index 318ac6e05..0e5f591fc 100644 --- a/pages/developer-docs/installation/migration.md +++ b/pages/developer-docs/installation/migration.md @@ -10,57 +10,52 @@ allowSearch: true ## Overview -Sunbird provides support for multiple external IDs for a user. In release 1.8, the table name and schema used to store user external IDs has changed. -* external id is the identity of user out-side of sunbird system. - Example: John belongs to organisation **X** and in **X** his identity is **jo_123** and idType is **employeeId**, -now he is trying to register with sunbird, then in sunbird he can use externalId as **jo_123** and provider as **X** and idType as **employeeId**. -* Similarly John can be part of multiple organisations and each organisations can have there own identity to recognise John, in that case during sunbird registration (create/update user api), he can pass all the externalIds and provider and idType. +Sunbird supports multiple external IDs for a user. An external ID is the user's identity outside the Sunbird system. For example; John is an employee of **XYZ Corporation**. In **XYZ Corporation**, his identity(ID) is **jo_123** and ID Type is **employeeId**. +When he registers with Sunbird, he can specify his external ID as **jo_123**, provider as **XYZ Corporation** and ID Type as **employeeId**. Similarly, John, being part of multiple organizations, (like also being an account holder in **ABC bank**), has a separate identity in each organization. The ID provided by that organization identifies and recognises John as being part of that particular organization. When John is registered in Sunbird, using the create or update user API, he can specify all the external IDs, providers and ID types as values in the associated parameters. -The purpose of this document is to describe the steps for migration of user external IDs from old "user_external_identity" table to new "usr_external_identity" table format. +As part of Sunbird release version 1.8, the table holding user external ID data is modified as follows: -Following are the changes: -* The table name changed from "user_external_identity" to "usr_external_identity". -* A new column "idtype" is introduced in "usr_external_identity" table for storing external ID type. e.g. UIDAI is provider of Aadhaar (ID type). -* In release 1.7, the external ID was unencrypted. In release 1.8, encryption of external ID is also performed to ensure privacy of user external ID information. +* The table name is modified from **user_external_identity** to **usr_external_identity**. +* A new column **idtype** is introduced in the **usr_external_identity** table to store external ID type. For example, UIDAI is the provider for ID type Aadhaar. +* In Sunbird release version 1.7, the external ID was unencrypted. In release 1.8, the external ID is encrypted to ensure privacy of user external ID information. -**Note:** This migration of data needs to be performed after deploying Sunbird learner service release version 1.8. +**Note:** This data migration should be done after deploying Sunbird learner service release version 1.8. ## Prerequisites -Following are the pre-requisites before running the migration script: +To run the migration script: -1. Define a mapping file which specifies ID type for each unique external ID provider available in "user_external_identity" table. - A. Get list of all providers using CQL query: select provider from user_external_identity; - - B. Create a CSV file (e.g. mapping.csv) which maps each **unique** external ID to desired ID type. - e.g. +1.Define a mapping file that specifies the ID type for each unique external ID provider available in the **user_external_identity** table. + a.Get a list of all providers using CQL query: `select provider from user_external_identity` + b.Create a CSV file (mapping.csv) that maps each unique external ID to its associated ID type + For example; ``` provider,idType ka,kaIdType ap,apIdType ``` -2. Before performing any migration take a backup of data in sunbird keyspace within Cassandra DB. +2. Before migrating any data, take a backup in Sunbird keyspace within the Cassandra DB. ## Configuration Parameters - The following are the configuration parameters: +The following parameters need to be configured for migrating user data: -| S.No. | Configuration parameter | Description | -| 1 | cassandra_server | Cassandra DB IP Address| -| 2 | cassandra_port | Cassandra DB Port Number | -| 3 | keyspace_name | Cassandra DB Keyspace Name| -| 4 | encryption_key | Key used in Sunbird for encryption of private user information | -| 5 | provider_idtype_mapping_file_path | Path to CSV file which defines mapping between external ID provider and type | +| S.No. | Parameter | Description | Example | +| 1 | cassandra_server | Cassandra DB IP Address| 50.34.16.33| +| 2 | cassandra_port | Cassandra DB Port Number | 9145 | +| 3 | keyspace_name | Cassandra DB Keyspace Name| demodb | +| 4 | encryption_key | Key used in Sunbird for encryption of private user information | 60B568970ASDFer321 | +| 5 | provider_idtype_mapping_file_path | Path to CSV file which defines mapping between external ID provider and type | \john\mapfile.csv | -## Running Migration Script +## Run the Migration Script -In order to migrate the user external ID data, follow below mentioned steps: +To migrate user external ID data: -1. Extract archive file (sunbird-utils/cassandra-migration-etl/r1.8/UserExternalIdentityMigrationBin.zip) which contains script for user external identity table migration. +1.Extract the archive file (sunbird-utils/cassandra-migration-etl/r1.8/UserExternalIdentityMigrationBin.zip) that contains the script for user external identity table migration -2. Based on OS, run the corresponding script for performing the migration. +2.Based on the OS, run the corresponding script to migrate the data - - For Linux, run the following command: +- For Linux, run the following command: ``` UserExternalIdentityMigration_run.sh --context_param cassandra_server="{cassandra_server}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_keyspace="{keyspace_name}" --context_param sunbird_encryption_key="{encryption_key}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" ``` @@ -71,34 +66,23 @@ In order to migrate the user external ID data, follow below mentioned steps: UserExternalIdentityMigration_run.bat --context_param cassandra_server="{cassandra_server}" --context_param cassandra_port="{cassandra_port}" --context_param cassandra_keyspace="{keyspace_name}" --context_param sunbird_encryption_key="{encryption_key}" --context_param provider_idtype_mapping_file_path="{provider_idtype_mapping_file_path}" ``` -3. Verify that external IDs are updated in usr_external_identity table with desired ID type and provider using CQL query: select * from usr_external_identity; +3. Verify that the external IDs are updated in the **usr_external_identity** table with the desired ID type and provider using CQL query: `select * from usr_external_identity;` -## Table Details -### Details description of old table user_external_identity -| S.No. | Field | Description | -| 1 | id | identity of the recoreds.Primary key| -| 2 | createdby | User id , By whom recoreds is created | -| 3 | createdon | On which date and time records inserted into database| -| 4 | externalid | Identity of user apart from sunbird system, so that when other system fetch records from sunbird, they can co-relate user within there system using externalid and provider combination | -| 5 | provider | provider can be any organisation| -| 6 | lastupdatedon | When recoreds was last updated| -| 7 | userid | User identity within sunbird| +## user_external_identity Table Details +**Note** The **user_external_identity** table is used prior to Sunbird release version 1.8 -### Details description of new table usr_external_identity | S.No. | Field | Description | -| 1 | provider | provider can be any organisation| -| 2 | idtype | idtype can be employeeId, studentid etc | -| 3 | externalid | Identity of user apart from sunbird system, so that when other system fetch records from sunbird, they can co-relate user within there system using externalid and provider combination| -| 4 | createdby | User id , By whom recoreds is created | -| 5 | createdon | On which date and time records inserted into database| -| 6 | lastupdatedon | When recoreds was last updated| -| 7 | userid | User identity within sunbird| - -##Note : Combination of provider,idtype,externalid must be unique. System is using these combination as primaryKey. +| 1 | id | Identity of the record. This is the primary key| +| 2 | createdby | The User ID of the person who created the record | +| 3 | createdon | The date and time when the record was inserted into the database| +| 4 | externalid | The user's identity apart from the Sunbird system. This field allows other systems fetch records from Sunbird and co-relate the user within their system using a combination of the externalid and provider | +| 5 | provider | The provider of the external ID. This can be any organisation| +| 6 | lastupdatedon | The date on which the record was last updated| +| 7 | userid | The User's identity within Sunbird| -The schema of the existing table is as follows: +### user_external_identity Table Schema ``` user_external_identity ( @@ -113,7 +97,23 @@ user_external_identity ( ) ``` -The schema of the new table is as follows: +### usr_external_identity Table Details + +**Note** The **usr_external_identity** table is used from Sunbird release version 1.8 + +| S.No. | Field | Description | +| 1 | provider | The provider of the external ID. This can be any organisation| +| 2 | idtype | The type of ID for example; employee ID, student ID, etc. | +| 3 | externalid | The user's identity apart from the Sunbird system. This field allows other systems fetch records from Sunbird and co-relate the user within their system using a combination of the externalid and provider | +| 4 | createdby | The User ID of the person who created the record | +| 5 | createdon | The date and time when the record was inserted into the database | +| 6 | lastupdatedon | The date on which the record was last updated | +| 7 | userid | The User's identity within Sunbird| + +**Note:** The combination of provider, idtype and externalid must be unique. The system uses this combination as a primary Key. + + +### usr_external_identity Table Schema ``` usr_external_identity (