From fd2ec19ea03a4313a4fffc8b8f4d7b9e92d6a806 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 17:51:27 +0100 Subject: [PATCH 01/12] draft of the database project --- projects/mysql-cluster/README.md | 218 +++++++++++++++++++++++++++++++ 1 file changed, 218 insertions(+) create mode 100644 projects/mysql-cluster/README.md diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md new file mode 100644 index 000000000..6d22710a6 --- /dev/null +++ b/projects/mysql-cluster/README.md @@ -0,0 +1,218 @@ + +# Clustered MySQL Configuration and Troubleshooting + +This project is designed to introduce you to MySQL database setup, schema creation, replication, and failover using Amazon EC2 instances. + +You will set up a primary MySQL server, test it, add a secondary server for replication, and demonstrate replication and failover processes. Most steps are designed to succeed, but some will require some troubleshooting and problem-solving. + +## Learning Objectives +- Be able to install MySQL server +- Learn how to configure MySQL as a cluster +- Troubleshoot the cluster configuration +- Fail-over the cluster + +Timebox: 5 days + +## Project + +### Task 1: Set Up the Primary MySQL Server + +1. **Launch an EC2 Instance** + - Name your instance `db-proj--primary` + - Choose an Ubuntu AMI + - Select a `t2.micro` instance type + - Use key pair if you have one (recommended) + - Select existing security group: + - allow ssh access + - open-database-port-internal + - Keep default settings for storage + - Review and launch the instance + - Connect to the instance using SSH + +2. **Install MySQL Server** + - Update the package list and install MySQL server: + ```bash + sudo apt-get update + sudo apt-get install mysql-server + ``` + - Secure the MySQL installation: + Answer `Y` for everything, and for this exercise choose *LOW* password validation policy) + ```bash + sudo mysql_secure_installation + ``` + +3. **Configure MySQL for Remote Access** + - Edit the MySQL configuration file to allow remote connections: + ```bash + sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf + ``` + - Change `bind-address` to `0.0.0.0`. + - Restart MySQL service: + ```bash + sudo systemctl restart mysql + ``` + +4. **Create a MySQL User for Replication** + - Log in to MySQL and create a user for replication: + ```bash + sudo mysql -u root + ``` + - Run the following SQL commands: + ```sql + CREATE USER 'replica_user'@'%' IDENTIFIED BY 'C90L6`!Doe{K'; + GRANT REPLICATION SLAVE ON *.* TO 'replica_user'@'%'; + FLUSH PRIVILEGES; + ``` + +5. **Create a Sample Database Schema** + - Still in the MySQL console, create a test database and table: + ```sql + CREATE DATABASE cyfdb; + USE cyfdb; + CREATE TABLE users ( + id INT AUTO_INCREMENT PRIMARY KEY, + name VARCHAR(100), + email VARCHAR(100) + ); + ``` + +### Task 2: Set Up the Secondary MySQL Server + +1. **Launch a Second EC2 Instance** + - Follow similar steps as for the primary instance, but tag it as `db-proj--replica` just for your reference. + +2. **Install MySQL Server on the Secondary Instance** + - Connect to the secondary instance using SSH. + - Repeat the installation steps as for the primary server. + +3. **Configure MySQL for Replication on the Secondary Server** + - Stop MySQL service: + ```bash + sudo systemctl stop mysql + ``` + - Edit the MySQL configuration file: + ```bash + sudo nano /etc/mysql/mysql.conf.d/mysqld.cnf + ``` + - Add the following lines at the end: + ```ini + server-id=2 + relay-log=/var/log/mysql/mysql-relay-bin.log + log_bin=/var/log/mysql/mysql-bin.log + ``` + - Start MySQL service: + ```bash + sudo systemctl start mysql + ``` + +4. **Configure Replication** + - Obtain the master status on the primary server: + ```sql + SHOW MASTER STATUS; + ``` + - Note down the `File` and `Position` values, also note down the private instance IP address (e.g. from your AWS console) + - On the secondary server, set up the replication: + ```sql + CHANGE MASTER TO + MASTER_HOST='', + MASTER_USER='replica_user', + MASTER_PASSWORD='C90L6`!Doe{K', + MASTER_LOG_FILE='', + MASTER_LOG_POS=; + START SLAVE; + ``` + +5. **Verify Replication** + - Check the replica status: + ```sql + SHOW REPLICA STATUS\G; + ``` + - Ensure `Slave_IO_Running` and `Slave_SQL_Running` are both `Yes`. + - Something isn't quite right. Can you figure out how to fix this? There may be a few things that need to be fixed. Use `SHOW REPLICA STATUS\G` to find out what is wrong. Can you see the `cyfdb` database on the replica? + +### Task 3: Demonstrate Replication + +1. **Insert Data into the Primary Server** + - On the primary server, insert a new record: + ```sql + INSERT INTO cyfdb.users (name, email) VALUES ('John Doe', 'john@example.com'); + ``` + +2. **Verify Data on the Secondary Server** + - On the secondary server, query the table: + ```sql + SELECT * FROM testdb.users; + ``` + - Verify that the data matches the primary server. + - Execute the following on the secondary server: + ```sql + INSERT INTO cyfdb.users (name, email) VALUES ('Jane Doe', 'jane@example.com'); + ``` + Did it work? If not, can you guess why? + +### Task 4: Demonstrate Failover + +1. **Simulate Primary Server Failure** + - Stop the MySQL service on the primary server: + ```bash + sudo systemctl stop mysql + ``` + +2. **Promote Secondary Server to Primary** + - On the secondary server, stop the replica: + ```sql + STOP REPLICA; + ``` + - Reset the replica configuration: + ```sql + RESET REPLICA ALL; + ``` + - Ensure the secondary server can accept writes by setting the read-only mode to off: + ```sql + SET GLOBAL read_only = OFF; + ``` + +3. **Test Write Operations on the New Primary** + - Insert new data into the secondary server (now acting as the primary): + ```sql + INSERT INTO cyfdb.users (name, email) VALUES ('Jane Doe', 'jane@example.com'); + ``` + - Query the table to ensure the new data is inserted: + ```sql + SELECT * FROM cyfd.users; + ``` + +### Task 5: Reconfigure Original Primary as Secondary + +1. **Reconfigure the Original Primary** + - Start the MySQL service on the original primary: + ```bash + sudo systemctl start mysql + ``` + - On the original primary server, set it up as the slave to the new primary: + ```sql + CHANGE MASTER TO + MASTER_HOST='', + MASTER_USER='replica_user', + MASTER_PASSWORD='C90L6`!Doe{K', + MASTER_LOG_FILE='', + MASTER_LOG_POS=; + START SLAVE; + ``` + - Check the replica status: + ```sql + SHOW REPLICA STATUS\G; + ``` + +2. **Verify Data Synchronization** + - Insert new data on the new primary and verify it replicates to the original primary. + + This step may again not work straight away. Refer to how you configured the original primary, did you follow every step for this server? + +### Task 6: Add another replica (Optional) + +Using the steps below, can you add another replica to this MySQL cluster? Describe the issues you are having, what do you think might be the solution? From 5b0c89946767014b423347bc5f901d7ce3e688c3 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 17:54:43 +0100 Subject: [PATCH 02/12] Update README.md --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 6d22710a6..d89c95820 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -215,4 +215,4 @@ Timebox: 5 days ### Task 6: Add another replica (Optional) -Using the steps below, can you add another replica to this MySQL cluster? Describe the issues you are having, what do you think might be the solution? +Can you add another replica to this MySQL cluster using the steps above? Describe the issues you are having, what do you think might be the solution? From ff76eae59c3c66a00a3252ff00dcfd7684aff2d5 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 19:10:07 +0100 Subject: [PATCH 03/12] Update projects/mysql-cluster/README.md Co-authored-by: Sally McGrath --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index d89c95820..2f1ad4626 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -10,7 +10,7 @@ This project is designed to introduce you to MySQL database setup, schema creati You will set up a primary MySQL server, test it, add a secondary server for replication, and demonstrate replication and failover processes. Most steps are designed to succeed, but some will require some troubleshooting and problem-solving. ## Learning Objectives -- Be able to install MySQL server +- Install MySQL server - Learn how to configure MySQL as a cluster - Troubleshoot the cluster configuration - Fail-over the cluster From 61775feaf87e818a5bc5f56890d84158acc04251 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 19:10:13 +0100 Subject: [PATCH 04/12] Update projects/mysql-cluster/README.md Co-authored-by: Sally McGrath --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 2f1ad4626..a9600fb33 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -11,7 +11,7 @@ You will set up a primary MySQL server, test it, add a secondary server for repl ## Learning Objectives - Install MySQL server -- Learn how to configure MySQL as a cluster +- Configure MySQL as a cluster - Troubleshoot the cluster configuration - Fail-over the cluster From 11e9a8598269be2fdb69de0db88b947c10ab3158 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 19:15:57 +0100 Subject: [PATCH 05/12] Update projects/mysql-cluster/README.md Co-authored-by: Sally McGrath --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index a9600fb33..d140b28ed 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -215,4 +215,4 @@ Timebox: 5 days ### Task 6: Add another replica (Optional) -Can you add another replica to this MySQL cluster using the steps above? Describe the issues you are having, what do you think might be the solution? +Can you add another replica to this MySQL cluster using the steps above? Describe the issues you are having; what do you think might be the solution? From eb286b175ec7937e32ea6d3d26adbb0e31a46354 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 19:17:55 +0100 Subject: [PATCH 06/12] Update README.md --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index d140b28ed..09a1aa88d 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -132,7 +132,7 @@ Timebox: 5 days SHOW REPLICA STATUS\G; ``` - Ensure `Slave_IO_Running` and `Slave_SQL_Running` are both `Yes`. - - Something isn't quite right. Can you figure out how to fix this? There may be a few things that need to be fixed. Use `SHOW REPLICA STATUS\G` to find out what is wrong. Can you see the `cyfdb` database on the replica? + - Something isn't quite right. Can you figure out how to fix this? There may be a few things that need to be fixed. Use `SHOW REPLICA STATUS\G` to find out what is wrong. Can you see the `cyfdb` database on the replica? Keep a log of all commands you are executing on each server while troubleshooting this. ### Task 3: Demonstrate Replication From 8bf37dee82f74a77fd7f8531d9591ac46ca9218f Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 22:32:51 +0100 Subject: [PATCH 07/12] Update README.md --- projects/mysql-cluster/README.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 09a1aa88d..65d358520 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -1,14 +1,17 @@ -# Clustered MySQL Configuration and Troubleshooting +# MySQL Replication - Configuration and Troubleshooting This project is designed to introduce you to MySQL database setup, schema creation, replication, and failover using Amazon EC2 instances. You will set up a primary MySQL server, test it, add a secondary server for replication, and demonstrate replication and failover processes. Most steps are designed to succeed, but some will require some troubleshooting and problem-solving. +There are different ways to configure MySQL replication. In this exercise, you will be configuring our servers for primary-replica (or master-slave) replication. +In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase scalability of the database for read-intensive operations (which is extremely common for web). In the primary-replica setup, the replica (or replicas) would normally be used for reads and primary for writes only. Even though it's technically possible to use the primary for the reads as well, it is not possible to write to the replica directly. + ## Learning Objectives - Install MySQL server - Configure MySQL as a cluster From a254e4f02687b09648aa37b684833afce835025b Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Wed, 29 May 2024 22:35:17 +0100 Subject: [PATCH 08/12] Update README.md --- projects/mysql-cluster/README.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 65d358520..1eed19343 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -9,9 +9,6 @@ This project is designed to introduce you to MySQL database setup, schema creati You will set up a primary MySQL server, test it, add a secondary server for replication, and demonstrate replication and failover processes. Most steps are designed to succeed, but some will require some troubleshooting and problem-solving. -There are different ways to configure MySQL replication. In this exercise, you will be configuring our servers for primary-replica (or master-slave) replication. -In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase scalability of the database for read-intensive operations (which is extremely common for web). In the primary-replica setup, the replica (or replicas) would normally be used for reads and primary for writes only. Even though it's technically possible to use the primary for the reads as well, it is not possible to write to the replica directly. - ## Learning Objectives - Install MySQL server - Configure MySQL as a cluster @@ -22,6 +19,10 @@ Timebox: 5 days ## Project +There are different ways to configure MySQL replication. In this exercise, you will be configuring our servers for primary-replica (or master-slave) replication. + +In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase the scalability of the database for read-intensive operations (which is extremely common for the web). In the primary-replica setup, the replica (or replicas) would normally be used for reads and primary for writes only. Even though it's technically possible to use the primary for the reads and the writes, it is impossible to write directly to the replica. + ### Task 1: Set Up the Primary MySQL Server 1. **Launch an EC2 Instance** From 81db61765db3b79da478d1aa14301c2b569e082e Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Mon, 3 Jun 2024 00:46:05 +0100 Subject: [PATCH 09/12] Update projects/mysql-cluster/README.md Co-authored-by: Daniel Wagner-Hall --- projects/mysql-cluster/README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 1eed19343..99985ef3b 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -161,6 +161,7 @@ In this type of replication, the primary server (or the master) takes all the wr ### Task 4: Demonstrate Failover 1. **Simulate Primary Server Failure** +We're going to stop the primary server, to simulate some real failure (e.g. hardware failure or loss of network). - Stop the MySQL service on the primary server: ```bash sudo systemctl stop mysql From e25b9738bb617c3703e2bc056dbeea20f438db33 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Mon, 3 Jun 2024 22:03:11 +0100 Subject: [PATCH 10/12] addressing comments --- projects/mysql-cluster/README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 99985ef3b..40e49a213 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -15,11 +15,11 @@ You will set up a primary MySQL server, test it, add a secondary server for repl - Troubleshoot the cluster configuration - Fail-over the cluster -Timebox: 5 days +Timebox: 2 days ## Project -There are different ways to configure MySQL replication. In this exercise, you will be configuring our servers for primary-replica (or master-slave) replication. +There are different ways to configure MySQL replication. In this exercise, you will be configuring your servers for primary-replica (or master-slave) replication. In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase the scalability of the database for read-intensive operations (which is extremely common for the web). In the primary-replica setup, the replica (or replicas) would normally be used for reads and primary for writes only. Even though it's technically possible to use the primary for the reads and the writes, it is impossible to write directly to the replica. @@ -152,11 +152,11 @@ In this type of replication, the primary server (or the master) takes all the wr SELECT * FROM testdb.users; ``` - Verify that the data matches the primary server. - - Execute the following on the secondary server: + - Before you execute the next statement, please write down what result you would expect from it: ```sql INSERT INTO cyfdb.users (name, email) VALUES ('Jane Doe', 'jane@example.com'); ``` - Did it work? If not, can you guess why? + Does the result match your prediction? If not, can you guess why? ### Task 4: Demonstrate Failover From 886a5052e99669957d4afdf35f26e6dc37c88cd4 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Mon, 3 Jun 2024 22:17:47 +0100 Subject: [PATCH 11/12] addressing comments --- projects/mysql-cluster/README.md | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 40e49a213..9d8b11377 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -11,9 +11,9 @@ You will set up a primary MySQL server, test it, add a secondary server for repl ## Learning Objectives - Install MySQL server -- Configure MySQL as a cluster -- Troubleshoot the cluster configuration -- Fail-over the cluster +- Configure MySQL for replication +- Troubleshoot the replication configuration +- Fail-over the replicating primary Timebox: 2 days @@ -21,7 +21,9 @@ Timebox: 2 days There are different ways to configure MySQL replication. In this exercise, you will be configuring your servers for primary-replica (or master-slave) replication. -In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase the scalability of the database for read-intensive operations (which is extremely common for the web). In the primary-replica setup, the replica (or replicas) would normally be used for reads and primary for writes only. Even though it's technically possible to use the primary for the reads and the writes, it is impossible to write directly to the replica. +In this type of replication, the primary server (or the master) takes all the writes and they are automatically replicated onto the replica server (or the slave). This technique is widely used to increase the scalability of the database for read-intensive operations (which is extremely common for the web). In the primary-replica setup, the primary would normally be used for writes and replica (or replicas) for reads only. Even though it's technically possible to use the primary for the reads and the writes, it is impossible to write directly to the replica. + +Another advantage of using such a replication setup is database resilience. For example, it is recommended to setup primary-replica with one primary and two or three replicas in different availability zones. In the event of one availability zone (or datacenter) going down, the database will continue functioning flawlessly as other replicas will be used for reading. In a different scenario of one replica crashing, it can be replaced while the remaining replicas are serving the reads. Should the primary crash, an operation called a 'fail-over' should be carried out: one replica is promoted to be a primary while another MySQL server is being stood up in place of a broken primary. ### Task 1: Set Up the Primary MySQL Server @@ -190,7 +192,9 @@ We're going to stop the primary server, to simulate some real failure (e.g. hard ```sql SELECT * FROM cyfd.users; ``` - +4. **Service Location** + - Write down your thoughts on how primary and replica can be conveniently located by their clients (e.g. a web application), given the fact that primary may be failed over and replicas replaced at any moment, and the new instances will receive a different IP address. + ### Task 5: Reconfigure Original Primary as Secondary 1. **Reconfigure the Original Primary** From 10783f317db5b37b4558fdc0eee7e67b995b46f6 Mon Sep 17 00:00:00 2001 From: Stas Maksimov Date: Mon, 3 Jun 2024 22:20:13 +0100 Subject: [PATCH 12/12] Update projects/mysql-cluster/README.md Co-authored-by: Sally McGrath --- projects/mysql-cluster/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/projects/mysql-cluster/README.md b/projects/mysql-cluster/README.md index 9d8b11377..b189872c9 100644 --- a/projects/mysql-cluster/README.md +++ b/projects/mysql-cluster/README.md @@ -220,7 +220,7 @@ We're going to stop the primary server, to simulate some real failure (e.g. hard 2. **Verify Data Synchronization** - Insert new data on the new primary and verify it replicates to the original primary. - This step may again not work straight away. Refer to how you configured the original primary, did you follow every step for this server? + This step may again not work straight away. Refer to how you configured the original primary: did you follow every step for this server? ### Task 6: Add another replica (Optional)