arcgis_classify.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p>\n",
    "  <img src=\"index/arcgis_classify_logo.jpg\" alt=\"Logo\", width=\"200\">\n",
    "</p>\n",
    "\n",
    "# ArcGIS_Classify\n",
    "* ArcGIS_Classify is a Python-based Jupyter Notebook for ArcGIS Pro for quick and easy land-use classifications.\n",
    "* This project was created as a final submission for the course \"Python in QGIS and ArcGIS Pro\" by Sven Harpering and Philippe Rieffel at Universität Münster.\n",
    "* The authors are Jonas Starke and Kieran Galbraith. For questions, contact us at: jstarke@uni-muenster.de or k_galb01@uni-muenster.de.\n",
    "* More info under: github.com/kgalb01/ArcGIS_Classify"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1 What are Land Use / Land Cover Classifications (LULC)?\n",
    "<details open>\n",
    "* LULC identify the physical material on the surface of the earth, like forests or water\n",
    "* Identification based mostly on reflective properties captured in multispectral imagery\n",
    "    - Images captured by satelites, like Sentinel-2 or by airborne drones\n",
    "\n",
    "![MultiSpectralImagery_Example](index/multi_spec_example.png)\n",
    "\n",
    "<sup>*Figure 1: Diagram showing how multispectral images are captured. Source: „Fernerkundung und maschinelle Lernverfahren“ by Hanna Meyer, 2021*</sup>\n",
    "\n",
    "* The classification is often done using a machine learning or deep learning algorithm-based model\n",
    "    * For the modeltraining a algorithm is fed with trainingdata that includes labeled examples of different land cover types\n",
    "    * This helps the algorithm recognize patterns and features associated with each type\n",
    "* Based on the model it is then possible to create a prediction map where previously unlabeled data is now classified, enabling efficient and large-scale land use and land cover mapping\n",
    "* A finished very basic LULC could look like this:\n",
    "\n",
    "![LULC_Example](index/lulc_terra-classifier_example.jpg)\n",
    "\n",
    "<sup>*Figure 2: Screenshot showing a LULC from Münster using a model trained with data from Dortmund. Source: Terra Classifier App created for the course \"Geosoftware II\" by Edzer Pebesma & Christian Knoth, 2024*</sup>\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 Necessities\n",
    "* For a proper LULC, you'll need:\n",
    "    1. An area of interest and remote sensing data of the AOI\n",
    "        - We recommend Sentinel-2 as they are easy to get and easy to use\n",
    "        - More info here: [Copernicus Browser](https://browser.dataspace.copernicus.eu)\n",
    "    2. An area of training and remote sensing data of the AOT (can be the same as the area of interest, but doesn't have to be)\n",
    "    3. Training data from the area of training: A .GeoJSON or .gpkg file containing labeled examples of land cover types of your area of training\n",
    "        - NOTE: You'll need at least three entries for each label and at least three labels for the LULC to work properly. The more you have, the better.\n",
    "\n",
    "In the following part, we'll start with a detailed tutorial. If you're already fairly familiar with training data, you can skip to the code block in \"3.1 Integrating training data\" and hit \"Run\".\n",
    "\n",
    "BUT! First things first! Hit the \"Run\" Button on the next code block to install all necessary Modules and Packages so we can be sure that everythings works as intended!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install necessary packages\n",
    "%pip install ipywidgets ipyfilechooser ipyleaflet shapely numpy gdal pyproj matplotlib geopandas scikit-learn joblib rasterio seaborn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import all necessary packages\n",
    "\n",
    "# Import standard libraries\n",
    "import os\n",
    "import json\n",
    "import pickle\n",
    "\n",
    "# Import IPython and Jupyter-related packages\n",
    "from IPython.display import display, clear_output\n",
    "import ipywidgets as widgets\n",
    "import ipyfilechooser as filechooser\n",
    "\n",
    "# Import geospatial and mapping packages\n",
    "from ipyleaflet import Map, basemaps, GeoJSON, DrawControl, basemap_to_tiles, Polygon as LeafletPolygon\n",
    "import geopandas as gpd\n",
    "from shapely.geometry import shape, mapping\n",
    "from pyproj import Proj, transform\n",
    "from osgeo import gdal\n",
    "\n",
    "# Import raster and image processing packages\n",
    "import rasterio\n",
    "from rasterio.windows import from_bounds\n",
    "import rasterio.mask\n",
    "\n",
    "# Import machine learning and data processing packages\n",
    "import numpy as np\n",
    "import joblib\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV, StratifiedKFold\n",
    "from sklearn.preprocessing import StandardScaler, LabelEncoder\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.metrics import accuracy_score, confusion_matrix, classification_report\n",
    "\n",
    "# Import visualization packages\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.colors as mcolors\n",
    "from matplotlib.backends.backend_pdf import PdfPages\n",
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "# In case you want to start all over again, hit the \"Run\" button on this cell.\n",
    "%reset -f"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 Tutorial: Create training Data\n",
    "<details open>\n",
    "\n",
    "* NOTE: This is not the ideal way of creating training data. The ideal way is through groundtruthing, but for testing purposes, this method should be fine for now.\n",
    "\n",
    "* If you're running into trouble creating your own training data and you want to try out a different method you can follow this tutorial on [Youtube](https://www.youtube.com/watch?v=O-yYfS1EFxg)\n",
    "* Step 1: \n",
    "    - Run the following code block to either a) upload already existing training data, b) use the example data included in this project, or c) create new data.\n",
    "\n",
    "![train_example_1](index/train_example_1.jpg)\n",
    "\n",
    "<sup>*Figure 6: Screenshot from the UI for the training data upload*</sup>\n",
    "\n",
    "* Step 2: \n",
    "    - NOTE: If you integrated your own training data or used the existing one, you can skip this step.\n",
    "    - Search for your area of training, which means the area you want to use to train your model.\n",
    "        * The more similar the area of interest and the area of training are to each other, the more accurate your results will be.\n",
    "    - Scan the map to find representative examples of each class and use the provided map tools to create labeled points or polygons.\n",
    "        * HINT: You can hit \"Toggle Map\" on top of the map to change the layer to a satellite map. This might help with the accuracy of your training data.\n",
    "    - Once you feel like you have enough classes and entries for each class, hit the \"Save GeoJSON\" button on top of the map and navigate to where you want to save your data.\n",
    "        * NOTE: You'll need to name your file \"your_filename.geojson\" to properly save your work!\n",
    "\n",
    "![train_example_2](index/train_example_2.jpg)\n",
    "\n",
    "<sup>*Figure 7: Screenshot from the UI for the training data creation process*</sup>\n",
    "\n",
    "* Step 3:\n",
    "    - The created training data should look something like the following:\n",
    "\n",
    "![train_example_0](index/train_example_0.jpg)\n",
    "\n",
    "<sup>*Figure 8: Screenshot from the training data from Dortmund*</sup>\n",
    "\n",
    "Your training data should now be integrated into the system. To test that, move on to the next code block and hit \"Run\".\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.1 Integrating training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Incorperate training data for the machine learning model\n",
    "\n",
    "\n",
    "# Global variables to store the paths of the TIF and GeoJSON files\n",
    "uploaded_tif_path = None\n",
    "uploaded_geojson_path = None\n",
    "\n",
    "# Define the button widgets\n",
    "button_set_path = widgets.Button(description=\"Set path\")  # Button to set the path for training data\n",
    "button_use_example = widgets.Button(description=\"Example data\")  # Button to use example data\n",
    "button_create_new = widgets.Button(description=\"New data\")  # Button to create new data\n",
    "button_abort = widgets.Button(description=\"Cancel\")  # Button to abort the operation\n",
    "button_save = widgets.Button(description=\"Save GeoJSON\", disabled=True)  # Button to save the GeoJSON\n",
    "button_toggle_map = widgets.ToggleButton(description=\"Toggle Map\", value=False)  # Toggle button to switch map layers\n",
    "\n",
    "# Variable to store the training data path\n",
    "training_data_path = \"\"\n",
    "geojson_data = None\n",
    "\n",
    "# Define the file chooser widget\n",
    "fc = filechooser.FileChooser()\n",
    "\n",
    "# Container for storing drawn features\n",
    "drawn_features = []\n",
    "\n",
    "# Create a map centered on Münster\n",
    "m = Map(center=(51.9606649, 7.6261347), zoom=12)\n",
    "\n",
    "# Create draw control and add it to the map\n",
    "draw_control = DrawControl()\n",
    "m.add_control(draw_control)\n",
    "\n",
    "# Widgets for user input\n",
    "label_input = widgets.Text(description=\"Label:\")  # Text input for label\n",
    "class_id_input = widgets.Text(description=\"ClassID:\")  # Text input for class ID\n",
    "submit_button = widgets.Button(description=\"Submit\")  # Button to submit user input\n",
    "\n",
    "# Container for user input widgets\n",
    "input_widgets = widgets.VBox([label_input, class_id_input, submit_button])\n",
    "display(input_widgets)\n",
    "input_widgets.layout.display = 'none'\n",
    "\n",
    "# Define the satellite layer and OSM layer\n",
    "satellite_layer = basemap_to_tiles(basemaps.Esri.WorldImagery)\n",
    "osm_layer = basemap_to_tiles(basemaps.OpenStreetMap.Mapnik)\n",
    "\n",
    "# Add initial OSM layer to the map\n",
    "m.add_layer(osm_layer)\n",
    "\n",
    "# Handle drawing created event\n",
    "def handle_draw_created(event=None, action=None, geo_json=None):\n",
    "    \"\"\"\n",
    "    Function to handle the event when a feature is drawn on the map.\n",
    "    Parameters:\n",
    "    - event: The event object.\n",
    "    - action: The action performed.\n",
    "    - geo_json: The GeoJSON representation of the drawn feature.\n",
    "    \"\"\"\n",
    "    global current_feature\n",
    "    current_feature = geo_json  # Extract the geojson data from the event\n",
    "    input_widgets.layout.display = 'block'  # Show input widgets for label and class ID\n",
    "\n",
    "# Attach the draw created event to the draw control\n",
    "draw_control.on_draw(handle_draw_created)\n",
    "\n",
    "# Handle submit button click\n",
    "def on_submit_clicked(b):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the submit button is clicked.\n",
    "    Parameters:\n",
    "    - b: The button object representing the submit button.\n",
    "    Global Variables:\n",
    "    - drawn_features: A list to store the drawn features.\n",
    "    - current_feature: A dictionary representing the current feature being submitted.\n",
    "    \"\"\"\n",
    "    global drawn_features, current_feature\n",
    "    current_feature['properties'] = {\n",
    "        'Label': label_input.value,\n",
    "        'ClassID': class_id_input.value,\n",
    "        'fid': len(drawn_features) + 1\n",
    "    }\n",
    "    drawn_features.append(current_feature)\n",
    "    \n",
    "    # Clear inputs\n",
    "    label_input.value = ''\n",
    "    class_id_input.value = ''\n",
    "    input_widgets.layout.display = 'none'\n",
    "    \n",
    "    # Enable save button\n",
    "    button_save.disabled = False\n",
    "\n",
    "submit_button.on_click(on_submit_clicked)\n",
    "\n",
    "# Define the button click event handlers\n",
    "def on_set_path_clicked(b):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the set path button is clicked.\n",
    "    Parameters:\n",
    "    - b: The button object representing the set path button.\n",
    "    \"\"\"\n",
    "    display(fc)\n",
    "    fc.register_callback(on_file_chosen)\n",
    "\n",
    "def on_file_chosen(chooser):\n",
    "    \"\"\"\n",
    "    Function to handle the event when a file is chosen using the file chooser.\n",
    "    Parameters:\n",
    "    - chooser: The file chooser object.\n",
    "    Global Variables:\n",
    "    - training_data_path: The path of the chosen file.\n",
    "    \"\"\"\n",
    "    global training_data_path, uploaded_geojson_path, geojson_data\n",
    "    training_data_path = chooser.selected\n",
    "    uploaded_geojson_path = training_data_path  # Set global variable for GeoJSON path\n",
    "    print(f\"Training data path set to: {training_data_path}\")\n",
    "    load_geojson()\n",
    "\n",
    "def on_use_example_clicked(b):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the use example button is clicked.\n",
    "    Parameters:\n",
    "    - b: The button object representing the use example button.\n",
    "    Global Variables:\n",
    "    - training_data_path: The path of the example data file.\n",
    "    \"\"\"\n",
    "    global training_data_path, uploaded_geojson_path, geojson_data\n",
    "    training_data_path = \"Data/training_dortmund.geojson\"\n",
    "    uploaded_geojson_path = training_data_path  # Set global variable for GeoJSON path\n",
    "    print(f\"Example data path set to: {training_data_path}\")\n",
    "    load_geojson()\n",
    "\n",
    "def on_create_new_clicked(b):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the create new button is clicked.\n",
    "    Parameters:\n",
    "    - b: The button object representing the create new button.\n",
    "    \"\"\"\n",
    "    print(\"Create new data clicked\")\n",
    "    clear_output()\n",
    "    display(button_save)\n",
    "    display(button_toggle_map)\n",
    "    display(m)\n",
    "    display(input_widgets)\n",
    "\n",
    "def on_abort_clicked(b):\n",
    "    # Clear the output to hide the buttons\n",
    "    clear_output()\n",
    "    print(\"User aborted the operation\")\n",
    "\n",
    "def load_geojson():\n",
    "    global geojson_data\n",
    "    try:\n",
    "        geojson_data = gpd.read_file(uploaded_geojson_path)\n",
    "        print(\"GeoJSON data loaded successfully.\")\n",
    "    except Exception as e:\n",
    "        print(f\"Error loading GeoJSON data: {e}\")\n",
    "\n",
    "def save_geojson(geojson_data, path):\n",
    "    \"\"\"\n",
    "    Function to save the GeoJSON data to a file.\n",
    "    Parameters:\n",
    "    - geojson_data: The GeoJSON data to be saved.\n",
    "    - path: The path of the file to save the GeoJSON data.\n",
    "    \"\"\"\n",
    "    with open(path, 'w') as f:\n",
    "        json.dump(geojson_data, f)\n",
    "    print(f\"GeoJSON data saved to: {path}\")\n",
    "\n",
    "def on_save_clicked(b):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the save button is clicked.\n",
    "    Parameters:\n",
    "    - b: The button object representing the save button.\n",
    "    \"\"\"\n",
    "    global drawn_features\n",
    "    clear_output()\n",
    "    fc_save = filechooser.FileChooser()\n",
    "    display(fc_save)\n",
    "    fc_save.register_callback(on_save_path_chosen)\n",
    "\n",
    "def on_save_path_chosen(chooser):\n",
    "    \"\"\"\n",
    "    Function to handle the event when a save path is chosen using the file chooser.\n",
    "    Parameters:\n",
    "    - chooser: The file chooser object.\n",
    "    \"\"\"\n",
    "    path = chooser.selected\n",
    "    geojson_data = {\n",
    "        \"type\": \"FeatureCollection\",\n",
    "        \"features\": drawn_features\n",
    "    }\n",
    "    save_geojson(geojson_data, path)\n",
    "    load_geojson()\n",
    "\n",
    "# Handle map toggle\n",
    "def on_toggle_map_clicked(change):\n",
    "    \"\"\"\n",
    "    Function to handle the event when the map toggle button is clicked.\n",
    "    Parameters:\n",
    "    - change: The change event object.\n",
    "    \"\"\"\n",
    "    if change['new']:\n",
    "        m.remove_layer(osm_layer)\n",
    "        m.add_layer(satellite_layer)\n",
    "    else:\n",
    "        m.remove_layer(satellite_layer)\n",
    "        m.add_layer(osm_layer)\n",
    "\n",
    "button_toggle_map.observe(on_toggle_map_clicked, 'value')\n",
    "\n",
    "# Assign the event handlers to the buttons\n",
    "button_set_path.on_click(on_set_path_clicked)\n",
    "button_use_example.on_click(on_use_example_clicked)\n",
    "button_create_new.on_click(on_create_new_clicked)\n",
    "button_abort.on_click(on_abort_clicked)\n",
    "button_save.on_click(on_save_clicked)\n",
    "\n",
    "# Display the buttons\n",
    "display(button_set_path, button_use_example, button_create_new, button_abort)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3.2 Validating the GeoJSON data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "6de600de89b54378af7477d40124769c",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Map(center=[51.56521874857406, 7.42048862632758], controls=(ZoomControl(options=['position', 'zoom_in_text', '…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Display the map with the GeoJSON data\n",
    "\n",
    "# Function to center the map on a random point from the GeoJSON data\n",
    "def get_random_center(gdf):\n",
    "    \"\"\"\n",
    "    Function to get a random center point from the GeoDataFrame.\n",
    "    Parameters:\n",
    "    - gdf: The GeoDataFrame containing the training data.\n",
    "    Returns:\n",
    "    - A tuple representing the coordinates of the random center point.\n",
    "    \"\"\"\n",
    "    random_feature = gdf.sample(1).geometry.iloc[0]\n",
    "    \n",
    "    if random_feature.geom_type == 'Point':\n",
    "        return random_feature.y, random_feature.x\n",
    "    elif random_feature.geom_type in ['Polygon', 'MultiPolygon']:\n",
    "        centroid = random_feature.centroid\n",
    "        return centroid.y, centroid.x\n",
    "    else:\n",
    "        raise ValueError(\"Unsupported geometry type\")\n",
    "\n",
    "# Check if geojson_data is already loaded\n",
    "if 'geojson_data' in globals() and isinstance(geojson_data, gpd.GeoDataFrame):\n",
    "    # Get a random center point from the geojson_data\n",
    "    try:\n",
    "        center_point = get_random_center(geojson_data)\n",
    "    except ValueError as e:\n",
    "        print(f\"Error in geometry type: {e}\")\n",
    "        center_point = (51.9606649, 7.6261347)  # Default to Münster if there's an error\n",
    "else:\n",
    "    print(\"No training data loaded. Please load the training data in the first notebook.\")\n",
    "    center_point = (51.9606649, 7.6261347)  # Default to Münster if no data is loaded\n",
    "\n",
    "# Create a map centered on the random point\n",
    "m = Map(center=center_point, zoom=12)\n",
    "\n",
    "# Define the satellite layer and OSM layer\n",
    "satellite_layer = basemap_to_tiles(basemaps.Esri.WorldImagery)\n",
    "osm_layer = basemap_to_tiles(basemaps.OpenStreetMap.Mapnik)\n",
    "\n",
    "# Add initial OSM layer to the map\n",
    "m.add_layer(osm_layer)\n",
    "\n",
    "def display_geojson():\n",
    "    \"\"\"\n",
    "    Function to display the GeoJSON data on the map.\n",
    "    \"\"\"\n",
    "    if geojson_data is None:\n",
    "        print(\"No training data loaded. Please load the training data first.\")\n",
    "    else:\n",
    "        geojson_layer = GeoJSON(data=json.loads(geojson_data.to_json()))\n",
    "        m.add_layer(geojson_layer)\n",
    "        display(m)\n",
    "\n",
    "# Display the map with GeoJSON data\n",
    "display_geojson()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4 Tutorial: Fetch Sentinel-2 data\n",
    "<details open>\n",
    "\n",
    "Next up, we’ll download and integrate Sentinel-2 data. If you already have remote sensing data, you can skip this tutorial and hit “run” on the next code block.\n",
    "\n",
    "* Step 0: Visit the Copernicus Browser and sign up/in.\n",
    "* Step 1: \n",
    "    - In the browser, switch to the \"Search\" tab.\n",
    "    - Set the search to \"Sentinel-2\", \"L2A\", and to your desired cloud coverage (we recommend no more than 20%).\n",
    "    - At the bottom, you'll need to determine the time period from which you want to gather the data.\n",
    "    - On the right side, you have to set your area of interest using the provided map tools.\n",
    "    - If you set everything correctly, hit the \"Search\" button at the bottom of the \"Search\" tab and wait for the results.\n",
    "\n",
    "![Copernicus_Example_1](index/cop_example_1.jpg)\n",
    "\n",
    "<sup>*Figure 3: Screenshot from the Copernicus browser, showing how to set the \"Search\" tab*</sup>\n",
    "    \n",
    "* Step 2:\n",
    "    - If the search was successful, you'll see which images overlap your area of interest. Click on the one that fits your needs/interests the most.\n",
    "        * If there isn't a single image that completely covers your area of interest, you'll need to create a mosaic. More about that later. In that case, you should download every image covering your area of interest.\n",
    "    - You'll be greeted with a new window showing previews of the available data. Hit the download button on the one you're confident has the least amount of clouds.\n",
    "    - Wait for the download to finish.\n",
    "    - NOTE: If you can't find images, go back to the \"Search\" tab from Step 1 and change either the cloud coverage or the timeframe for your data.\n",
    "\n",
    "![Copernicus_Example_2](index/cop_example_2.jpg)\n",
    "\n",
    "<sup>*Figure 4: Screenshot from the Copernicus browser, showing how to set the \"Download\" section*</sup>\n",
    "\n",
    "* Step 3:\n",
    "    - Once downloaded, you'll have a .zip file containing your satellite data. Extract that file and navigate to \"GRANULE/.../IMG_DATA/R10m\".\n",
    "    - Here you'll find single files for the \"Red\", \"Green\", and \"Blue\" bands of the Sentinel-2 satellite, as well as the \"Near infrared\" bands.\n",
    "    - Copy the \"..._B02_...\", \"..._B03_...\", \"..._B04_...\", and \"..._B08_...\" files into a separate folder so you can easily find them.\n",
    "    - Continue with the rest of the tutorial; we'll need these files later.\n",
    "\n",
    "![Copernicus_Example_3](index/cop_example_3.jpg)\n",
    "\n",
    "<sup>*Figure 5: Screenshot from the Windows file browser, showing the file selection for the \"Red\", \"Green\", \"Blue\", and \"Near infrared\" bands of the downloaded data*</sup>\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.1 Integrating Sentinel-2 data of the Area of Interest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.1 Incorperating and processing multiple bands"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Incorpate the Sentinel-2 data for the machine learning model in form of bands in multiple files for the Area of Interest (AOI)\n",
    "\n",
    "aoi_path = None\n",
    "class Sentinel2Processor:\n",
    "    def __init__(self):\n",
    "        self.bbox = None\n",
    "        self.blue_path = None\n",
    "        self.green_path = None\n",
    "        self.red_path = None\n",
    "        self.nir_path = None\n",
    "        self.create_widgets()\n",
    "\n",
    "    def create_widgets(self):\n",
    "        self.question1()\n",
    "\n",
    "    def question1(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user if they have Sentinel-2 data in the form of bands in multiple files.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have Sentinel-2 data in the form of bands in multiple files?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1_yes)\n",
    "        no_button.on_click(self.q1_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_yes(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks \"Yes\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you need / want to crop your raster data?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q2_yes)\n",
    "        no_button.on_click(self.q2_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_no(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks \"No\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Please use the tutorial to gather Sentinel-2 data\"))\n",
    "\n",
    "    def q2_yes(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks \"Yes\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Draw a rectangle on the map to select the bounding box\"))\n",
    "\n",
    "        m = Map(center=(51.23, 9.35), zoom=9, basemap=basemap_to_tiles(basemaps.OpenStreetMap.Mapnik))\n",
    "        draw_control = DrawControl(rectangle={'shapeOptions': {'color': '#0000FF'}})\n",
    "        m.add_control(draw_control)\n",
    "        display(m)\n",
    "\n",
    "        finish_button = widgets.Button(description=\"Finish\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        def handle_draw(target, action, geo_json):\n",
    "            \"\"\"\n",
    "            Function to handle the event when a feature is drawn on the map.\n",
    "            \"\"\"\n",
    "            bbox = shape(geo_json['geometry']).bounds\n",
    "            self.bbox = self.convert_bbox_to_utm(bbox)\n",
    "            rect = LeafletPolygon(locations=[[[bbox[1], bbox[0]], [bbox[1], bbox[2]], [bbox[3], bbox[2]], [bbox[3], bbox[0]], [bbox[1], bbox[0]]]], color=\"blue\", fill_opacity=0.5)\n",
    "            m.add_layer(rect)\n",
    "            print(f\"BBOX coordinates (Lat/Lon): {bbox}\")\n",
    "            print(f\"BBOX coordinates (UTM): {self.bbox}\")\n",
    "\n",
    "        def finish(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks \"Finish\" in response to the second question.\n",
    "            \"\"\"\n",
    "            if self.bbox:\n",
    "                self.q3()\n",
    "            else:\n",
    "                display(widgets.Label(\"No bounding box selected.\"))\n",
    "\n",
    "        def cancel(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks \"Cancel\" in response to the second question.\n",
    "            \"\"\"\n",
    "            self.clear_output()\n",
    "            display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "        draw_control.on_draw(handle_draw)\n",
    "        finish_button.on_click(finish)\n",
    "        cancel_button.on_click(cancel)\n",
    "\n",
    "        display(finish_button, cancel_button)\n",
    "\n",
    "    def q2_no(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks \"No\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.q3()\n",
    "\n",
    "    def q3(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user for the path to the blue band data.\n",
    "        \"\"\"\n",
    "        self.ask_for_band(\"blue\", \"B02\")\n",
    "\n",
    "    def ask_for_band(self, color, code):\n",
    "        \"\"\"\n",
    "        Function to ask the user for the path to a specific band data.\n",
    "        Parameters:\n",
    "        - color: The color of the band.\n",
    "        - code: The code of the band.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(f\"Please set the path to the data of the {color} band ('_ _ _ {code} _ _ _ .jp2')\")\n",
    "        file_chooser = filechooser.FileChooser()\n",
    "\n",
    "        upload_button = widgets.Button(description=\"Upload\")\n",
    "\n",
    "        def handle_upload(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks \"Upload\" to select a file.\n",
    "            \"\"\"\n",
    "            selected_file = file_chooser.selected\n",
    "            if selected_file and code in selected_file:\n",
    "                setattr(self, f\"{color}_path\", selected_file)\n",
    "                if color == \"blue\":\n",
    "                    self.ask_for_band(\"green\", \"B03\")\n",
    "                elif color == \"green\":\n",
    "                    self.ask_for_band(\"red\", \"B04\")\n",
    "                elif color == \"red\":\n",
    "                    self.ask_for_band(\"nir\", \"B08\")\n",
    "                elif color == \"nir\":\n",
    "                    self.ask_for_final_confirmation()\n",
    "            else:\n",
    "                self.clear_output()\n",
    "                display(widgets.Label(f\"Data integration of the {color} band not successful\"))\n",
    "\n",
    "        upload_button.on_click(handle_upload)\n",
    "\n",
    "        display(question, file_chooser, upload_button)\n",
    "\n",
    "    def ask_for_final_confirmation(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user for final confirmation before starting the processing.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"All bands successfully incorporated. Processing now ready - do you want to continue?\"))\n",
    "        \n",
    "        continue_button = widgets.Button(description=\"Continue\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        continue_button.on_click(self.process_bands)\n",
    "        cancel_button.on_click(self.cancel_operation)\n",
    "\n",
    "        display(continue_button, cancel_button)\n",
    "\n",
    "    def cancel_operation(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks \"Cancel\" in response to the final confirmation.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "    def clear_output(self):\n",
    "        \"\"\"\n",
    "        Function to clear the output area.\n",
    "        \"\"\"\n",
    "        clear_output()\n",
    "\n",
    "    def process_bands(self, b=None):\n",
    "        \"\"\"\n",
    "        Function to process the selected bands.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Starting the processing. This might take a while.\"))\n",
    "\n",
    "        if not self.bbox:\n",
    "            display(widgets.Label(\"No bounding box selected, using default bbox.\"))\n",
    "            bbox = (394861, 5746419, 420134, 5767397)\n",
    "        else:\n",
    "            bbox = self.bbox\n",
    "\n",
    "        blue_tif = self.convert_jp2_to_tif(self.blue_path)\n",
    "        green_tif = self.convert_jp2_to_tif(self.green_path)\n",
    "        red_tif = self.convert_jp2_to_tif(self.red_path)\n",
    "        nir_tif = self.convert_jp2_to_tif(self.nir_path)\n",
    "\n",
    "        blue, _ = self.load_and_clip_raster(blue_tif, bbox)\n",
    "        green, _ = self.load_and_clip_raster(green_tif, bbox)\n",
    "        red, _ = self.load_and_clip_raster(red_tif, bbox)\n",
    "        nir, _ = self.load_and_clip_raster(nir_tif, bbox)\n",
    "\n",
    "        if blue.size == 0 or green.size == 0 or red.size == 0 or nir.size == 0:\n",
    "            self.clear_output()\n",
    "            display(widgets.Label(\"Error: One of the bands is empty after clipping. Please check the bounding box and try again.\"))\n",
    "            return\n",
    "\n",
    "        # Stack the bands to create a composite image\n",
    "        global Sen2AOI\n",
    "        Sen2AOI = np.dstack((red, green, blue, nir))\n",
    "\n",
    "        display(widgets.Label(\"Processing successful, you can now go on to the next code block!\"))\n",
    "\n",
    "    def convert_jp2_to_tif(self, input_path):\n",
    "        \"\"\"\n",
    "        Function to convert a JP2 file to a TIFF file.\n",
    "        Parameters:\n",
    "        - input_path: The path to the input JP2 file.\n",
    "        Returns:\n",
    "        - The path to the output TIFF file.\n",
    "        \"\"\"\n",
    "        in_image = gdal.Open(input_path)\n",
    "        driver = gdal.GetDriverByName(\"GTiff\")\n",
    "        out_tif_path = input_path.replace('.jp2', '.tif')\n",
    "        out_image = driver.CreateCopy(out_tif_path, in_image, 0)\n",
    "        in_image = None\n",
    "        out_image = None\n",
    "        return out_tif_path\n",
    "\n",
    "    def load_and_clip_raster(self, file_path, bbox):\n",
    "        \"\"\"\n",
    "        Function to load and clip a raster image based on a bounding box.\n",
    "        Parameters:\n",
    "        - file_path: The path to the raster image file.\n",
    "        - bbox: The bounding box coordinates.\n",
    "        Returns:\n",
    "        - The clipped image and its transformation.\n",
    "        \"\"\"\n",
    "        with rasterio.open(file_path, 'r') as src:\n",
    "            print(f\"Loading with bbox: {bbox}\")\n",
    "            window = from_bounds(*bbox, src.transform)\n",
    "            print(f\"Window: {window}\")\n",
    "            clipped_image = src.read(1, window=window)\n",
    "            print(f\"Dimensions after clipping: {clipped_image.shape}\")\n",
    "            transform = src.window_transform(window)\n",
    "        return clipped_image, transform\n",
    "\n",
    "    def convert_bbox_to_utm(self, bbox):\n",
    "        \"\"\"\n",
    "        Function to convert a bounding box from WGS84 to UTM Zone 32N.\n",
    "        Parameters:\n",
    "        - bbox: The input bounding box.\n",
    "        Returns:\n",
    "        - The converted bounding box in UTM coordinates.\n",
    "        \"\"\"\n",
    "        in_proj = Proj(init='epsg:4326')  # WGS84\n",
    "        out_proj = Proj(init='epsg:32632')  # UTM Zone 32N\n",
    "\n",
    "        x_min, y_min = transform(in_proj, out_proj, bbox[0], bbox[1])\n",
    "        x_max, y_max = transform(in_proj, out_proj, bbox[2], bbox[3])\n",
    "        return (x_min, y_min, x_max, y_max)\n",
    "\n",
    "# Run the application\n",
    "interest = Sentinel2Processor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.1.2 Incorperating and processing single file data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Incorporating the Sentinel-2 data for the machine learning model in the form of a single .TIF file for the Area of Interest (AOI)\n",
    "\n",
    "aoi_path = None\n",
    "class Sentinel2Processor:\n",
    "    def __init__(self):\n",
    "        self.bbox = None\n",
    "        self.file_path = None\n",
    "        self.aoi_path = None  # AOI path\n",
    "        self.create_widgets()\n",
    "\n",
    "    def create_widgets(self):\n",
    "        \"\"\"Create and display the initial widgets for user interaction.\"\"\"\n",
    "        self.question1()\n",
    "\n",
    "    def question1(self):\n",
    "        \"\"\"\n",
    "        Ask the user if they have Sentinel-2 data in the form of a single .TIF file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have Sentinel-2 data in the form of a single file containing multiple bands?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1_yes)\n",
    "        no_button.on_click(self.q1_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you need / want to crop your raster data?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q2_yes)\n",
    "        no_button.on_click(self.q2_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_no(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Please use the tutorial to gather Sentinel-2 data.\"))\n",
    "\n",
    "    def q2_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Draw a rectangle on the map to select the bounding box\"))\n",
    "\n",
    "        self.map_widget = Map(center=(51.23, 9.35), zoom=9, basemap=basemap_to_tiles(basemaps.OpenStreetMap.Mapnik))\n",
    "        draw_control = DrawControl(rectangle={'shapeOptions': {'color': '#0000FF'}})\n",
    "        self.map_widget.add_control(draw_control)\n",
    "        display(self.map_widget)\n",
    "\n",
    "        finish_button = widgets.Button(description=\"Finish\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        draw_control.on_draw(self.handle_draw)\n",
    "        finish_button.on_click(self.finish_bbox)\n",
    "        cancel_button.on_click(self.cancel_bbox)\n",
    "\n",
    "        display(finish_button, cancel_button)\n",
    "\n",
    "    def q2_no(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.q3()\n",
    "\n",
    "    def handle_draw(self, target, action, geo_json):\n",
    "        \"\"\"\n",
    "        Handle the event when a feature is drawn on the map.\n",
    "        \"\"\"\n",
    "        bbox = shape(geo_json['geometry']).bounds\n",
    "        self.bbox = self.convert_bbox_to_utm(bbox)\n",
    "        rect = LeafletPolygon(locations=[[[bbox[1], bbox[0]], [bbox[1], bbox[2]], [bbox[3], bbox[2]], [bbox[3], bbox[0]], [bbox[1], bbox[0]]]], color=\"blue\", fill_opacity=0.5)\n",
    "        self.map_widget.add_layer(rect)\n",
    "        print(f\"BBOX coordinates (Lat/Lon): {bbox}\")\n",
    "        print(f\"BBOX coordinates (UTM): {self.bbox}\")\n",
    "\n",
    "    def finish_bbox(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Finish\" after drawing the bounding box.\n",
    "        \"\"\"\n",
    "        if self.bbox:\n",
    "            self.q3()\n",
    "        else:\n",
    "            display(widgets.Label(\"No bounding box selected.\"))\n",
    "\n",
    "    def cancel_bbox(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Cancel\" in response to the bounding box question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "    def q3(self):\n",
    "        \"\"\"\n",
    "        Ask the user if they have a .TIF file or a .grd file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have a .TIF file or .grd?\")\n",
    "        tif_button = widgets.Button(description=\".TIF\")\n",
    "        grd_button = widgets.Button(description=\".grd\")\n",
    "\n",
    "        tif_button.on_click(self.q3_tif)\n",
    "        grd_button.on_click(self.q3_grd)\n",
    "\n",
    "        display(question, tif_button, grd_button)\n",
    "\n",
    "    def q3_tif(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \".TIF\" in response to the third question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        self.ask_for_file(\".tif\")\n",
    "\n",
    "    def q3_grd(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \".grd\" in response to the third question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"The used Python modules can't incorporate .grd files. Do you want to convert your data to the .TIF file format?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q4_yes)\n",
    "        no_button.on_click(self.q4_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q4_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the fourth question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        self.ask_for_file(\".grd\")\n",
    "\n",
    "    def q4_no(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the fourth question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation. Please use the tutorial to gather Sentinel-2 data.\"))\n",
    "\n",
    "    def ask_for_file(self, extension):\n",
    "        \"\"\"\n",
    "        Ask the user for the path to a file of a specific type.\n",
    "        Parameters:\n",
    "        - extension: The file extension to look for (e.g., \".tif\" or \".grd\").\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(f\"Please set the path to the data of the file with extension '{extension}'\")\n",
    "        file_chooser = filechooser.FileChooser()\n",
    "\n",
    "        upload_button = widgets.Button(description=\"Upload\")\n",
    "\n",
    "        def handle_upload(b):\n",
    "            \"\"\"\n",
    "            Handle the event when the user clicks \"Upload\" to select a file.\n",
    "            \"\"\"\n",
    "            selected_file = file_chooser.selected\n",
    "            if selected_file and selected_file.endswith(extension):\n",
    "                self.file_path = selected_file\n",
    "                self.aoi_path = selected_file  # Set AOI path\n",
    "                global aoi_path\n",
    "                print(f\"File path set to: {self.file_path}\")\n",
    "                if extension == \".grd\":\n",
    "                    self.convert_grd_to_tif()\n",
    "                else:\n",
    "                    self.process_tiff_and_plot()\n",
    "            else:\n",
    "                display(widgets.Label(f\"Data integration not successful. File must be a {extension} file.\"))\n",
    "\n",
    "        upload_button.on_click(handle_upload)\n",
    "\n",
    "        display(question, file_chooser, upload_button)\n",
    "\n",
    "    def convert_grd_to_tif(self):\n",
    "        \"\"\"\n",
    "        Convert a .grd file to .tif format and process the .tif file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        output_tiff_path = self.file_path.replace('.grd', '.tif')\n",
    "\n",
    "        try:\n",
    "            self._convert_grd_to_tiff(self.file_path, output_tiff_path)\n",
    "            display(widgets.Label(\"Data conversion successful\"))\n",
    "            self.file_path = output_tiff_path\n",
    "            self.process_tiff_and_plot()\n",
    "        except Exception as e:\n",
    "            display(widgets.Label(\"Data integration and transformation not successful\"))\n",
    "\n",
    "    def _convert_grd_to_tiff(self, input_grd_path, output_tiff_path):\n",
    "        \"\"\"\n",
    "        Convert a GRD file to a TIF file using GDAL.\n",
    "        \"\"\"\n",
    "        dataset = gdal.Open(input_grd_path)\n",
    "        if dataset is None:\n",
    "            raise FileNotFoundError(f\"Cannot open {input_grd_path}\")\n",
    "\n",
    "        geo_transform = dataset.GetGeoTransform()\n",
    "        projection = dataset.GetProjection()\n",
    "\n",
    "        driver = gdal.GetDriverByName('GTiff')\n",
    "        if driver is None:\n",
    "            raise RuntimeError(\"GTiff driver is not available\")\n",
    "\n",
    "        output_dataset = driver.Create(output_tiff_path, dataset.RasterXSize, dataset.RasterYSize, dataset.RasterCount, gdal.GDT_Float32)\n",
    "        output_dataset.SetGeoTransform(geo_transform)\n",
    "        output_dataset.SetProjection(projection)\n",
    "\n",
    "        for band in range(1, dataset.RasterCount + 1):\n",
    "            input_band = dataset.GetRasterBand(band)\n",
    "            output_band = output_dataset.GetRasterBand(band)\n",
    "            data = input_band.ReadAsArray()\n",
    "            output_band.WriteArray(data)\n",
    "\n",
    "        dataset = None\n",
    "        output_dataset = None\n",
    "\n",
    "    def process_tiff_and_plot(self):\n",
    "        \"\"\"\n",
    "        Process the .tif file, display the number of bands, and plot the RGB composite.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        dataset = gdal.Open(self.file_path)\n",
    "        if dataset is None:\n",
    "            display(widgets.Label(\"Failed to open the TIF file.\"))\n",
    "            return\n",
    "\n",
    "        num_bands = dataset.RasterCount\n",
    "        display(widgets.Label(f\"Number of bands: {num_bands}\"))\n",
    "\n",
    "        if num_bands >= 3:\n",
    "            self.plot_rgb_composite(dataset)\n",
    "\n",
    "    def plot_rgb_composite(self, dataset):\n",
    "        \"\"\"\n",
    "        Plot an RGB composite of the Sentinel-2 image using the first 4 bands.\n",
    "        \"\"\"\n",
    "        global Sen2AOI\n",
    "        bands = [dataset.GetRasterBand(i + 1).ReadAsArray() for i in range(4)]\n",
    "        \n",
    "        for i, band in enumerate(bands[:4]):\n",
    "            print(f\"Band {i+1} min: {band.min()}, max: {band.max()}\")\n",
    "\n",
    "        Sen2AOI = np.stack(bands[:4], axis=-1)\n",
    "        print(Sen2AOI.shape)\n",
    "\n",
    "        Sen2AOI = Sen2AOI.astype(np.float32)\n",
    "        for i in range(Sen2AOI.shape[-1]):\n",
    "            Sen2AOI[..., i] = (Sen2AOI[..., i] - Sen2AOI[..., i].min()) / (Sen2AOI[..., i].max() - Sen2AOI[..., i].min())\n",
    "\n",
    "        plt.imshow(Sen2AOI)\n",
    "        plt.title('Sentinel-2 Stacked Image')\n",
    "        plt.show()\n",
    "\n",
    "    def convert_bbox_to_utm(self, bbox):\n",
    "        \"\"\"\n",
    "        Convert latitude/longitude bounding box to UTM coordinates.\n",
    "        \"\"\"\n",
    "        in_proj = Proj(init='epsg:4326')  # WGS84\n",
    "        out_proj = Proj(init='epsg:32632')  # UTM Zone 32N\n",
    "\n",
    "        x_min, y_min = transform(in_proj, out_proj, bbox[0], bbox[1])\n",
    "        x_max, y_max = transform(in_proj, out_proj, bbox[2], bbox[3])\n",
    "        return (x_min, y_min, x_max, y_max)\n",
    "\n",
    "    def clear_output(self):\n",
    "        \"\"\"\n",
    "        Clear the output area to remove previous widgets and information.\n",
    "        \"\"\"\n",
    "        clear_output(wait=True)\n",
    "\n",
    "\n",
    "# Run the application\n",
    "interest = Sentinel2Processor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 4.2 Integrating Sentinel-2 data of the Area of Training\n",
    "<details>\n",
    "    <summary>Click me for more Info</summary>\n",
    "    \n",
    "* Now that we have processed the Sentinel-2 data for the Area of Interest, we need to do the same for the Training Area. Run the code whenever you're ready, and we'll guide you through the process again.\n",
    "* Remember: The Training Area is where your training data is located and where your model will be trained. The Area of Interest is where the algorithm will predict land use/land cover.\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true
   },
   "source": [
    "#### 4.2.1 Incorperating and processing multiple bands for the Area of Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "hidden": true
   },
   "outputs": [],
   "source": [
    "# Incorporating the Sentinel-2 data for the machine learning model in form of mu file for the Area of Training (AOT)\n",
    "\n",
    "class Sentinel2Processor:\n",
    "    def __init__(self):\n",
    "        self.bbox = None\n",
    "        self.blue_path = None\n",
    "        self.green_path = None\n",
    "        self.red_path = None\n",
    "        self.nir_path = None\n",
    "        self.create_widgets()\n",
    "\n",
    "    def create_widgets(self):\n",
    "        self.question1()\n",
    "\n",
    "    def question1(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user if they have Sentinel-2 data in the form of bands in multiple files.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have Sentinel-2 data in the form of bands in multiple files?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1_yes)\n",
    "        no_button.on_click(self.q1_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_yes(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'Yes' in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Is your Area of Training, as in the place where your Training data are located, the same as your Area of Interest, for which you want to create a prediction map?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1a_yes)\n",
    "        no_button.on_click(self.q1a_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_no(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'No' in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you need / want to crop your raster data?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q2_yes)\n",
    "        no_button.on_click(self.q2_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1a_yes(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'Yes' in response to the second question about Area of Training.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        try:\n",
    "            global Sen2AOT\n",
    "            if 'Sen2AOI' in globals():\n",
    "                print(\"Sen2AOI is present in globals.\")\n",
    "                print(f\"Type of Sen2AOI: {type(Sen2AOI)}\")\n",
    "                print(f\"Size of Sen2AOI: {Sen2AOI.shape}\")\n",
    "                if isinstance(Sen2AOI, np.ndarray):\n",
    "                    # Copying the data\n",
    "                    Sen2AOT = Sen2AOI.copy()\n",
    "                    display(widgets.Label(\"Data for the Area of Interest found. Data copied to Sen2AOT.\"))\n",
    "                else:\n",
    "                    display(widgets.Label(\"Data for the Area of Interest is not a valid ndarray.\"))\n",
    "            else:\n",
    "                display(widgets.Label(\"No data for your Area of Interest found. Please upload these data first.\"))\n",
    "        except Exception as e:\n",
    "            display(widgets.Label(f\"An error occurred: {e}\"))\n",
    "\n",
    "    def q1a_no(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'No' in response to the second question about Area of Training.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Do you need / want to crop your raster data?\"))\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q2_yes)\n",
    "        no_button.on_click(self.q2_no)\n",
    "\n",
    "        display(yes_button, no_button)\n",
    "\n",
    "    def q2_yes(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'Yes' in response to the second question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Draw a rectangle on the map to select the bounding box\"))\n",
    "\n",
    "        m = Map(center=(51.23, 9.35), zoom=9, basemap=basemap_to_tiles(basemaps.OpenStreetMap.Mapnik))\n",
    "        draw_control = DrawControl(rectangle={'shapeOptions': {'color': '#0000FF'}})\n",
    "        m.add_control(draw_control)\n",
    "        display(m)\n",
    "\n",
    "        finish_button = widgets.Button(description=\"Finish\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        def handle_draw(target, action, geo_json):\n",
    "            \"\"\"\n",
    "            Function to handle the event when a feature is drawn on the map.\n",
    "            \"\"\"\n",
    "            bbox = shape(geo_json['geometry']).bounds\n",
    "            self.bbox = self.convert_bbox_to_utm(bbox)\n",
    "            rect = LeafletPolygon(locations=[[[bbox[1], bbox[0]], [bbox[1], bbox[2]], [bbox[3], bbox[2]], [bbox[3], bbox[0]], [bbox[1], bbox[0]]]], color=\"blue\", fill_opacity=0.5)\n",
    "            m.add_layer(rect)\n",
    "            print(f\"BBOX coordinates (Lat/Lon): {bbox}\")\n",
    "            print(f\"BBOX coordinates (UTM): {self.bbox}\")\n",
    "\n",
    "        def finish(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks 'Finish' in response to the second question.\n",
    "            \"\"\"\n",
    "            if self.bbox:\n",
    "                self.q3()\n",
    "            else:\n",
    "                display(widgets.Label(\"No bounding box selected.\"))\n",
    "\n",
    "        def cancel(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks 'Cancel' in response to the second question.\n",
    "            \"\"\"\n",
    "            self.clear_output()\n",
    "            display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "        draw_control.on_draw(handle_draw)\n",
    "        finish_button.on_click(finish)\n",
    "        cancel_button.on_click(cancel)\n",
    "\n",
    "        display(finish_button, cancel_button)\n",
    "\n",
    "    def q2_no(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'No' in response to the second question.\n",
    "        \"\"\"\n",
    "        self.q3()\n",
    "\n",
    "    def q3(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user for the path to the blue band data.\n",
    "        \"\"\"\n",
    "        self.ask_for_band(\"blue\", \"B02\")\n",
    "\n",
    "    def ask_for_band(self, color, code):\n",
    "        \"\"\"\n",
    "        Function to ask the user for the path to a specific band data.\n",
    "        Parameters:\n",
    "        - color: The color of the band.\n",
    "        - code: The code of the band.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(f\"Please set the path to the data of the {color} band ('_ _ _ {code} _ _ _ .jp2')\")\n",
    "        file_chooser = filechooser.FileChooser()\n",
    "\n",
    "        upload_button = widgets.Button(description=\"Upload\")\n",
    "\n",
    "        def handle_upload(b):\n",
    "            \"\"\"\n",
    "            Function to handle the event when the user clicks 'Upload' to select a file.\n",
    "            \"\"\"\n",
    "            selected_file = file_chooser.selected\n",
    "            if selected_file and code in selected_file:\n",
    "                setattr(self, f\"{color}_path\", selected_file)\n",
    "                if color == \"blue\":\n",
    "                    self.ask_for_band(\"green\", \"B03\")\n",
    "                elif color == \"green\":\n",
    "                    self.ask_for_band(\"red\", \"B04\")\n",
    "                elif color == \"red\":\n",
    "                    self.ask_for_band(\"nir\", \"B08\")\n",
    "                elif color == \"nir\":\n",
    "                    self.ask_for_final_confirmation()\n",
    "            else:\n",
    "                self.clear_output()\n",
    "                display(widgets.Label(f\"Data integration of the {color} band not successful\"))\n",
    "\n",
    "        upload_button.on_click(handle_upload)\n",
    "\n",
    "        display(question, file_chooser, upload_button)\n",
    "\n",
    "    def ask_for_final_confirmation(self):\n",
    "        \"\"\"\n",
    "        Function to ask the user for final confirmation before starting the processing.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"All bands successfully incorporated. Processing now ready - do you want to continue?\"))\n",
    "        \n",
    "        continue_button = widgets.Button(description=\"Continue\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        continue_button.on_click(self.process_bands)\n",
    "        cancel_button.on_click(self.cancel_operation)\n",
    "\n",
    "        display(continue_button, cancel_button)\n",
    "\n",
    "    def cancel_operation(self, b):\n",
    "        \"\"\"\n",
    "        Function to handle the event when the user clicks 'Cancel' in response to the final confirmation.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "    def clear_output(self):\n",
    "        \"\"\"\n",
    "        Function to clear the output area.\n",
    "        \"\"\"\n",
    "        clear_output()\n",
    "\n",
    "    def process_bands(self, b=None):\n",
    "        \"\"\"\n",
    "        Function to process the selected bands.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Starting the processing. This might take a while.\"))\n",
    "\n",
    "        if not self.bbox:\n",
    "            display(widgets.Label(\"No bounding box selected, using default bbox.\"))\n",
    "            bbox = (394861, 5746419, 420134, 5767397)\n",
    "        else:\n",
    "            bbox = self.bbox\n",
    "\n",
    "        blue_tif = self.convert_jp2_to_tif(self.blue_path)\n",
    "        green_tif = self.convert_jp2_to_tif(self.green_path)\n",
    "        red_tif = self.convert_jp2_to_tif(self.red_path)\n",
    "        nir_tif = self.convert_jp2_to_tif(self.nir_path)\n",
    "\n",
    "        blue, _ = self.load_and_clip_raster(blue_tif, bbox)\n",
    "        green, _ = self.load_and_clip_raster(green_tif, bbox)\n",
    "        red, _ = self.load_and_clip_raster(red_tif, bbox)\n",
    "        nir, _ = self.load_and_clip_raster(nir_tif, bbox)\n",
    "\n",
    "        if blue.size == 0 or green.size == 0 or red.size == 0 or nir.size == 0:\n",
    "            self.clear_output()\n",
    "            display(widgets.Label(\"Error: One of the bands is empty after clipping. Please check the bounding box and try again.\"))\n",
    "            return\n",
    "\n",
    "        # Stack the bands to create a composite image\n",
    "        global Sen2AOT\n",
    "        Sen2AOT = np.dstack((red, green, blue, nir))\n",
    "\n",
    "        display(widgets.Label(\"Processing successful, you can now go on to the next code block!\"))\n",
    "\n",
    "    def convert_jp2_to_tif(self, input_path):\n",
    "        \"\"\"\n",
    "        Function to convert a JP2 file to a TIFF file.\n",
    "        Parameters:\n",
    "        - input_path: The path to the input JP2 file.\n",
    "        Returns:\n",
    "        - The path to the output TIFF file.\n",
    "        \"\"\"\n",
    "        in_image = gdal.Open(input_path)\n",
    "        driver = gdal.GetDriverByName(\"GTiff\")\n",
    "        out_tif_path = input_path.replace('.jp2', '.tif')\n",
    "        out_image = driver.CreateCopy(out_tif_path, in_image, 0)\n",
    "        in_image = None\n",
    "        out_image = None\n",
    "        return out_tif_path\n",
    "\n",
    "    def load_and_clip_raster(self, file_path, bbox):\n",
    "        \"\"\"\n",
    "        Function to load and clip a raster image based on a bounding box.\n",
    "        Parameters:\n",
    "        - file_path: The path to the raster image file.\n",
    "        - bbox: The bounding box coordinates.\n",
    "        Returns:\n",
    "        - The clipped image and its transformation.\n",
    "        \"\"\"\n",
    "        with rasterio.open(file_path, 'r') as src:\n",
    "            print(f\"Loading with bbox: {bbox}\")\n",
    "            window = from_bounds(*bbox, src.transform)\n",
    "            print(f\"Window: {window}\")\n",
    "            clipped_image = src.read(1, window=window)\n",
    "            print(f\"Dimensions after clipping: {clipped_image.shape}\")\n",
    "            transform = src.window_transform(window)\n",
    "        return clipped_image, transform\n",
    "\n",
    "    def convert_bbox_to_utm(self, bbox):\n",
    "        \"\"\"\n",
    "        Function to convert a bounding box from WGS84 to UTM Zone 32N.\n",
    "        Parameters:\n",
    "        - bbox: The input bounding box.\n",
    "        Returns:\n",
    "        - The converted bounding box in UTM coordinates.\n",
    "        \"\"\"\n",
    "        in_proj = Proj(init='epsg:4326')  # WGS84\n",
    "        out_proj = Proj(init='epsg:32632')  # UTM Zone 32N\n",
    "\n",
    "        x_min, y_min = transform(in_proj, out_proj, bbox[0], bbox[1])\n",
    "        x_max, y_max = transform(in_proj, out_proj, bbox[2], bbox[3])\n",
    "        return (x_min, y_min, x_max, y_max)\n",
    "\n",
    "# Run the application\n",
    "train = Sentinel2Processor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4.2.2 Incorperating and processing single file data for the Area of Training"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Incorporating the Sentinel-2 data for the machine learning model in form of multiple bands in a single file for the Area of Training (AOT)\n",
    "\n",
    "class Sentinel2Processor:\n",
    "    def __init__(self):\n",
    "        self.bbox = None\n",
    "        self.file_path = None\n",
    "        self.create_widgets()\n",
    "\n",
    "    def create_widgets(self):\n",
    "        \"\"\"Create and display the initial widgets for user interaction.\"\"\"\n",
    "        self.question1()\n",
    "\n",
    "    def question1(self):\n",
    "        \"\"\"\n",
    "        Ask the user if they have Sentinel-2 data in the form of a single .TIF file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have Sentinel-2 data in the form of a single file containing multiple bands?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1_yes)\n",
    "        no_button.on_click(self.q1_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Is your Area of Training, as in the place where your Training data are located, the same as your Area of Interest, for which you want to create a prediction map?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q1_yes_aoi)\n",
    "        no_button.on_click(self.q1_no_aoi)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q1_no(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the first question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Please use the tutorial to gather Sentinel-2 data.\"))\n",
    "\n",
    "    def q1_yes_aoi(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the Area of Training question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        if 'Sen2AOI' in globals():\n",
    "            sen2aoi = globals()['Sen2AOI']\n",
    "            # Copy data to Sen2AOT\n",
    "            globals()['Sen2AOT'] = sen2aoi\n",
    "            display(widgets.Label(\"Data for your Area of Training has been successfully copied to Sen2AOT.\"))\n",
    "        else:\n",
    "            display(widgets.Label(\"No data for your Area of Interest found. Please upload these data first!\"))\n",
    "\n",
    "    def q1_no_aoi(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the Area of Training question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        self.q2_no()\n",
    "\n",
    "    def q2_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"Draw a rectangle on the map to select the bounding box\"))\n",
    "\n",
    "        self.map_widget = Map(center=(51.23, 9.35), zoom=7, basemap=basemap_to_tiles(basemaps.OpenStreetMap.Mapnik))\n",
    "        draw_control = DrawControl(rectangle={'shapeOptions': {'color': '#0000FF'}})\n",
    "        self.map_widget.add_control(draw_control)\n",
    "        display(self.map_widget)\n",
    "\n",
    "        finish_button = widgets.Button(description=\"Finish\")\n",
    "        cancel_button = widgets.Button(description=\"Cancel\")\n",
    "\n",
    "        draw_control.on_draw(self.handle_draw)\n",
    "        finish_button.on_click(self.finish_bbox)\n",
    "        cancel_button.on_click(self.cancel_bbox)\n",
    "\n",
    "        display(finish_button, cancel_button)\n",
    "\n",
    "    def q2_no(self, b=None):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the second question.\n",
    "        \"\"\"\n",
    "        self.q3()\n",
    "\n",
    "    def handle_draw(self, target, action, geo_json):\n",
    "        \"\"\"\n",
    "        Handle the event when a feature is drawn on the map.\n",
    "        \"\"\"\n",
    "        bbox = shape(geo_json['geometry']).bounds\n",
    "        self.bbox = self.convert_bbox_to_utm(bbox)\n",
    "        rect = LeafletPolygon(locations=[[[bbox[1], bbox[0]], [bbox[1], bbox[2]], [bbox[3], bbox[2]], [bbox[3], bbox[0]], [bbox[1], bbox[0]]]], color=\"blue\", fill_opacity=0.5)\n",
    "        self.map_widget.add_layer(rect)\n",
    "        print(f\"BBOX coordinates (Lat/Lon): {bbox}\")\n",
    "        print(f\"BBOX coordinates (UTM): {self.bbox}\")\n",
    "\n",
    "    def finish_bbox(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Finish\" after drawing the bounding box.\n",
    "        \"\"\"\n",
    "        if self.bbox:\n",
    "            self.q3()\n",
    "        else:\n",
    "            display(widgets.Label(\"No bounding box selected.\"))\n",
    "\n",
    "    def cancel_bbox(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Cancel\" in response to the bounding box question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation.\"))\n",
    "\n",
    "    def q3(self):\n",
    "        \"\"\"\n",
    "        Ask the user if they have a .TIF file or a .grd file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"Do you have a .TIF file or .grd?\")\n",
    "        tif_button = widgets.Button(description=\".TIF\")\n",
    "        grd_button = widgets.Button(description=\".grd\")\n",
    "\n",
    "        tif_button.on_click(self.q3_tif)\n",
    "        grd_button.on_click(self.q3_grd)\n",
    "\n",
    "        display(question, tif_button, grd_button)\n",
    "\n",
    "    def q3_tif(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \".TIF\" in response to the third question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        self.ask_for_file(\".tif\")\n",
    "\n",
    "    def q3_grd(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \".grd\" in response to the third question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(\"The used Python modules can't incorporate .grd files. Do you want to convert your data to the .TIF file format?\")\n",
    "        yes_button = widgets.Button(description=\"Yes\")\n",
    "        no_button = widgets.Button(description=\"No\")\n",
    "\n",
    "        yes_button.on_click(self.q4_yes)\n",
    "        no_button.on_click(self.q4_no)\n",
    "\n",
    "        display(question, yes_button, no_button)\n",
    "\n",
    "    def q4_yes(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"Yes\" in response to the fourth question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        self.ask_for_file(\".grd\")\n",
    "\n",
    "    def q4_no(self, b):\n",
    "        \"\"\"\n",
    "        Handle the event when the user clicks \"No\" in response to the fourth question.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        display(widgets.Label(\"User aborted the operation. Please use the tutorial to gather Sentinel-2 data.\"))\n",
    "\n",
    "    def ask_for_file(self, extension):\n",
    "        \"\"\"\n",
    "        Ask the user for the path to a file of a specific type.\n",
    "        Parameters:\n",
    "        - extension: The file extension to look for (e.g., \".tif\" or \".grd\").\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        question = widgets.Label(f\"Please set the path to the data of the file with extension '{extension}'\")\n",
    "        file_chooser = filechooser.FileChooser()\n",
    "\n",
    "        upload_button = widgets.Button(description=\"Upload\")\n",
    "\n",
    "        def handle_upload(b):\n",
    "            \"\"\"\n",
    "            Handle the event when the user clicks \"Upload\" to select a file.\n",
    "            \"\"\"\n",
    "            selected_file = file_chooser.selected\n",
    "            if selected_file and selected_file.endswith(extension):\n",
    "                global uploaded_tif_path  # Store the file path in a global variable\n",
    "                self.file_path = selected_file\n",
    "                uploaded_tif_path = self.file_path  # Assign to global variable\n",
    "                if extension == \".grd\":\n",
    "                    self.convert_grd_to_tif()\n",
    "                else:\n",
    "                    self.process_tiff_and_plot()\n",
    "            else:\n",
    "                display(widgets.Label(f\"Data integration not successful. File must be a {extension} file.\"))\n",
    "\n",
    "        upload_button.on_click(handle_upload)\n",
    "\n",
    "        display(question, file_chooser, upload_button)\n",
    "\n",
    "    def convert_grd_to_tif(self):\n",
    "        \"\"\"\n",
    "        Convert a .grd file to .tif format and process the .tif file.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        output_tiff_path = self.file_path.replace('.grd', '.tif')\n",
    "\n",
    "        try:\n",
    "            self._convert_grd_to_tiff(self.file_path, output_tiff_path)\n",
    "            display(widgets.Label(\"Data conversion successful\"))\n",
    "            self.file_path = output_tiff_path\n",
    "            global uploaded_tif_path\n",
    "            uploaded_tif_path = self.file_path  # Update global variable with the new file path\n",
    "            self.process_tiff_and_plot()\n",
    "        except Exception as e:\n",
    "            display(widgets.Label(\"Data integration and transformation not successful\"))\n",
    "\n",
    "    def _convert_grd_to_tiff(self, input_grd_path, output_tiff_path):\n",
    "        \"\"\"\n",
    "        Convert a GRD file to a TIF file using rasterio.\n",
    "        \"\"\"\n",
    "        # Implement the conversion logic using rasterio or any other appropriate library\n",
    "        pass  # Placeholder for actual conversion logic\n",
    "\n",
    "    def process_tiff_and_plot(self):\n",
    "        \"\"\"\n",
    "        Process the .tif file, display the number of bands, and plot the RGB composite.\n",
    "        \"\"\"\n",
    "        self.clear_output()\n",
    "        with rasterio.open(self.file_path) as dataset:\n",
    "            bands = []\n",
    "            for i in range(1, dataset.count + 1):\n",
    "                band = dataset.read(i)\n",
    "                bands.append(band)\n",
    "\n",
    "            stacked_bands = np.stack(bands, axis=-1)\n",
    "            print(stacked_bands.shape)\n",
    "\n",
    "        num_bands = stacked_bands.shape[2]\n",
    "        display(widgets.Label(f\"Number of bands: {num_bands}\"))\n",
    "\n",
    "        if num_bands >= 3:\n",
    "            self.plot_rgb_composite(stacked_bands)\n",
    "\n",
    "    def plot_rgb_composite(self, stacked_bands):\n",
    "        \"\"\"\n",
    "        Plot an RGB composite of the Sentinel-2 image using the first 3 bands.\n",
    "        \"\"\"\n",
    "        global Sen2AOT\n",
    "        for i in range(stacked_bands.shape[2]):\n",
    "            print(f\"Band {i+1} min: {stacked_bands[..., i].min()}, max: {stacked_bands[..., i].max()}\")\n",
    "\n",
    "        Sen2AOT = stacked_bands.astype(np.float32)\n",
    "        for i in range(Sen2AOT.shape[-1]):\n",
    "            Sen2AOT[..., i] = (Sen2AOT[..., i] - Sen2AOT[..., i].min()) / (Sen2AOT[..., i].max() - Sen2AOT[..., i].min())\n",
    "\n",
    "        plt.imshow(Sen2AOT[..., :3])\n",
    "        plt.title('Sentinel-2 RGB Composite')\n",
    "        plt.show()\n",
    "\n",
    "    def convert_bbox_to_utm(self, bbox):\n",
    "        \"\"\"\n",
    "        Convert latitude/longitude bounding box to UTM coordinates.\n",
    "        \"\"\"\n",
    "        p1 = Proj(proj='latlong', datum='WGS84')\n",
    "        p2 = Proj(proj='utm', zone=33, datum='WGS84')\n",
    "        utm_bbox = []\n",
    "        for coord in bbox:\n",
    "            utm_bbox.append(transform(p1, p2, coord[0], coord[1]))\n",
    "        return utm_bbox\n",
    "\n",
    "    def clear_output(self):\n",
    "        \"\"\"\n",
    "        Clear the output area to remove previous widgets and information.\n",
    "        \"\"\"\n",
    "        clear_output(wait=True)\n",
    "\n",
    "# Instantiate and run the Sentinel2Processor\n",
    "train = Sentinel2Processor()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5 Modeltraining\n",
    "* This part should be fairly easy: Just follow the steps in the UI and if you've integrated all of the other data correctly, the modeltraining should be smooth and quick\n",
    "* For any metrics to the precision of the model and explanations of the values you can skip to the end of this document\n",
    "* It should be noted though that we'll need to calculate the \"NDVI\", or \"Normalized Difference Vegetation Index\", which will help the algorithm differentiate vegetation a bit better. You don't have to do anything for that except for making sure that at least the \"red\" and the \"nir\" band are included in your rasterdata\n",
    "\n",
    "![NDVI_example](index/ndvi_example.png)\n",
    "\n",
    "<sup>*Figure 5: Calculated NDVI based on the values in the 'red' and the 'nir' chanel of the area of training (Dortmund)*</sup>\n",
    "\n",
    "* NOTE: Calculating the NDVI lead to the majority of errors during this project; We still plan to include it in the future, as it enhances the models ability to see and understand naturally occuring patterns but right now it's removed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start the model training process using the Sentinel-2 data\n",
    "\n",
    "# Ensure that the paths are set\n",
    "if 'uploaded_tif_path' not in globals() or 'uploaded_geojson_path' not in globals():\n",
    "    raise ValueError(\"Please upload the .tif and .geojson files and confirm the paths before starting the model training.\")\n",
    "\n",
    "tif_path = uploaded_tif_path\n",
    "geojson_path = uploaded_geojson_path\n",
    "\n",
    "# Ensure that the paths are not None\n",
    "if not tif_path or not geojson_path:\n",
    "    raise ValueError(\"One of the paths is not correctly set. Please make sure both files are uploaded correctly.\")\n",
    "\n",
    "# Load GeoJSON training data\n",
    "train_data = gpd.read_file(geojson_path)\n",
    "\n",
    "# Display CRS of the GeoJSON dataset\n",
    "print(\"GeoJSON CRS:\", train_data.crs)\n",
    "\n",
    "# Convert labels to numerical values\n",
    "label_encoder = LabelEncoder()\n",
    "train_data['Label'] = label_encoder.fit_transform(train_data['Label'])\n",
    "\n",
    "# Load Sentinel-2 bands as raster stack\n",
    "with rasterio.open(tif_path) as src:\n",
    "    sentinel_meta = src.meta\n",
    "    sentinel_crs = src.crs  # Get CRS of the raster\n",
    "    \n",
    "    # Display CRS of the raster dataset\n",
    "    print(\"Raster CRS:\", sentinel_crs)\n",
    "\n",
    "    # Check if the CRS of the GeoJSON matches the raster's CRS and reproject if necessary\n",
    "    if train_data.crs != sentinel_crs:\n",
    "        print(\"Reprojecting GeoJSON data to match the raster's CRS.\")\n",
    "        train_data = train_data.to_crs(sentinel_crs)\n",
    "\n",
    "    # Extract polygon geometries from the GeoJSON file\n",
    "    geoms = train_data.geometry.apply(mapping)\n",
    "\n",
    "    # Initialize list to store the extracted data\n",
    "    extracted_data = []\n",
    "\n",
    "    # Loop through each polygon geometry and label\n",
    "    for geom, label in zip(geoms, train_data['Label']):\n",
    "        try:\n",
    "            # Mask the raster with the current polygon geometry\n",
    "            out_image, out_transform = rasterio.mask.mask(src, [geom], crop=True)\n",
    "            # Reshape the masked image into a 2D array, where each row is a pixel and the columns are the bands\n",
    "            out_image = out_image.reshape(out_image.shape[0], -1).T\n",
    "            # Create an array of the same length as out_image, filled with the current label\n",
    "            labels = np.full(out_image.shape[0], label)\n",
    "            # Add data to the list\n",
    "            extracted_data.append(np.column_stack((out_image, labels)))\n",
    "        except ValueError:\n",
    "            print(f\"Polygon with label {label} does not intersect with the raster and will be skipped.\")\n",
    "\n",
    "# Combine all extracted data into an array\n",
    "if extracted_data:  # Only proceed if valid data is present\n",
    "    extracted_data = np.vstack(extracted_data)\n",
    "else:\n",
    "    raise ValueError(\"No data could be extracted. Please check your input data.\")\n",
    "\n",
    "# Split predictors (bands) and labels\n",
    "X = extracted_data[:, :-1]  # Predictors (B02, B03, etc.)\n",
    "y = extracted_data[:, -1]   # Labels\n",
    "\n",
    "# Split data into training and test sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.5, stratify=y)\n",
    "\n",
    "# Train the Random Forest model\n",
    "rf_model = RandomForestClassifier(n_estimators=500, random_state=42)\n",
    "rf_model.fit(X_train, y_train)\n",
    "\n",
    "# Display trained model and variable importance\n",
    "print(\"Random Forest Model:\")\n",
    "print(rf_model)\n",
    "\n",
    "# Feature Importances\n",
    "importances = rf_model.feature_importances_\n",
    "indices = np.argsort(importances)[::-1]\n",
    "for i in range(len(importances)):\n",
    "    print(f\"Band {i+1}: {importances[indices[i]]}\")\n",
    "\n",
    "# Calculate accuracy on the test set\n",
    "y_pred = rf_model.predict(X_test)\n",
    "accuracy = accuracy_score(y_test, y_pred)\n",
    "print(f\"Model accuracy: {accuracy:.4f}\")\n",
    "\n",
    "# Save trained model and label encoder\n",
    "joblib.dump(rf_model, \"RF_Model.pkl\")\n",
    "joblib.dump(label_encoder, \"Label_Encoder.pkl\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6 Prediction Map\n",
    "* As the last part this part should be fairly easy: Just follow the steps in the UI\n",
    "* NOTE: You still have to have all data integrated in the Python Environment, so make sure to not reset anything!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start the prediction process using the trained model and the Area of Interest (AOI) data\n",
    "\n",
    "if 'aoi_path' not in globals() or 'Sen2AOI' not in globals():\n",
    "    raise ValueError(\"AOI data was not loaded correctly. Please ensure that the AOI data is loaded correctly.\")\n",
    "\n",
    "# Load the model and label encoder\n",
    "rf_model = joblib.load(\"RF_Model.pkl\")\n",
    "label_encoder = joblib.load(\"Label_Encoder.pkl\")\n",
    "\n",
    "with rasterio.open(aoi_path) as src:\n",
    "    aoi_meta = src.meta\n",
    "    aoi_bands = src.read()  # Read all bands\n",
    "\n",
    "aoi_reshaped = aoi_bands.reshape(len(aoi_bands), -1).T\n",
    "\n",
    "print(\"Calculating the prediction! This might take a while. I suggest to go and grab a coffee while this loads.\")\n",
    "\n",
    "aoi_prediction = rf_model.predict(aoi_reshaped)\n",
    "aoi_prediction = aoi_prediction.reshape(aoi_bands.shape[1:])\n",
    "\n",
    "# Assign colors dynamically based on the number of classes in the label encoder\n",
    "num_classes = len(label_encoder.classes_)\n",
    "colors = plt.get_cmap('Dark2')(np.linspace(0, 1, num_classes))  # Use 'tab10' or another color scheme\n",
    "cmap = mcolors.ListedColormap(colors)\n",
    "\n",
    "# Plot the prediction for the AOI\n",
    "plt.figure(figsize=(10, 10))\n",
    "im = plt.imshow(aoi_prediction, cmap=cmap)\n",
    "plt.title(\"Random Forest Prediction\")\n",
    "\n",
    "# Create a dynamic legend based on the labels\n",
    "legend_labels = {label: colors[i] for i, label in enumerate(label_encoder.classes_)}\n",
    "cbar = plt.colorbar(im, ticks=np.arange(num_classes))\n",
    "cbar.ax.set_yticklabels(label_encoder.classes_)\n",
    "plt.show()\n",
    "\n",
    "# Adjust the output path to save in the same directory as the AOI file\n",
    "output_dir = os.path.dirname(aoi_path)\n",
    "output_tif_filename = \"RF_Prediction_AOI.tif\"\n",
    "output_pdf_filename = \"RF_Prediction_AOI.pdf\"\n",
    "output_tif_path = os.path.join(output_dir, output_tif_filename)\n",
    "output_pdf_path = os.path.join(output_dir, output_pdf_filename)\n",
    "\n",
    "# Check if the directory exists and create it if it doesn't\n",
    "if not os.path.exists(output_dir):\n",
    "    os.makedirs(output_dir)\n",
    "\n",
    "# Adjust metadata and set a valid nodata value for uint8\n",
    "output_meta = aoi_meta.copy()\n",
    "output_meta.update({\"count\": 1, \"dtype\": 'uint8', \"nodata\": None})\n",
    "\n",
    "# Save the predictions as a new TIFF file in the specified directory\n",
    "with rasterio.open(output_tif_path, 'w', **output_meta) as dst:\n",
    "    dst.write(aoi_prediction.astype(rasterio.uint8), 1)\n",
    "\n",
    "print(f\"Prediction saved to: {output_tif_path}\")\n",
    "\n",
    "# Save the prediction as a PDF\n",
    "with PdfPages(output_pdf_path) as pdf:\n",
    "    plt.figure(figsize=(10, 10))\n",
    "    im = plt.imshow(aoi_prediction, cmap=cmap)\n",
    "    plt.title(\"Random Forest Prediction\")\n",
    "    cbar = plt.colorbar(im, ticks=np.arange(num_classes))\n",
    "    cbar.ax.set_yticklabels(label_encoder.classes_)\n",
    "    pdf.savefig()  # Save the current figure to the PDF\n",
    "    plt.close()\n",
    "\n",
    "print(f\"Prediction saved as PDF to: {output_pdf_path}\")\n",
    "\n",
    "print(\"Thank you for testing out our App! There is a lot of room for improvements and we're always grateful for helpful criticism and suggestions!\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* After the prediction is done you should be able to save your results as a .TIF or .PDF File\n",
    "* You can also read the models metrics such as accuracy in Chapter 5: Modeltraining if you'd like\n",
    "* The finished model could look like this:\n",
    "\n",
    "\n",
    "![class_example](index/class_example.jpg)\n",
    "\n",
    "<sup>*Figure 6: Calculated prediction of Münster based on a spatial prediction model of Dortmund*</sup>\n",
    "\n",
    "Thank you for testing our App! For any bugs or recommendations feel free to contact us!"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "ArcGISPro",
   "language": "Python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}