[ENHANCEMENT] Automate GitLab Integration Steps

[aryanpingle]

The Post-Install Configuration section from gitlab.md talks about changing the configuration file gitlab.rb within the gitlab container. This step may be removed as the configuration is already provided in compose.gitlab.yml via the GITLAB_OMNIBUS_CONFIG variable:

      GITLAB_OMNIBUS_CONFIG: |
        external_url 'https://${SERVER_DNS}/gitlab'
        gitlab_rails['gitlab_shell_ssh_port'] = 2424
        nginx['listen_port'] = 80
        nginx['enable'] = true
        nginx['listen_https'] = false
        nginx['redirect_http_to_https'] = false
        letsencrypt['enable'] = false

Passing the config through the gitlab compose file DOES NOT modify the file /etc/gitlab/gitlab.rb, but is automatically applied prior to running the container (which is why I didn't catch it earlier). This can be verified by setting up the integrate gitlab instance and running it without the post-install configuration step.

The password will be generated without needing to run gitlab-ctl reconfigure, so it may be possible to directly fetch it by running:

$ docker exec -it gitlab bash -c 'cat /etc/gitlab/initial_root_password'

(Haven't tested this specific command yet)

EDIT: Tested the command, it does indeed work.

[aryanpingle]

Setting up the integrated GitLab instance contains several steps which require special attention. It is simply not feasible to perform these manually every time we need a clean installation. Thankfully, GitLab and its database (which can be interfaced with Ruby-on-Rails) have pretty well documented API's, so those steps may be automated using Bash or Python scripts.

Creating a Personal Access Token

For now, I've used the GUI on the integrated GitLab instance's website to create the Personal Access Token. But this too should be automated using the gitlab-rails console command.

Creating DTaaS Server + Client Application Tokens

Using the excellent GitLab Applications API documentation, here is a Python 3.9 script I put together (no dependencies needed) to automate the process of creating application tokens:

import json
import requests
from typing import List

# Configuration
PRIVATE_TOKEN = "xyz"  # GitLab Administrator Private Token
SERVER_DNS = "foo.com"  # Server DNS with an integrated GitLab instance
IS_LOCALHOST = False  # SSL certificate verification will be disabled on localhost

# Variables
API_BASE_URL = f"{SERVER_DNS}/gitlab/api/v4/applications"


def create_application(
    name: str, redirect_uri: List[str], confidential: bool, scopes: List[str]
):
    """Create an application on the DTaaS server with specified settings."""

    payload = {
        "name": name,
        "redirect_uri": redirect_uri,
        "confidential": "true" if confidential else "false",
        "scopes": scopes,
    }
    response = requests.post(
        API_BASE_URL,
        headers={"PRIVATE-TOKEN": PRIVATE_TOKEN},
        data=payload,
        verify=not IS_LOCALHOST,
    )
    return response


def create_server_application() -> None:
    """Create the "DTaaS Server Authorization" token."""
    app_name = "DTaaS Server Authorization"
    scopes = "read_user"
    print(f"Creating '{app_name}'...")
    response = create_application(
        name=app_name,
        redirect_uri=f"{SERVER_DNS}/_oauth",
        confidential=True,
        scopes=scopes,
    )
    print(response.text)


def create_client_application() -> None:
    """Create the "DTaaS Client Authorization" token."""

    app_name = "DTaaS Client Authorization"
    scopes = "api openid profile read_repository read_user"
    print(f"Creating '{app_name}'...")
    response = create_application(
        name=app_name,
        redirect_uri=f"{SERVER_DNS}/Library",
        confidential=False,
        scopes=scopes,
    )
    print(response.text)


def list_all_applications():
    """List all applications."""

    response = requests.get(
        API_BASE_URL,
        headers={"PRIVATE-TOKEN": PRIVATE_TOKEN},
        verify=not IS_LOCALHOST,
    )
    all_applications = response.json()
    print(json.dumps(all_applications, indent=2))


def delete_application(application_id: int):
    """Delete an application token."""

    response = requests.delete(
        f"{API_BASE_URL}/{application_id}",
        headers={"PRIVATE-TOKEN": PRIVATE_TOKEN},
        verify=not IS_LOCALHOST,
    )

    if response.status_code == 204:
        print("Application successfully deleted.")
    else:
        print("Application could not be deleted.")
        print(response.text)


if __name__ == "__main__":
    list_all_applications()

All it requires is the administrator's Personal Access Token and the server DNS with an active integrated GitLab instance. I've used Python 3.9.6, but I believe the version can be further reduced.

[prasadtalasila]

@aryanpingle The script does look relevant. Perhaps it can be saved as deploy/services/configure.py and then gradually integrated into cli.

[aryanpingle]

Glad it looks good! A few more ideas for when this is implemented:

1. The web API is well and good, but it still requires the gitlab container to be up and running. If there's a gitlab-rails console approach to this, that would be perfect.
2. Instead of printing to the console, we could save the tokens to a gitignored file?
3. The initial password should be printed without the additional comments generated in the initial_root_password file.