Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🚑️ (src/db_redis): fix empty request check #30

Merged
merged 15 commits into from
Oct 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,13 @@ VAULT_CLIENT_SECRET=
VAULT_API_BASE_URL=
VAULT_CONF_URL=
SECRET_KEY=
ADMIN_USERS=
```

> [!NOTE]
> - `VAULT_API_BASE_URL` should be similar to `<ip>/v1/`.
> - `VAULT_CONF_URL` should be similar to `<ip>/v1/identity/oidc/provider/<provider>/.well-known/openid-configuration`. The `<provider>` string should be `default`.
> - **DON'T FORGET THE PROTOCOL (`http://` or `https://`) BEFORE THE `<ip>` STRING !!**
> - `SECRET_KEY` should be invented (not provided by Voult).
> - `ADMIN_USERS` must be a list of users id. Something like this `ADMIN_USERS='["[email protected]", "[email protected]", "[email protected]"]'`.
> - Replace email with usernames or whatever you want. Be careful with `'` and `"`, these must be used exactly as in the example.

One example of a configuration is shown below:
Expand All @@ -44,7 +42,6 @@ VAULT_CLIENT_SECRET=hvo_secret_TcKGRPh3sjC1WE4PrS2GV3XYpY2AkL0FEgYWRNQUPw7rLTYSS
VAULT_API_BASE_URL=http://localhost:8200/v1/
VAULT_CONF_URL=http://127.0.0.1:8200/v1/identity/oidc/provider/default/.well-known/openid-configuration
SECRET_KEY=!secret
ADMIN_USERS='["[email protected]", "[email protected]", "[email protected]"]'
```

## Run
Expand Down
4 changes: 3 additions & 1 deletion src/app.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from logging.config import dictConfig

Check failure on line 1 in src/app.py

View workflow job for this annotation

GitHub Actions / Pylint

src/app.py#L1

Missing module docstring (missing-module-docstring, C0114)
from authlib.integrations.flask_client import OAuth
from flask import Flask
from .db_redis import DBManager
Expand Down Expand Up @@ -39,14 +39,16 @@
app.logger.debug("Loading configs from envs")
app.config.from_object('src.config')

app.logger.debug(f"Setting secret key: {app.config['SECRET_KEY']}")

Check failure on line 42 in src/app.py

View workflow job for this annotation

GitHub Actions / Pylint

src/app.py#L42

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)
app.secret_key = app.config['SECRET_KEY']

app.logger.debug("Setting REDIS ip and password: %s %s", app.config['REDIS_IP'], app.config['REDIS_PASSWORD'])

Check failure on line 45 in src/app.py

View workflow job for this annotation

GitHub Actions / Pylint

src/app.py#L45

Line too long (110/100) (line-too-long, C0301)
app.redis_ip = app.config['REDIS_IP']
app.redis_password = app.config['REDIS_PASSWORD']

app.logger.debug(f"ADMIN USERS: {app.config['ADMIN_USERS']}")
app.logger.debug("Setting LDAPSYNC ip and port: %s %s", app.config['LDAPSYNC_IP'], app.config['LDAPSYNC_PORT'])

Check failure on line 49 in src/app.py

View workflow job for this annotation

GitHub Actions / Pylint

src/app.py#L49

Line too long (111/100) (line-too-long, C0301)
app.ldapsync_ip = app.config['LDAPSYNC_IP']
app.ldapsync_port = app.config['LDAPSYNC_PORT']

app.logger.debug("SERVER NAME: %s", app.config['SERVER_NAME'])

Expand All @@ -65,4 +67,4 @@
}
)

app.logger.debug("OAUTH CONFIGS: %s", oauth._registry)

Check failure on line 70 in src/app.py

View workflow job for this annotation

GitHub Actions / Pylint

src/app.py#L70

Access to a protected member _registry of a client class (protected-access, W0212)
3 changes: 2 additions & 1 deletion src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
VAULT_CONF_URL = os.getenv('VAULT_CONF_URL')
SERVER_NAME = os.getenv('WEB_PUBLIC_URL')
SECRET_KEY = os.getenv('SECRET_KEY')
ADMIN_USERS = json.loads(os.getenv('ADMIN_USERS'))
REDIS_IP = os.getenv('REDIS_IP')
REDIS_PASSWORD = os.getenv('REDIS_PASSWORD')
LDAPSYNC_IP = os.getenv('LDAPSYNC_IP')
LDAPSYNC_PORT = os.getenv('LDAPSYNC_PORT')
49 changes: 48 additions & 1 deletion src/db_redis.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,16 @@
from datetime import datetime

Check failure on line 1 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L1

Missing module docstring (missing-module-docstring, C0114)
import redis
import requests


class DBManager():

Check failure on line 6 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L6

Missing class docstring (missing-class-docstring, C0115)

Check failure on line 6 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L6

Too many instance attributes (10/7) (too-many-instance-attributes, R0902)

def __init__(self, app):
self.logger = app.logger
self.REDIS_IP = app.redis_ip

Check failure on line 10 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L10

Attribute name "REDIS_IP" doesn't conform to snake_case naming style (invalid-name, C0103)
self.REDIS_PASSWORD = app.redis_password

Check failure on line 11 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L11

Attribute name "REDIS_PASSWORD" doesn't conform to snake_case naming style (invalid-name, C0103)
self.LDAPSYNC_IP = app.ldapsync_ip

Check failure on line 12 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L12

Attribute name "LDAPSYNC_IP" doesn't conform to snake_case naming style (invalid-name, C0103)
self.LDAPSYNC_PORT = app.ldapsync_port

Check failure on line 13 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L13

Attribute name "LDAPSYNC_PORT" doesn't conform to snake_case naming style (invalid-name, C0103)

self.connection = redis.Redis(
host=self.REDIS_IP,
Expand All @@ -18,6 +21,7 @@

# consts
self.__idx = 'req'
self.__infoidx = 'info'
self.__keys = {
'startdate': 'startdate',
'enddate': 'enddate',
Expand All @@ -30,6 +34,10 @@
'synced': 'synced',
}

def __notify_ldapsync(self):
self.logger.info('NOTIFY LDAPSYNC')
r = requests.get(url=f"http://{self.LDAPSYNC_IP}:{self.LDAPSYNC_PORT}")

Check failure on line 39 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L39

Missing timeout argument for method 'requests.get' can cause your program to hang indefinitely (missing-timeout, W3101)
self.logger.debug("TRIGGERED LDAPSYNC, RESPONSE: %s", r)

def __del__(self):
self.logger.debug("Closing DB connection")
Expand Down Expand Up @@ -57,6 +65,10 @@
self.logger.debug("SETTING KEY: %s --> %s", key, value)
self.connection.set(key, value)

def __add_to_set(self, key: str, values: list):
self.logger.debug("ADDING %s TO %s", values, key)
self.connection.sadd(key, *values)

def __get_key(self, key: str) -> str:
value = self.connection.get(key)
self.logger.debug("GET KEY: %s --> %s", key, value)
Expand Down Expand Up @@ -85,7 +97,7 @@
# case self.__request_statuses.get('approved'):
# return self.__request_statuses['pending']

def add_request(self, user: str):

Check failure on line 100 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L100

Missing function or method docstring (missing-function-docstring, C0116)
self.__valid_user(user)

self.logger.info("NEW REQUEST FOR USER: %s", user)
Expand All @@ -107,12 +119,14 @@
self.__request_statuses['pending'],
)

self.__notify_ldapsync()

def delete_request(self, user: str):

Check failure on line 124 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L124

Missing function or method docstring (missing-function-docstring, C0116)
self.__valid_user(user)

# an approved/synced request cannot be removed !!
request_status = self.__get_key(f'{self.__idx}:{user}:{self.__keys["status"]}')
if request_status in [self.__request_statuses['approved'], self.__request_statuses['synced']]:

Check failure on line 129 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L129

Line too long (102/100) (line-too-long, C0301)
self.logger.warning("TRYING TO REMOVE AN APPROVED REQUEST ! Ignoring")
return

Expand All @@ -124,7 +138,7 @@
for key in self.connection.scan_iter(q):
self.__del_key(key)

def update_request_status(self, user: str):

Check failure on line 141 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L141

Missing function or method docstring (missing-function-docstring, C0116)
self.__valid_user(user)

self.logger.info("UPDATING USER REQUEST STATUS: %s", user)
Expand All @@ -141,10 +155,13 @@

if new_request_status == self.__request_statuses['approved']:
self.__set_key(f'{self.__idx}:{user}:{self.__keys["enddate"]}', str(datetime.now()))
self.__add_to_set(f'{self.__idx}:{user}:{self.__keys["groups"]}', ["users"])
self.__notify_ldapsync()

else:
self.__del_key(f'{self.__idx}:{user}:{self.__keys["enddate"]}')

def get_request_data(self, user: str) -> dict[str, str]:

Check failure on line 164 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L164

Missing function or method docstring (missing-function-docstring, C0116)
self.logger.info("GETTING REQUEST DATA: %s", user)

self.__valid_user(user)
Expand All @@ -158,7 +175,11 @@
else:
request_data[key] = self.__get_key(f'{self.__idx}:{user}:{key}')

if request_data[key] is not None:
# `request_data[key]` could be `None` (because it's a string Key) or
# `set()` (because it's a set Key), respectively generated by `__get_key()` and
# `__get_skey`. Need to check that req. data is not None and then is not an
# empty set.
if request_data[key] is not None and request_data[key] != set():
request_empty = False

if request_empty:
Expand All @@ -170,14 +191,14 @@
request_data[key] = datetime.strptime(request_data[key], '%Y-%m-%d %H:%M:%S.%f')

except (ValueError, TypeError,):
self.logger.error("Wrong datetime format in %s: %s", f'{user}:{key}', request_data[key])

Check failure on line 194 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L194

Line too long (104/100) (line-too-long, C0301)
request_data[key] = None

continue

return request_data

def get_all_request_data(self):

Check failure on line 201 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L201

Missing function or method docstring (missing-function-docstring, C0116)
self.logger.info("GETTING ALL REQUESTS DATA")

users = []
Expand All @@ -195,7 +216,33 @@
for user in users:
request_data = self.get_request_data(user)
request_data['user'] = user
request_data['infos'] = self.get_user_infos(user)

all_requests_data.append(request_data)

return all_requests_data

def get_user_infos(self, user: str) -> dict:

Check failure on line 225 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L225

Missing function or method docstring (missing-function-docstring, C0116)
self.logger.info("GETTING USER %s INFOS", user)

self.__valid_user(user)

user_infos = {}
infos_empty = True
infokeys = self.connection.keys(f'{self.__infoidx}:{user}:*')

self.logger.debug("USER %s KEYS: %s", user, infokeys)

for key in infokeys:
dictkey = key.split(':')[-1]
user_infos[dictkey] = self.__get_key(key)

if user_infos[dictkey] is not None:
infos_empty = False

self.logger.debug("USER INFOS: %s", user_infos)

if infos_empty:
return {}

Check failure on line 247 in src/db_redis.py

View workflow job for this annotation

GitHub Actions / Pylint

src/db_redis.py#L247

Trailing whitespace (trailing-whitespace, C0303)
return user_infos
4 changes: 3 additions & 1 deletion src/routes.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,20 @@
from flask import url_for, session, request, render_template, redirect

Check failure on line 1 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L1

Missing module docstring (missing-module-docstring, C0114)
from .app import app, dbms, oauth


@app.route('/', methods=['GET', 'POST', 'DELETE'])
def homepage():

Check failure on line 6 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L6

Missing function or method docstring (missing-function-docstring, C0116)
user = session.get('user')

app.logger.debug(f"/ User value: {user}")

Check failure on line 9 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L9

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

if user:
_username = user['metadata']['name']
_groups = dbms.get_request_data(_username)['groups']
_groups = dbms.get_request_data(_username).get('groups', [])
_infos = dbms.get_user_infos(_username)
user['username'] = _username
user['groups'] = _groups
user['infos'] = _infos
user['uninuvolaurl'] = 'https://compute.uninuvola.unipg.it/hub/oauth_login?next='

# ADMIN ROLE
Expand All @@ -22,12 +24,12 @@

if request.method == 'POST':
user_to_update = request.form['id']
app.logger.debug(f"UPDATING USER {user_to_update} REQUEST")

Check failure on line 27 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L27

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

dbms.update_request_status(user_to_update)

user['request_data'] = dbms.get_all_request_data()
app.logger.debug(f"ADMIN REQUEST DATA: {user['request_data']}")

Check failure on line 32 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L32

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

return render_template('users/admin.html', user=user)

Expand All @@ -43,7 +45,7 @@
print(request.method)

user['request_data'] = dbms.get_request_data(_username)
app.logger.debug(f"USER REQUEST DATA: {user['request_data']}")

Check failure on line 48 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L48

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

return render_template('users/user.html', user=user)

Expand All @@ -51,7 +53,7 @@


@app.route('/login')
def login():

Check failure on line 56 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L56

Missing function or method docstring (missing-function-docstring, C0116)
app.logger.debug('Redirecting to Vault')
redirect_uri = url_for('auth', _external=True, _scheme='https')

Expand All @@ -61,17 +63,17 @@


@app.route('/auth')
def auth():

Check failure on line 66 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L66

Missing function or method docstring (missing-function-docstring, C0116)
try:
token = oauth.vault.authorize_access_token()

except Exception as e: # TODO: choose better Exception

Check failure on line 70 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L70

TODO: choose better Exception (fixme, W0511)

Check failure on line 70 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L70

Catching too general exception Exception (broad-exception-caught, W0718)
app.logger.error("Error while autorizing access token, %s", e)

return redirect(url_for('homepage'))

app.logger.debug('Success user auth')
app.logger.debug(f'Token: {token}')

Check failure on line 76 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L76

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

session['user'] = token['userinfo']
session['token'] = token
Expand All @@ -79,10 +81,10 @@
return redirect(url_for('homepage'))

@app.route('/logout')
def logout():

Check failure on line 84 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L84

Missing function or method docstring (missing-function-docstring, C0116)
token = session.get('token')

app.logger.debug(f'Logging out {token}')

Check failure on line 87 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L87

Use lazy % formatting in logging functions (logging-fstring-interpolation, W1203)

oauth.vault.post('auth/token/revoke-self', token=token)

Expand All @@ -93,5 +95,5 @@
return redirect('/')

@app.route('/info')
def info():

Check failure on line 98 in src/routes.py

View workflow job for this annotation

GitHub Actions / Pylint

src/routes.py#L98

Missing function or method docstring (missing-function-docstring, C0116)
return render_template('info.html')
4 changes: 3 additions & 1 deletion src/templates/users/admin.html
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
{% endblock%}

{% block body %}
<h3>Welcome {{user['username']}}</h3>
<h3>Welcome {{user['infos'].get('name', user['username'])}}</h3>

<div class="position-absolute top-50 start-50 translate-middle">
<a href="{{ user['uninuvolaurl'] }}"><button type="button" class="btn btn-primary btn-lg">
Expand All @@ -19,6 +19,7 @@ <h3 class="mt-5">Request status</h3>
<thead>
<tr>
<th>Date</th>
<th>Name</th>
<th>User</th>
<th>State</th>
<th>Accepted Date</th>
Expand All @@ -29,6 +30,7 @@ <h3 class="mt-5">Request status</h3>
{% for req in user['request_data'] %}
<tr>
<td>{{req['startdate']}}</td>
<td>{{req['infos'].get('name', '-')}}</td>
<td>{{req['user']}}</td>
<td>{{req['status']}}</td>
<td>{{ req['enddate'] if req['enddate'] else '-'}}</td>
Expand Down
2 changes: 1 addition & 1 deletion src/templates/users/user.html
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@
{% endblock %}

{% block body %}
<h3>Welcome {{user['username']}}</h3>
<h3>Welcome {{user['infos'].get('name', user['username'])}}</h3>

{% if user['request_data']|length == 0 %}
<h3 class="mt-5">Request status</h3>
Expand Down
Loading