Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

User Management backend - Base - DB migrations #46

Open
wants to merge 14 commits into
base: master
Choose a base branch
from
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -17,3 +17,5 @@ _pycache_/
# Python
*.pyc
*.orig
*.pyc.*
simzacks marked this conversation as resolved.
Show resolved Hide resolved
/.cache
33 changes: 33 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
import pytest
import psycopg2
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might want to use sqlalchemy to test that the database was created properly, so that your tests are uniform with your code. Either way works the same, one difference is if you switch database engines (one of the benefits of using an ORM) you would have to rewrite this code.



@pytest.fixture
def db_connection():
connection = psycopg2.connect(
host="localhost",
database="edison",
user="postgres",
password="edison")

yield connection
connection.close()

@pytest.fixture
def cursor(db_connection):
cursor = db_connection.cursor()
yield cursor
db_connection.rollback()

@pytest.fixture
def build_users_table(cursor):
cursor.execute("""
INSERT INTO users (username) VALUES (%s);
""",
("ahinoam",))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume you don't want to save this data because its a test. If you did, you would need to call commit


@pytest.fixture
def reset_users_primary_key(cursor):
Copy link
Collaborator Author

@ahinoamta ahinoamta May 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @ifireball 😄
I wrote a few tests for checking the DB like you asked me in the last PR.
Can you please help me to find a way to reset the primary key sequence after the tests will be finished?
I tried to enter this fixture in the 'cursor' fixture (right after the 'yield'), but it didn't work, I assume it happened because after the 'yield' the cursor is not available anymore?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why would you want to reset the primary key sequence? It should have no special meaning, so it won't matter if the first user is 1 or 140

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In any case, probably the reason it didn't reset your sequence is because you didn't commit the transaction.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One last comment. The tests that you wrote here verify basic database functionality, which doesn't need to be tested.
What you should probably test is:

  • the database exists
  • you can connect to it
  • the tables you created exist.

Any other tests are really redundant.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I want to reset the primary key in order to clean up my database after the tests and return it to the initial state. I think it will make more sense when the database show you data that begins with id 1.
  2. About the commit after the execute, I used the “cursor.execute” without commit after it but still the query was executed. So why in this case I should add the commit?
    Furthermore, I don’t know where to call this fixture, because if I inject it to “cursor” it will activate it in the beginning of the “cursor” fixture, no?
  3. Ack

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 - the alter sequence statement is correct and it should work. Try to remember, in the lifetime of an app the database sequence will very rarely stay sequential.
2 - Have you tried executing this code and then going into the database using psql and seeing if the data you added was still there? As far as i know pyscopg does not have automated commits. One possibly confusing thing about sequences and transactions is when you get a new sequence that ignores the transaction, so if you insert and then rollback the transaction, the sequence is still at the next number.
On line 20 you have db_connection.rollback() when is this code called? A rollback means erase everything that I've done during this current transaction.

Copy link
Collaborator Author

@ahinoamta ahinoamta May 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you tried executing this code and then going into the database using psql and seeing if the data you added was still there?

I thought that cursor.execute is enough because when I ran the test def test_username(cursor, build_users_table) it was passed, so the row 'ahinoam' did enter to the DB.
But when I checked it in the DB itself with psql, it didn't appear there (and it's not because db_connection.rollback() because I put it in a comment).
Do you have any idea what is the reason for that?
Maybe it's because I remain in the same transaction?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On line 20 you have db_connection.rollback() when is this code called? A rollback means erase everything that I've done during this current transaction.

I want it to be called after the cursor will finish his job every test, therefore I put db_connection.rollback() right after the yield cursor statement. But I have to be honest I don't understand the yield statements into their depth...

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The way the database works is that during the same transaction everything looks like it is saved, but it really isn't until you call commit.
If you don't manually call commit, it is the same as calling rollback.

cursor.execute("""
ALTER SEQUENCE users_id_seq RESTART;
""")
17 changes: 17 additions & 0 deletions tests/db_test.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
import pytest
import psycopg2


def test_users_table_rows_count(cursor, build_users_table):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this cursor defined? The only place I see it is in conftest.py and this script doesn't import that one.
It also might make sense to have one database test file and not split it up.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if I understood the code in the conftest.py correctly, you always call rollback, which erases all of your changes. So if you are expecting the data to be here because of that execution, it might not be.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Where is this cursor defined? The only place I see it is in conftest.py and this script doesn't import that one.
It also might make sense to have one database test file and not split it up.

The cursor is defined in conftest.py file that aggregate all the fixtures in one place. I tried to keep the db_test.py file short.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, if I understood the code in the conftest.py correctly, you always call rollback, which erases all of your changes. So if you are expecting the data to be here because of that execution, it might not be.

I call rollback after the cursor finish his job so the data still in the DB in this point. This test was passed.

cursor.execute("""
SELECT * FROM users;
""")
result = cursor.fetchall()
assert len(result) == 1

def test_username(cursor, build_users_table):
cursor.execute("""
SELECT username FROM users;
""")
result = cursor.fetchone()
assert result[0] == "ahinoam"
11 changes: 0 additions & 11 deletions tests/init_test.py

This file was deleted.