Skip to content

Commit

Permalink
Merge pull request #15 from JustinFrizzell/sqlconnector-methods
Browse files Browse the repository at this point in the history
Sqlconnector methods
  • Loading branch information
JustinFrizzell authored Jan 7, 2024
2 parents d314e20 + 1bf1a08 commit 1a914f9
Show file tree
Hide file tree
Showing 5 changed files with 184 additions and 83 deletions.
76 changes: 39 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,77 +12,79 @@
<a href="https://github.com/psf/black"><img alt="Code style: black" src="https://img.shields.io/badge/code%20style-black-000000.svg"></a>
</p>

**SQLconnect** is a Python package designed to simplify the process of connecting to SQL databases. It uses a `sqlconnect.yaml` file for database configuration and a `sqlconnect.env` file for secure credentials management. SQLconnect supports executing SQL from `.sql` files or Python strings, and retrieving data into pandas DataFrames. This package is particularly useful for data analysts and developers who need a straightforward way to interact with SQL databases.
**SQLconnect** is a Python package designed to provide a straightforward way to interact with SQL databases (Postgres, Microsoft SQL Server, Oracle ect.). It enables direct population of DataFrames from .sql files. A `sqlconnect.yaml` file is used for database configuration and a `sqlconnect.env` file for secure credentials management.

## Features

- Allows easy configuration and management of database connections using a `sqlconnect.yaml` file.
- Turn SQL queries into DataFrames in as few as 3 lines of code

- Supports the use of a `sqlconnect.env` file for secure storage of database credentials, enhancing security.
- Easy management of multiple database connections using a configuration file

- Leverages SQLAlchemy for database connections, providing a robust and flexible framework for SQL operations.
- Secure storage of database credentials using environment variables

- Enables executing SQL queries and directly retrieving the results into pandas DataFrames, facilitating easy data manipulation and analysis.
- Execute SQL queries and commands directly from .sql files or from a string

- Capable of handling multiple database connections, allowing users to switch between different databases as needed.
- Integration with SQLAlchemy & Pandas providing robust and flexible SQL operations

- Provides functionality to execute SQL queries either by reading from an external file or directly from a string.
```python
import sqlconnect as sc

## Installation
connection = sc.Sqlconnector("My_Database") # Set up a database connection based on sqlconnect.yaml

```bash
pip install sqlconnect
df = connection.sql_to_df("path/to/sql_query.sql") # Assign the results of a query to a DataFrame

print(df.describe()) # Explore the DataFrame with Pandas
```


## Configuration

To use SQLconnect, create a `sqlconnect.yaml` file in the root of your project (or in your home directory) with the following example structure:

```yaml
connections:
Database_PROD:
sqlalchemy_driver: 'mssql+pyodbc'
odbc_driver: 'SQL+Server'
server: 'prod-server.database.com'
database: 'ProdDB'
username: '${DB_PROD_USERNAME}' # References DB_PROD_USERNAME in sqlconnect.env
password: '${DB_PROD_PASSWORD}' # References DB_PROD_PASSWORD in sqlconnect.env
options:
- 'Trusted_Connection=No'
My_Database:
dialect: 'mssql'
dbapi: 'pyodbc'
host: 'prod-server.example.com'
database: 'master'
username: '${MSSQL_USERNAME}' # References MSSQL_USERNAME in sqlconnect.env
password: '${MSSQL_PASSWORD}' # References MSSQL_PASSWORD in sqlconnect.env
options:
driver: 'ODBC Driver 17 for SQL Server'

A_Postgres_Database:
dialect: 'postgresql'
dbapi: 'psycopg2'
host: 'dbserver123.company.com'
database: 'postgres'
username: '${POSTGRES_USERNAME}'
password: '${POSTGRES_PASSWORD}'
```
Also create a `sqlconnect.env` file in the root of your project (or in your home directory) with the following example structure:

```bash
# This file should be kept secure and not checked into version control (add to .gitignore)
# Production Database Credentials
DB_PROD_USERNAME=prodUser
DB_PROD_PASSWORD=prodPassword
MSSQL_USERNAME=prodUsername
MSSQL_PASSWORD=actualprodPassword
POSTGRES_USERNAME=postgresProdUsername
POSTGRES_PASSWORD=actualprodPassword
```

Replace the example values with your actual database connection details.

## Usage

Here's a quick example to get you started:

```python
import sqlconnect as sc
## Documentation

# Set up a database connection, all configuration is handled with sqlconnect.yaml and sqlconnect.env
connection = sc.Sqlconnector("Database_PROD")
Full documentation for SQLconnect can be found at https://sqlconnect.readthedocs.io/

# Assign the results of a query to a pandas DataFrame
df = connection.sql_to_df("your_query.sql")
## Installation

# Explore the dataframe with pandas
print(df.describe())
```bash
pip install sqlconnect
```

## Documentation

Full documentation for SQLconnect can be found at https://sqlconnect.readthedocs.io/

## License

This project is licensed under the [MIT License](https://raw.githubusercontent.com/JustinFrizzell/sqlconnect/main/LICENCE).
3 changes: 2 additions & 1 deletion docs/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,8 @@


# The name of the Pygments (syntax highlighting) style to use.
pygments_style = "sphinx"
pygments_style = "default"
pygments_dark_style = "material"

# We need headers to be linkable to so ask MyST-Parser to autogenerate anchor IDs for
# headers up to and including level 3.
Expand Down
6 changes: 4 additions & 2 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@
.. raw:: html

<div style="text-align: center;">
<a href="https://github.com/JustinFrizzell/sqlconnect/actions/workflows/ci.yaml">
<img src="https://github.com/JustinFrizzell/sqlconnect/actions/workflows/ci.yaml/badge.svg" alt="CI-testing">
</a>
<a href="https://pypi.org/project/sqlconnect/">
<img src="https://img.shields.io/pypi/v/sqlconnect" alt="PyPI">
</a>
Expand All @@ -20,7 +23,7 @@
</div>
<div style="height: 20px;"></div> <!-- Spacer -->

**SQLconnect** is a Python package designed to simplify the process of connecting to SQL databases. It uses a ``sqlconnect.yaml`` file for database configuration and a ``sqlconnect.env`` file for secure credentials management. SQLconnect supports executing SQL from ``.sql`` files or Python strings, and retrieving data into pandas DataFrames. This package is particularly useful for data analysts and developers who need a straightforward way to interact with SQL databases.
**SQLconnect** is a Python package designed to provide a straightforward way to interact with SQL databases (Postgres, Microsoft SQL Server, Oracle ect.). It enables direct population of DataFrames from .sql files. A ``sqlconnect.yaml`` file is used for database configuration and a ``sqlconnect.env`` file for secure credentials management.

Contents
========
Expand All @@ -35,7 +38,6 @@ Contents
:caption: User Guide

install
tutorial
documentation

.. toctree::
Expand Down
6 changes: 3 additions & 3 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@

setuptools.setup(
name="sqlconnect",
version="0.2.1",
version="0.3.0",
author="Justin Frizzell",
description="Package to simplify connections to SQL databases.",
description="Simplifies connections to SQL databases for data analysts. Populate DataFrames with the results of queries directly from .sql files.",
long_description=Path("README.md").read_text(encoding="utf-8"),
long_description_content_type="text/markdown",
url="https://github.com/JustinFrizzell/sqlconnect",
Expand All @@ -14,7 +14,7 @@
"Source": "https://github.com/JustinFrizzell/sqlconnect",
},
packages=setuptools.find_packages(exclude=["tests", "tests.*"]),
install_requires=["pandas", "sqlalchemy", "pyyaml", "pyodbc", "python-dotenv"],
install_requires=["pandas", "sqlalchemy", "pyyaml", "python-dotenv", "pyodbc", "psycopg2-binary", "oracledb", "PyMySQL"],
classifiers=[
"License :: OSI Approved :: MIT License",
"Intended Audience :: Developers",
Expand Down
Loading

0 comments on commit 1a914f9

Please sign in to comment.