Skip to content

Commit

Permalink
Update readme. Added ability to delete record. Fixed bugs around vers…
Browse files Browse the repository at this point in the history
…ion tracking.
  • Loading branch information
scnerd committed Jun 3, 2024
1 parent 1ce6963 commit 21884fe
Show file tree
Hide file tree
Showing 7 changed files with 150 additions and 22 deletions.
5 changes: 5 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,8 @@ repos:
hooks:
- id: ruff
args: [ "--fix" ]

- repo: https://github.com/python-poetry/poetry
rev: '1.8.0'
hooks:
- id: poetry-check
73 changes: 66 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# PynamoDB Single Table

A [Pydantic](https://docs.pydantic.dev/latest/) ORM built on top of [PynamoDB](https://github.com/pynamodb/PynamoDB).

[![PyPI](https://img.shields.io/pypi/v/pynamodb_single_table.svg)][pypi status]
[![Status](https://img.shields.io/pypi/status/pynamodb_single_table.svg)][pypi status]
[![Python Version](https://img.shields.io/pypi/pyversions/pynamodb_single_table)][pypi status]
Expand All @@ -21,11 +23,73 @@

## Features

- TODO
Provides a Django-inspired "Active Record"-style ORM using single-table design built on top of Django.

```python
import abc
from datetime import datetime
from pydantic import Field, EmailStr
from pynamodb_single_table import SingleTableBaseModel

class BaseTableModel(SingleTableBaseModel, abc.ABC):
class _PynamodbMeta:
table_name = "MyDynamoDBTable"

class User(BaseTableModel):
__table_name__ = "user"
__str_id_field__ = "username"
username: str
email: EmailStr
account_activated_on: datetime = Field(default_factory=datetime.now)

# Make sure the table exists in DynamoDB
BaseTableModel.ensure_table_exists(billing_mode="PAY_PER_REQUEST")

# Create a record
john, was_created = User.get_or_create(username="john_doe", email="[email protected]")
assert was_created

# Retrieve
john_again = User.get_by_str("john_doe")
assert john_again.email == "[email protected]"

# Update
now = datetime.now()
john_again.account_activated_on = now
john_again.save()

assert User.get_by_str("john_doe").account_activated_on == now

# Delete
john_again.delete()
```

## Motivation

Many use cases need little more than structured CRUD operations with a table-like design (e.g., for storing users and groups), but figuring out how to host that efficiently in the cloud can be a pain.

DynamoDB is awesome for CRUD when you have clean keys.
It's a truly serverless NoSQL database, including nice features like:
- High performance CRUD operations when you know your primary keys
- Scale-to-zero usage-based pricing available
- Official local testing capability
- Conditional CRUD operations to avoid race conditions
- Multiple methods of indexing into data
- Scalable with reliable performance

This project, in part, emerges from my frustration with the lack of many truly serverless SQL database services.
By "truly serverless", I mean purely usage-based pricing (generally a combination of storage costs and query costs).
Many small, startup applications use trivial amounts of query throughput and story trivial amounts of data, but finding a way to deploy such an application into the cloud without shelling out $10-$100's per month is tricky.
In AWS, now that Aurora Serverless V1 is being replaced, there is _no_ way to do this.

However, DynamoDB provides not just the basic functionality needed to do this, it's actually a really good option if your data usage patterns can fit within its constraints.
That means, primarily, that you can always do key-based lookups, and that you can avoid changing your indexing strategy or database schema too much (e.g. modifying a table from having nullable columns into non-nullable).
DynamoDB _can_ do custom queries at tolerable rates, but you're going to get sub-par speed and cost efficiency if you're regularly doing searches across entire tables instead of direct hash key lookups.

## Requirements

- TODO
This project is built on the backs of Pydantic and Pynamodb.
I am incredibly grateful to the developers and communities of both of those projects.

## Installation

Expand All @@ -35,10 +99,6 @@ You can install _PynamoDB Single Table_ via [pip] from [PyPI]:
$ pip install pynamodb_single_table
```

## Usage

Please see the [Command-line Reference] for details.

## Contributing

Contributions are very welcome.
Expand Down Expand Up @@ -68,4 +128,3 @@ This project was generated from [@cjolowicz]'s [Hypermodern Python Cookiecutter]

[license]: https://github.com/scnerd/pynamodb_single_table/blob/main/LICENSE
[contributor guide]: https://github.com/scnerd/pynamodb_single_table/blob/main/CONTRIBUTING.md
[command-line reference]: https://pynamodb_single_table.readthedocs.io/en/latest/usage.html
38 changes: 37 additions & 1 deletion poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 2 additions & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "pynamodb_single_table"
version = "0.1.0"
version = "0.1.1"
description = "PynamoDB Single Table"
authors = ["David Maxson <[email protected]>"]
license = "MIT"
Expand Down Expand Up @@ -45,6 +45,7 @@ sphinx = ">=4.3.2"
sphinx-autobuild = ">=2021.3.14"
#sphinx-click = ">=3.0.2"
myst-parser = {version = ">=0.16.1"}
pydantic = {extras = ["email"], version = "*"}

[tool.poetry.group.constraints.dependencies]
urllib3 = "<1.27"
Expand Down
4 changes: 4 additions & 0 deletions src/pynamodb_single_table/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1,5 @@
"""PynamoDB Single Table."""

from .base import SingleTableBaseModel

__all__ = ("SingleTableBaseModel",)
30 changes: 21 additions & 9 deletions src/pynamodb_single_table/base.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ class SingleTableBaseModel(BaseModel):
__pynamodb_model__: Type[RootModelPrototype] = None

uid: Optional[uuid.UUID] = None
version: int = None

def __init_subclass__(cls, **kwargs):
if cls.__pynamodb_model__:
Expand All @@ -69,6 +70,10 @@ class RootModel(RootModelPrototype):
if not getattr(cls, "__str_id_field__", None):
raise TypeError(f"Must define the string ID field for {cls}")

@classmethod
def ensure_table_exists(cls, **kwargs) -> None:
cls.__pynamodb_model__.create_table(wait=True, **kwargs)

@computed_field
@property
def str_id(self) -> str:
Expand Down Expand Up @@ -122,35 +127,42 @@ def get_by_uid(cls, uuid_: uuid.UUID) -> Self:

@classmethod
def _from_item(cls, item) -> Self:
return cls(uid=item.uid, **item.data)
return cls(uid=item.uid, version=item.version, **item.data)

def create(self):
def _to_item(self) -> RootModelPrototype:
item = self.__pynamodb_model__(
self.__table_name__,
str_id=self.str_id,
data=self.model_dump(mode="json", exclude={"uid", "str_id"}),
data=self.model_dump(mode="json", exclude={"uid", "version", "str_id"}),
)
if self.uid is not None:
item.uid = self.uid
if self.version is not None:
item.version = self.version
return item

def create(self):
item = self._to_item()

condition = (
self.__pynamodb_model__.table_name.does_not_exist()
& self.__pynamodb_model__.uid.does_not_exist()
)
item.save(condition=condition, add_version_condition=False)
assert item.uid is not None
self.uid = item.uid
self.version = item.version
return self

def save(self):
item = self.__pynamodb_model__(
self.__table_name__,
uid=self.uid,
str_id=self.str_id,
data=self.model_dump(mode="json", exclude={"uid", "str_id"}),
)
item = self._to_item()
item.save(add_version_condition=False)
assert item.uid is not None
self.uid = item.uid
self.version = item.version

def delete(self):
self._to_item().delete()

@classmethod
def count(cls, *args, **kwargs):
Expand Down
19 changes: 15 additions & 4 deletions tests/test_basics.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

import pytest

from pynamodb_single_table.base import SingleTableBaseModel
from pynamodb_single_table import SingleTableBaseModel


class _BaseTableModel(SingleTableBaseModel, abc.ABC):
Expand Down Expand Up @@ -34,9 +34,7 @@ class Group(_BaseTableModel):

@pytest.fixture(scope="function", autouse=True)
def recreate_pynamodb_table() -> Type[SingleTableBaseModel]:
_BaseTableModel.__pynamodb_model__.create_table(
wait=True, billing_mode="PAY_PER_REQUEST"
)
_BaseTableModel.ensure_table_exists(billing_mode="PAY_PER_REQUEST")

try:
yield _BaseTableModel
Expand Down Expand Up @@ -159,3 +157,16 @@ def test_scan():
assert len(users) == 2
groups = list(Group.scan())
assert len(groups) == 1


def test_delete():
user1, _ = User.get_or_create(name="John Doe")
user2, _ = User.get_or_create(name="Joe Schmoe")

assert len(list(User.query())) == 2

print(list(User.__pynamodb_model__.scan()))
user1.delete()
assert len(list(User.query())) == 1
(userx,) = list(User.query())
assert userx.name == "Joe Schmoe"

0 comments on commit 21884fe

Please sign in to comment.