Skip to content

Commit

Permalink
Cleaning Up typing, examples, and imports (#14)
Browse files Browse the repository at this point in the history
* using `import` instead of `from` imports to cleanup namespace

* Cleaning up typing

* Added examples on how to use `ModifiableItemsDict`

* Cleaning up code and optimizing imports

* Updating Version in setup.py

* General fixes to the examples

* changed `typing.Type` to use `typing.TypeVar` for 3.6 compatibility

* Changing doctest import

* fixing imports

* Renaming `modifiable_items_dict` directory to `modifiable_items_dictionary`

* changing version on setup.py

* changed coverage directory

* removed doctest-modules for pytest

* Added back doctest to pytest

* Removed redundant import from doctest

* changed pytest to only test in tests/ directory
  • Loading branch information
tybruno authored Aug 17, 2023
1 parent cc281f8 commit 899f0e5
Show file tree
Hide file tree
Showing 9 changed files with 400 additions and 317 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/code-coverage-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
python -m pip install -r requirements-test.txt
- name: Test and generate coverage report
run: |
python -m pytest --cov=modifiable_items_dict --cov-report=xml --doctest-modules
python -m pytest --cov=modifiable_items_dictionary --cov-report=xml --doctest-modules
- name: Upload coverage to Codecov
uses: codecov/codecov-action@v3
with:
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/pytest-workflow.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,4 +26,4 @@ jobs:
python -m pip install -r requirements-test.txt
- name: Test
run: |
pytest --doctest-modules
pytest tests/
141 changes: 66 additions & 75 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,30 @@
[![Code Style](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)
[![License: MIT](https://img.shields.io/badge/License-MIT-blueviolet.svg)](https://opensource.org/licenses/MIT)
[![codecov](https://codecov.io/gh/tybruno/modifiable-items-dictionary/branch/main/graph/badge.svg?token=ZO94EJFI3G)](https://codecov.io/gh/tybruno/modifiable-items-dictionary)

# modifiable-items-dict
A simple, fast, typed, and tested implementation for a python3.6+ Modifiable Items dictionary. `ModifiableItemsDict` extends `dict` with the ability to modify key's and value's on creation, insertion, and retrieval.

A simple, fast, typed, and tested implementation for a python3.6+ Modifiable Items dictionary. `ModifiableItemsDict`
extends `dict` with the ability to modify key's and value's on creation, insertion, and retrieval.
This class extends and maintains the original functionality of the builtin `dict`.

#### Key Features:

* **Easy**: Flexable and easy to add Key and/or Value modifiers to the `ModifiableItemsDict`
* **Great Developer Experience**: Being fully typed makes it great for editor support.
* **Fully Tested**: Our test suit fully tests the functionality to ensure that `ModifiableItemsDict` runs as expected.
* **There is More!!!**:
* [CaselessDict](https://github.com/tybruno/caseless-dictionary): `CaselessDict` extends `ModifiableItemsDict` which is a `CaselessDict` which ignores the case of the keys.
* [CaselessDict](https://github.com/tybruno/caseless-dictionary): `CaselessDict` extends `ModifiableItemsDict` which
is a `CaselessDict` which ignores the case of the keys.

## Installation

`pip install modifiable-items-dictionary`

## Simple Example
```python
from modifiable_items_dict import ModifiableItemsDict


def _lower(_key):
if isinstance(_key, str):
_key = _key.lower()
return _key
```python
import modifiable_items_dictionary


def _add_1(_value):
Expand All @@ -33,121 +34,111 @@ def _add_1(_value):
return _value


ModifiableItemsDict._key_modifiers = [_lower]
ModifiableItemsDict._value_modifiers = (_lower, _add_1)
# Or
# ModifiableItemsDict._key_modifiers = staticmethod(_lower)
# ModifiableItemsDict._value_modifiers = [_lower, _add_1]
def _case_fold_string(_value):
if isinstance(_value, str):
_value = _value.casefold()
return _value

modifiable_items_dict = ModifiableItemsDict({"lower":1, "UPPER":2 }, CamelCase=3, snake_case="FoUR")

print(modifiable_items_dict) # {'lower': 2, 'upper': 3, 'camelcase': 4, 'snake_case': 'four'}
modifiable_items_dictionary.ModifiableItemsDict._key_modifiers = [str.casefold]
modifiable_items_dictionary.ModifiableItemsDict._value_modifiers = (_add_1, _case_fold_string)
# Or
# modifiable_items_dict.ModifiableItemsDict._key_modifiers = staticmethod(str.casefold)
# modifiable_items_dict.ModifiableItemsDict._value_modifiers = [_case_fold_string, _add_1]

del modifiable_items_dict["LOWER"]
del modifiable_items_dict["UPPER"]
modifiable_items_dict.pop("SNAKE_CAse")
modifiable_items_dictionary = modifiable_items_dictionary.ModifiableItemsDict({"lower": 1, "UPPER": 2}, CamelCase=3,
snake_case="FoUR")

modifiable_items_dict["HeLLO"] = 5
print(modifiable_items_dictionary) # {'lower': 2, 'upper': 3, 'camelcase': 4, 'snake_case': 'four'}

print(modifiable_items_dict) # {'camelcase': 4, 'hello': 6}
del modifiable_items_dictionary["LOWER"]
del modifiable_items_dictionary["UPPER"]
modifiable_items_dictionary.pop("SNAKE_CAse")

```
## Example
Let's say that there is a `.json` file that has Url hosts and their IP address.
Our Goal is to load the json data into a dictionary like structure that will have it's items modified during creation, insertion, and retrieval.
This example highlights how to inherit from `ModifiableItemsDict` and had key and value modifiers.
```python
import contextlib
import ipaddress
import json
from typing import Any, Union
modifiable_items_dictionary["HeLLO"] = 5

from modifiable_items_dict import ModifiableItemsDict
print(modifiable_items_dictionary) # {'camelcase': 4, 'hello': 6}
```

def _strip(_value: Any):
if isinstance(_value, str):
_value = _value.strip()
return _value
return _value
## Example

Let's say that there is a `.json` file that has Url hosts and their IP address.
Our Goal is to load the json data into a dictionary like structure that will have it's items modified during creation,
insertion, and retrieval.
This example highlights how to inherit from `ModifiableItemsDict` and had key and value modifiers.

def _case_fold(_value: Any) -> Any:
if isinstance(_value, str):
_case_folded_string: str = _value.casefold()
return _case_folded_string
return _value
```python
import ipaddress

def _to_ipaddress(_value: Any) -> Any:
if isinstance(_value, str):
with contextlib.suppress(ValueError):
_ip_address: Union[
ipaddress.IPv4Address, ipaddress.IPv6Address
] = ipaddress.ip_address(_value)
return _ip_address
import modifiable_items_dictionary

return _value

class HostDict(ModifiableItemsDict):
_key_modifiers = [_strip, _case_fold]
_value_modifiers = [_to_ipaddress]
class HostDict(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = [str.casefold, str.strip]
_value_modifiers = [ipaddress.ip_address]
# Or
# _value_modifiers = @staticmethod(_to_ipaddress)
# _value_modifiers = @staticmethod(ipaddress.ip_address)

_json: str = '{" GooGle.com ": "142.250.69.206", " duckDUCKGo.cOM ": "52.250.42.157"}'

host_dict: HostDict = json.loads(_json, object_hook=HostDict)
browsers = HostDict({" GooGle.com ": "142.250.69.206", " duckDUCKGo.cOM ": "52.250.42.157"})

print(host_dict) # {'google.com': IPv4Address('142.250.69.206'), 'duckduckgo.com': IPv4Address('52.250.42.157')}
print(browsers) # {'google.com': IPv4Address('142.250.69.206'), 'duckduckgo.com': IPv4Address('52.250.42.157')}

_old_browser = host_dict.pop(" gOOgle.com ")
_old_browser = browsers.pop(" gOOgle.com ")
# or
# del host_dict[" GooGle.com "]

host_dict[" BrAvE.com "] = "2600:9000:234c:5a00:6:d0d2:780:93a1"
browsers[" BrAvE.com "] = "2600:9000:234c:5a00:6:d0d2:780:93a1"

print(host_dict) # {'duckduckgo.com': IPv4Address('52.250.42.157'), 'brave.com': IPv6Address('2600:9000:234c:5a00:6:d0d2:780:93a1')}
print(
browsers) # {'duckduckgo.com': IPv4Address('52.250.42.157'), 'brave.com': IPv6Address('2600:9000:234c:5a00:6:d0d2:780:93a1')}
```

### Threading Example
It is easy to add Threading to a `ModifiableItemsDict`.

It is easy to add Threading to a `ModifiableItemsDict`.

*NOTE: Since `ModifiableItemsDict` is not pickable it does not work with Multiprocessing.*

```python
from multiprocessing.pool import ThreadPool as Pool
from string import ascii_letters
from time import perf_counter, sleep
import multiprocessing.pool
import string
import time

from modifiable_items_dict.modifiable_items_dict import ModifiableItemsDict
import modifiable_items_dictionary

pool = Pool(10)
pool = multiprocessing.pool.ThreadPool(10)


def _slow_function(x):
sleep(.05)
time.sleep(.05)
return x

class TimeDictWithThreading(ModifiableItemsDict):

class TimeDictWithThreading(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = (_slow_function,)
_value_modifiers = (_slow_function,)
_map_function = pool.imap_unordered
# or if order matters
# _map_function = pool.imap

class TimeDict(ModifiableItemsDict):


class TimeDict(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = (_slow_function,)
_value_modifiers = (_slow_function,)

iterable = {_letter: _index for _index, _letter in enumerate(ascii_letters)}

iterable = {_letter: _index for _index, _letter in enumerate(string.ascii_letters)}

# Without Threading
start = perf_counter()
start = time.perf_counter()
TimeDict(iterable)
end = perf_counter()
end = time.perf_counter()
print(f"{end - start:.2f} seconds") # 5.54 seconds

# With Threading
start = perf_counter()
start = time.perf_counter()
TimeDictWithThreading(iterable)
end = perf_counter()
end = time.perf_counter()
print(f"{end - start:.2f} seconds") # 0.64 seconds
```
111 changes: 111 additions & 0 deletions examples/examples.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,111 @@
import ipaddress
import multiprocessing.pool
import string
import time

import modifiable_items_dictionary


def simple_example():
import modifiable_items_dictionary

def _add_1(_value):
if isinstance(_value, int):
_value += 1
return _value

def _case_fold_string(_value):
if isinstance(_value, str):
_value = _value.casefold()
return _value

modifiable_items_dictionary.ModifiableItemsDict._key_modifiers = [str.casefold]
modifiable_items_dictionary.ModifiableItemsDict._value_modifiers = (_add_1, _case_fold_string)
# Or
# modifiable_items_dict.ModifiableItemsDict._key_modifiers = staticmethod(_lower)
# modifiable_items_dict.ModifiableItemsDict._value_modifiers = [_lower, _add_1]

modifiable_items_dict = modifiable_items_dictionary.ModifiableItemsDict({"lower": 1, "UPPER": 2}, CamelCase=3,
snake_case="FoUR")

print(modifiable_items_dict) # {'lower': 2, 'upper': 3, 'camelcase': 4, 'snake_case': 'four'}

del modifiable_items_dictionary["LOWER"]
del modifiable_items_dictionary["UPPER"]
modifiable_items_dict.pop("SNAKE_CAse")

modifiable_items_dict["HeLLO"] = 5

print(modifiable_items_dict) # {'camelcase': 4, 'hello': 6}


def simple_inheritance_example():
"""This example shows how to create a Host Dictionary that will casefold keys, strip keys, and convert values to ip addresses
"""

# Inherit from `ModifiableItemsDict` and set the `_key_modifiers` and `_value_modifiers` class variables.
class HostDict(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = [str.casefold, str.strip]
_value_modifiers = [ipaddress.ip_address]
# Or
# _value_modifiers = @staticmethod(ipaddress.ip_address)

browsers = HostDict({" GooGle.com ": "142.250.69.206", " duckDUCKGo.cOM ": "52.250.42.157"})

print(browsers) # {'google.com': IPv4Address('142.250.69.206'), 'duckduckgo.com': IPv4Address('52.250.42.157')}

_old_browser = browsers.pop(" gOOgle.com ")
# or
# del host_dict[" GooGle.com "]

browsers[" BrAvE.com "] = "2600:9000:234c:5a00:6:d0d2:780:93a1"

print(
browsers)
# {'duckduckgo.com': IPv4Address('52.250.42.157'), 'brave.com': IPv6Address('2600:9000:234c:5a00:6:d0d2:780:93a1')}


def threading_example():
"""This example shows how to use Threading with `ModifiableItemsDict`."""

pool = multiprocessing.pool.ThreadPool(10)

def _slow_function(x):
time.sleep(.05)
return x

class TimeDictWithThreading(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = (_slow_function,)
_value_modifiers = (_slow_function,)
_map_function = pool.imap_unordered
# or if order matters
# _map_function = pool.imap

class TimeDict(modifiable_items_dictionary.ModifiableItemsDict):
_key_modifiers = (_slow_function,)
_value_modifiers = (_slow_function,)

iterable = {_letter: _index for _index, _letter in enumerate(string.ascii_letters)}

# Without Threading
start = time.perf_counter()
TimeDict(iterable)
end = time.perf_counter()
print(f"{end - start:.2f} seconds") # 5.54 seconds

# With Threading
start = time.perf_counter()
TimeDictWithThreading(iterable)
end = time.perf_counter()
print(f"{end - start:.2f} seconds") # 0.64 seconds


def main():
simple_example()
simple_inheritance_example()
threading_example()


if __name__ == "__main__":
main()
3 changes: 0 additions & 3 deletions modifiable_items_dict/__init__.py

This file was deleted.

3 changes: 3 additions & 0 deletions modifiable_items_dictionary/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
from modifiable_items_dictionary.modifiable_items_dictionary import ModifiableItemsDict

__all__ = (ModifiableItemsDict.__name__,)
Loading

0 comments on commit 899f0e5

Please sign in to comment.