Add documentation (#123)

* Initialise mkdocs

* Added mkdocs documentation site

* Added docstrings to main APIs

* Added API reference

* Make docstrings google style

* Fix tests

* Added config documentation page and license details

* Update README.md

* Add contributing guidelines and PR template

* Add ruff to dependencies and styleguide

* Fix links

* Adde workflow to deploy documentation

* Change main to master in workflow files

* Add link to contribution page

* Fix docs requirements extension

* Fix ci

* Match ubuntu and python versions in git workflows

* Update pyproject.toml

* Add installation instructions in README.md

* Update documentation

* Changes to Annotator abstract class to match API documentation and moved lookup back to package data for a default option

* Added github workflow for publishing to pypi

* Formatting

.github/PULL_REQUEST_TEMPLATE.md

@@ -0,0 +1,39 @@
+# Pull Request Template for MiADE
+
+## Description
+
+Please include a summary of the change and which issue is fixed. Also, include relevant motivation and context. List any dependencies that are required for this change.
+
+Fixes # (issue)
+
+## Type of change
+
+Please delete options that are not relevant.
+
+- [ ] Bug fix (non-breaking change which fixes an issue)
+- [ ] New feature (non-breaking change which adds functionality)
+- [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
+- [ ] This change requires a documentation update
+
+## How Has This Been Tested?
+
+Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration.
+
+- [ ] Test A
+- [ ] Test B
+
+## Checklist:
+
+Before submitting your pull request, please review the following checklist:
+
+- [ ] I have performed a self-review of my own code.
+- [ ] I have commented my code, particularly in hard-to-understand areas.
+- [ ] I have made corresponding changes to the documentation.
+- [ ] My changes generate no new warnings.
+- [ ] I have added tests that prove my fix is effective or that my feature works.
+- [ ] New and existing unit tests pass locally with my changes.
+- [ ] Any dependent changes have been merged and published in downstream modules.
+
+## Additional Information:
+
+Any additional information that you would like to provide about the pull request.

.github/workflows/build_documentation.yml

@@ -0,0 +1,21 @@
+name: Build documentation
+
+on:
+  pull_request:
+    branches: [master]
+
+jobs:
+  build:
+    name: Build
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.11
+
+      - name: Build the documentation
+        run: |
+          pip install -r docs-requirements.txt
+          pip install ./
+          mkdocs build

.github/workflows/deploy_documentation.yml

@@ -0,0 +1,30 @@
+name: Publish documentation
+
+on:
+  push:
+    branches:
+      - master
+
+permissions:
+  contents: write
+
+jobs:
+  deploy:
+    runs-on: ubuntu-20.04
+    steps:
+      - uses: actions/checkout@v2
+      - uses: actions/setup-python@v2
+        with:
+          python-version: 3.11
+      - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
+      - uses: actions/cache@v3
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+      - run: |
+          pip install -r docs-requirements.txt
+          pip install ./
+      - name: Build documentation
+        run: mkdocs gh-deploy --force

.github/workflows/publish.yml

@@ -0,0 +1,53 @@
+name: Publish Python 🐍 distribution 📦 to PyPI
+
+on:
+  release:
+    types:
+      - created
+
+jobs:
+  build:
+    name: Build distribution 📦
+    runs-on: ubuntu-latest
+
+    steps:
+    - uses: actions/checkout@v4
+    - name: Set up Python
+      uses: actions/setup-python@v5
+      with:
+        python-version: "3.x"
+    - name: Install pypa/build
+      run: >-
+        python3 -m
+        pip install
+        build
+        --user
+    - name: Build a binary wheel and a source tarball
+      run: python3 -m build
+    - name: Store the distribution packages
+      uses: actions/upload-artifact@v3
+      with:
+        name: python-package-distributions
+        path: dist/
+
+  publish-to-pypi:
+    name: >-
+      Publish Python 🐍 distribution 📦 to PyPI
+    if: startsWith(github.ref, 'refs/tags/')  # only publish to PyPI on tag pushes
+    needs:
+    - build
+    runs-on: ubuntu-latest
+    environment:
+      name: release
+      url: https://pypi.org/p/miade  # Replace <package-name> with your PyPI project name
+    permissions:
+      id-token: write  # IMPORTANT: mandatory for trusted publishing
+    steps:
+    - name: Download all the dists
+      uses: actions/download-artifact@v3
+      with:
+        name: python-package-distributions
+        path: dist/
+    - name: Publish distribution 📦 to PyPI
+      uses: pypa/gh-action-pypi-publish@release/v1
+

.gitignore

@@ -43,6 +43,7 @@ venv.bak/
 *.zip
 src/miade/model_builders/output/
 !src/miade/data/*
+.DS_Store
 
 #testing
 tests/data/

CONTRIBUTING.md

@@ -1,3 +1,94 @@
-## Versioning
+# Contributing to MiADE
+
+Thank you for considering contributing to MiADE!
+
+## Code of Conduct
+
+This project and everyone participating in it is governed by the MiADE Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to [email protected].
+
+## I don't want to read this whole thing I just have a question!!!
+
+INSERT LINK TO SOME SORT OF FORUM
+
+## What should I know before I get started?
+
+### Dependencies
+
+You can find a list of dependencies in our [pyproject.toml](https://github.com/uclh-criu/miade/blob/master/pyproject.toml) file. MiADE is compatible with Python 3.8 and above.
+
+To install the project with the dev dependencies, run:
+
+```bash
+pip install -e .[dev]
+```
+The `-e` flag sets the install to auto-update, useful when developing.
+
+### Testing
+
+MiADE uses [pytest](https://docs.pytest.org/en/8.2.x/), which can be run with:
+
+```bash
+pytest ./tests/*
+```
+> Remember, if using a virtual environment, to install pytest within your environment itself, otherwise you will be using the system python install of pytest, which will use system python and will not find your modules.
+
+### Formatting and Linting
+
+We use [ruff](https://docs.astral.sh/ruff/) for linting and formatting. Run:
+
+```bash
+ruff format 
+ruff check --fix
+```
+
+## How Can I Contribute?
+
+### Reporting Bugs
+
+#### Before Submitting A Bug Report
+
+* Check the documentation for tips on how to fix the issue on your own.
+* Determine which repository the problem should be reported in - MiADE wraps around [MedCAT](https://github.com/CogStack/MedCAT/tree/master?tab=readme-ov-file), so if you encounter an issue related to MedCAT models, it is better to report it to these folks!
+* Check if the issue has already been reported. If it has **and the issue is still open**, add a comment to the existing issue instead of opening a new one.
+
+#### How Do I Submit A (Good) Bug Report?
+
+Bugs are tracked as [GitHub issues](https://github.com/uclh-criu/miade/issues). Explain the problem and include additional details to help maintainers reproduce the problem:
+
+* **Use a clear and descriptive title** for the issue to identify the problem.
+* **Describe the exact steps which reproduce the problem** in as many details as possible.
+* **Provide specific examples to demonstrate the steps**. Include links to files or GitHub projects, or copy/pasteable snippets, which you use in those examples.
+
+
+### Your First Code Contribution
+
+Unsure where to begin contributing to MiADE? You can start by looking through these `beginner` and `help-wanted` issues:
+
+* [Good first issues](https://github.com/uclh-criu/miade/issues?q=is:open+is:issue+label:%22good+first+issue%22) - issues which should only require a few lines of code, and a test or two.
+* [Help wanted issues](https://github.com/uclh-criu/miade/issues?q=is:open+is:issue+label:%22help+wanted%22) - issues which should be a bit more involved than `beginner` issues.
+
+### Pull Requests
+
+The process described here has several goals:
+
+- Maintain MiADE's quality
+- Fix problems that are important to users
+- Engage the community in working toward the best possible version of MiADE
+- Enable a sustainable system for MiADE's maintainers to review contributions
+
+Please follow these steps to have your contribution considered by the maintainers:
+
+1. Follow all instructions in [the template](https://github.com/uclh-criu/miade/blob/documentation/.github/PULL_REQUEST_TEMPLATE.md)
+2. Follow the [styleguides](#styleguides)
+3. After you submit your pull request, verify that all tests are passing
+
+## Styleguides
+
+We use [Google Python style docstring](https://google.github.io/styleguide/pyguide.html).
+
+### Versioning
 Versioning is performed through git tags, which should follow the [semantic versioning](https://semver.org/) approach prefixed with a "v".
-E.g.: `v0.1.2-alpha`
+E.g.: `v0.1.2-alpha`
+
+
+Thank you for reading through the contributing guide and for your interest in making MiADE better. We look forward to your contributions!

README.md

@@ -1,49 +1,86 @@
 # <img src="assets/miade-logo.png" width="40%">
 
 [![Build Status](https://github.com/uclh-criu/miade/actions/workflows/ci.yml/badge.svg?branch=master)](https://github.com/uclh-criu/miade/actions/workflows/ci.yml?query=Tests)
+![License: Elastic License 2.0](https://img.shields.io/badge/License-Elastic%202.0-blue.svg)
 
-A set of tools for extracting formattable data from clinical notes stored in electronic health record systems.
-For the reference server implementation, see: [miade-server](https://github.com/uclh-criu/miade-server).
+A set of tools for extracting formattable data from clinical notes stored in electronic health record systems. Powered by [MedCAT](https://github.com/CogStack/MedCAT) models.
 
-Built with Cogstack's [MedCAT](https://github.com/CogStack/MedCAT) package.
 
-## Contents
-
-1. [Contributors](#Contributors)
-2. [Installing](#Installing)
-3. [Testing](#Testing)
-4. [Contributing](#Contributing)
-5. [Licence](#Licence)
+## Installing
 
+### Install MiADE
 
-## Contributors
+To install the stable release:
+```bash
+pip install miade
+```
 
-| Name            | Email                       |
-|-----------------|-----------------------------|
-| James Brandreth | [email protected]       |
-| Jennifer Jiang  | [email protected] |
+To install the latest development version of MiADE, clone this repository and run:
+```bash
+pip install .
+```
 
-## Installing
+### Downloading models
+You may also need to download these additional models to run MiADE:
 
-As the drug dosage extraction module uses Med7, you will need to download the model:
+[spaCy](https://spacy.io/models/en) - required for MedCAT
 ```bash
-pip install https://huggingface.co/kormilitzin/en_core_med7_lg/resolve/main/en_core_med7_lg-any-py3-none-any.whl
+python -m spacy download en_core_web_md
 ```
-Then install MiADE:
+[med7](https://huggingface.co/kormilitzin/en_core_med7_lg) - required for medication dosage extraction
 ```bash
-pip install -e .
+pip install https://huggingface.co/kormilitzin/en_core_med7_lg/resolve/main/en_core_med7_lg-any-py3-none-any.whl
 ```
-The `-e` flag sets the install to auto-update, useful when developing. Remove for production.
 
-## Testing
+## Quickstart
 
-This project uses pytest tests, which can be run with:
-```bash
-pytest ./tests/*
+Initialise MiADE with the path that you have saved your trained MedCAT models:
+
+```python
+miade = NoteProcessor(Path("path/to/model/dir"))
+```
+Add annotators:
+
+```python
+miade.add_annotator("problems")
+miade.add_annotator("meds/allergies")
+```
+
+Create a note:
+
+```python
+text = "Patient has penicillin allergy with rash"
+note = Note(text)
+```
+
+Extract concepts:
+
+```python
+concepts = miade.process(note)
+
+for concept in concepts:
+    print(concept)
+
+# {name: breaking out - eruption, id: 271807003, category: Category.REACTION, start: 204, end: 208, dosage: None, negex: False, meta: None} 
+# {name: penicillin, id: 764146007, category: Category.ALLERGY, start: 191, end: 201, dosage: None, negex: False, meta: None} 
 ```
-> Remember, if using a virtual environment, to install pytest within your environment itself, otherwise you will be using the system python install of pytest, which will use system python and will not find your modules.
 
 ## Contributing
 See [contributing](CONTRIBUTING.md)
 
+### Maintainers
+
+| Name            | Email                       |
+|-----------------|-----------------------------|
+| James Brandreth | [email protected]       |
+| Jennifer Jiang  | [email protected] |
+
+
+## Acknowledgement
+
+This project wouldn't be possible without the work at [Cogstack](https://cogstack.org/), [spaCy](https://spacy.io/), and [med7](https://huggingface.co/kormilitzin/en_core_med7_lg)!
+
+
 ## Licence
+
+This project is licensed under the Elastic License 2.0. See [LICENSE](https://github.com/uclh-criu/miade/blob/documentation/LICENCE.md) for the full license text.

configs/miade_config.yaml

@@ -9,10 +9,4 @@ general:
     lookup_data_path: ./lookup_data/
     negation_detection: None
     structured_list_limit: 0  # if more than this number of concepts in structure section, ignore concepts in prose
-    disable: []
-    add_numbering: True
-  meds/allergies:
-    lookup_data_path: ./lookup_data/
-    negation_detection: None
-    disable: []
-    add_numbering: False
+    add_numbering: True

docs-requirements.txt

@@ -0,0 +1,4 @@
+mkdocs
+mkdocs-material
+mkdocstrings
+mkdocstrings-python