Implementation Questions [TeachBooks Package Development]

[rlanzafame]

first specific requests:

@CasparJungbacker would be happy to hear your opinion! Also, my apologies for making your code much much more ugly and sloppy than the original version. Any ideas on how to improve that are welcome . At least there are a few more tests now :)
@Tom-van-Woudenberg maybe you have an opinion on item no. 1 and 4?
@BSchilperoort your opinion is always welcome! But if short on time, maybe a quick check if any of these things below would conflict with your toc tool?

Explanation

I recently finished some improvements to the CLI. For example, PR 44 improves teachbooks serve. Along the way I acquired questions for how to implement/modify the package in the future, and I would like your thoughts on how!

1. First a (hopefully) simple one: dev branch collects improvements via PR's which become releases as dev is merged into main. Can set up a PR from dev into main with a title like "Next release". If interested in commenting on local computer setup for development, see Discussion 49.
2. I began the process of permanently changing the location of the temporary folder ./book/.teachbooks/ to ./.teachbooks/ as Issue 41 made it clear that the ./book/ directory was not a good place to keep it (this can change for non-standard repo setups).
3. The CLI serve command now uses global constants (./teachbooks/__init__.py) to enforce default directory structures for an arbitrary book setup (./book/) and build directory (./book/_build/html/), as well as the server working directory (.teachbooks/server/). Does anyone object to using global constants?
4. I would like to allow the user to use a config to override the global constants. What could be an elegant option for doing this, given that it should be 100% optional and easy to write? I am thinking a ./_teachbooks_config.yml would be best, but I am not sure where it would be "used"...would a good approach be to process it in ./teachbooks/__init__.py and then save the global constants (and anything else that comes along) in a state file in the temp directory ./.teachbooks/?
5. Our vision for the package is to provide pre- and post-processing steps for jupyter book build, but this is not formally implemented in the package. It is only implicitly incorporated based on what is in the CLI setup before and after jupyter book build book. I think this is OK for now, but some time in the future we may need to formalize how this works.
6. Python plugins seem like a very practical way to influence the pre- and post-processing steps (item no. 5), as well as help with things like custom configurations (item no. 4). Can someone describe how a user might specify a
7. I think the reference issue could be an excellent case study for the item above as the easiest fix is simply moving a sepcific set of files/subdirectory during the pre-processing step. See Issue 41, as well as the detailed description in Project 17: APA Referencing.

[Tom-van-Woudenberg]

I don't know the answers to any of this.

[BSchilperoort]

1. dev branch collects improvements via PR's which become releases as dev is merged into main. Can set up a PR from dev into main with a title like "Next release".

Yeah that's good common practice when people use the code from github directly. It's not needed as much if people only work with releases (pypi/github releases) as then they would not use the branch version of the code.
Having a PR always open from dev to main could also be nice to have an overview of the code that's not in main yet.

The alternative is to leave main as the development branch, but also have a stable branch for people to use.

3. global constants (./teachbooks/__init__.py) to enforce default directory structures for an arbitrary book setup (./book/) and build directory (./book/_build/html/), as well as the server working directory (.teachbooks/server/). Does anyone object to using global constants?

I wouldn't call these "global constants". Globals are something completely different in programming.
What you have are hardcoded paths.

4. _teachbooks_config.yml would be best, but I am not sure where it would be "used"...would a good approach be to process it in ./teachbooks/__init__.py and then save the global constants (and anything else that comes along) in a state file in the temp directory ./.teachbooks/?

You can setup a config manager. This should be quite straightforward. The builtin python library yaml can read yaml files into a dictionary:

import yaml

# define defaults
config = {"path1": "test/test", "path2": "test/abc"}  # these can also be in a yaml file in the teachbooks package, for consistency.

# update config with info from yaml (replace the string with text read from a file)
config.update(yaml.safe_load("path1: test/def"))

It would be good to add some validation after loading so you can rely on some paths being there or other settings not breaking the workflow elsewhere.

5. Our vision for the package is to provide pre- and post-processing steps for jupyter book build, but this is not formally implemented in the package. It is only implicitly incorporated based on what is in the CLI setup before and after jupyter book build book. I think this is OK for now, but some time in the future we may need to formalize how this works.

Easily extendable pre- and post-processing would be great to have! Although if those pre/post processors need a certain order of execution (e.g., applying preproc_A, preproc_B works, but preproc_B then preproc_A not. This might have to be configured in the teachbooks config.

Python plugins seem like a very practical way to influence the pre- and post-processing steps (item no. 5), as well as help with things like custom configurations (item no. 4). Can someone describe how a user might specify a

You can use entrypoints. They are a bit complex to set up the first time, but once you got the first one working and you have an example how to use it it becomes quite straightforward.
You can give all plugins access to the main teachbooks config as well.

[rlanzafame]

Thanks Bart, this is really helpful! I'll get started on formalizing the teachbooks repo. I think I will go with stable for latest release and change main to develop to make its role explicit; develop will be the default branch.

the yaml config and entry points sound like good solutions too, but will take a bit longer for me to get set up. Maybe a good holiday break hobby :)

[rlanzafame]

repo is now adjusted, documented in README and first minor release is out! https://github.com/TeachBooks/TeachBooks/releases/tag/v0.1.0