Add modularization documentation (#50)

* add modularization documentation + smaller docs updates

- migrate modularization documentation from dropbox/djinni
- add credits to original authors in readme
- update `--idl-include-path`: also applies to (at)extern
- add documentation on how to preview docs
- add requirements.txt for mkdocs preview dependencies

README.md

@@ -52,3 +52,17 @@ Add the generator as a build requirement in `conanfile.txt`:
 [build_requires]
 djinni-generator/0.3.1
 ```
+
+## Credits
+
+[Thanks goes to these contributors!](https://github.com/cross-language-cpp/djinni-generator/graphs/contributors)
+
+The code in this repository is in large portions copied from [dropbox/djinni](https://github.com/dropbox/djinni) which was created by
+
+- Kannan Goundan
+- Tony Grue
+- Derek He
+- Steven Kabbes
+- Jacob Potter
+- Iulia Tamas
+- Andrew Twyman

docs/cli-usage.md

@@ -43,9 +43,10 @@ djinni \
 | `--help` | Print help  |
 | `--version` | Print version |
 | `--idl <in-file>` | The IDL file with the type definitions, typically with extension `.djinni`. |
-| `--idl-include-path <path> ...` | An include path to search for Djinni `@import` directives. Can specify multiple paths. |
+| `--idl-include-path <path> ...` | An include path to search for Djinni `@import` and `@extern` directives. Can specify multiple paths. |
 
 ### Java
+
 | Argument | Description |
 | -------- | ----------- |
 | `--java-out <out-folder>` | The output for the Java files (Generator disabled if unspecified). |
@@ -135,8 +136,8 @@ djinni \
 | -------- | ----------- |
 | `--cppcli-out <out-folder>` | The output folder for C++/CLI files (Generator disabled if unspecified). |
 | `--cppcli-namespace ...`    | The namespace name to use for generated C++/CLI classes. |
-| `--cppcli-include-cpp-prefix` | The prefix for #include of the main C++ header files from C++/CLI files. |
-| `--cppcli-base-lib-include-prefix ...` | The C++/CLI base support library's include path (default: `djinni/cppcli/`). |
+| `--cppcli-include-cpp-prefix <prefix>` | The prefix for `#include` of the main C++ header files from C++/CLI files. |
+| `--cppcli-base-lib-include-prefix <prefix>` | The C++/CLI base support library's include path (default: `djinni/cppcli/`). |
 
 
 ### Yaml Generation

docs/developer-guide.md

@@ -119,6 +119,19 @@ On Windows the file must be renamed to `djinni.bat` to make it executable.
 13. Folder containing the integration testing code.
 14. djinni-generator source code.
 
+## Preview Documentation
+
+The documentation in `docs` will be rendered as a part of [djinni.xlcpp.dev](https://djinni.xlcpp.dev/).
+
+You can preview how the docs will look like:
+
+```sh
+# install required dependencies
+pip install -r mkdocs-requirements.txt
+# render a live preview of the docs under http://127.0.0.1:8000
+mkdocs serve 
+```
+
 ## Release process
 
 To release a new version of the generator, the following steps must be followed:

docs/modularization.md

@@ -0,0 +1,172 @@
+# Modularization and Library Support
+
+When generating the interface for your project and wish to make it available to other users in all of
+C++/Objective-C/Java/C# you can tell Djinni to generate a special YAML file as part of the code generation process.
+This file then contains all the information Djinni requires to include your types in a different project.
+Instructing Djinni to create these YAML files is controlled by the [YAML generation parameters](cli-usage.md#yaml-generation).
+
+!!! caution
+
+    External types defined in YAML are not yet supported for Python.
+
+## YAML file structure
+
+Such a YAML file looks as follows:
+
+```yaml
+---
+name: mylib_record1
+typedef: 'record +c deriving(eq, ord)'
+params: []
+prefix: 'mylib'
+cpp:
+  typename: '::mylib::Record1'
+  header: '"MyLib/Record1.hpp"'
+  byValue: false
+objc:
+  typename: 'MLBRecord1'
+  header: '"MLB/MLBRecord1.h"'
+  boxed: 'MLBRecord1'
+  pointer: true
+  hash: '%s.hash'
+objcpp:
+  translator: '::mylib::djinni::objc::Record1'
+  header: '"mylib/djinni/objc/Record1.hpp"'
+java:
+  typename: 'com.example.mylib.Record1'
+  boxed: 'com.example.mylib.Record1'
+  reference: true
+  generic: true
+  hash: '%s.hashCode()'
+jni:
+  translator: '::mylib::djinni::jni::Record1'
+  header: '"Duration-jni.hpp"'
+  typename: jobject
+  typeSignature: 'Lcom/example/mylib/Record1;'
+cs:
+  translator: '::djinni::Record1_h'
+  header: '"Record1-cs.hpp"'
+  typename: 'Record1^'
+  reference: false
+---
+name: mylib_interface1
+typedef: 'interface +j +o +s'
+    (...)
+---
+name: mylib_enum1
+typedef: 'enum'
+    (...)
+```
+
+Each document in the YAML file describes one extern type.
+
+You can also check these files for some real working examples of what you can do with it:
+
+- [`date.yaml`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/djinni/vendor/third-party/date.yaml)
+- [`duration.yaml`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/djinni/vendor/third-party/duration.yaml)
+
+## Usage
+
+To use a library type in your project simply include it in your IDL file and refer to it using its name identifier:
+
+```
+@extern "mylib.yaml"
+
+client_interface = interface +c {
+  foo(): mylib_record1;
+}
+```
+
+## Defining custom types
+
+The YAML files can be created by hand as long as you follow the required format.
+This allows you to support types not generated by Djinni.
+
+See [`duration.yaml`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/djinni/vendor/third-party/duration.yaml)
+and the accompanying translators in [`Duration-objc.hpp`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/handwritten-src/cpp/Duration-objc.hpp),
+[`Duration-jni.hpp`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/handwritten-src/cpp/Duration-jni.hpp)
+and [`Duration-cs.hpp`](https://github.com/cross-language-cpp/djinni-support-lib/blob/main/test-suite/handwritten-src/cpp/Duration-cs.hpp) for an advanced example.
+
+Handwritten translators implement the following concept:
+
+```cpp
+// For C++ <-> Objective-C
+struct Record1
+{
+    using CppType = ::mylib::Record1;
+    using ObjcType = MLBRecord1*;
+
+    static CppType toCpp(ObjcType o) { return /* your magic here */; }
+    static ObjcType fromCpp(CppType c) { return /* your magic here */; }
+
+    // Option 1: use this if no boxing is required
+    using Boxed = Record1;
+    // Option 2: or this if you do need dedicated boxing behavior
+    struct Boxed
+    {
+        using ObjcType = MLBRecord1Special*;
+        static CppType toCpp(ObjcType o) { return /* your magic here */; }
+        static ObjcType fromCpp(CppType c) { return /* your magic here */; }
+    }
+};
+```
+
+```cpp
+// For C++ <-> JNI
+#include "djinni_support.hpp"
+struct Record1
+{
+    using CppType = ::mylib::Record1;
+    using JniType = jobject;
+
+    static CppType toCpp(JniType j) { return /* your magic here */; }
+    // The return type *must* be LocalRef<T> if T is not a primitive!
+    static ::djinni::LocalRef<jobject> JniType fromCpp(CppType c) { return /* your magic here */; }
+
+    using Boxed = Record1;
+};
+```
+
+```cpp
+// For C++ <-> C++/CLI
+public ref class Record1 {
+public:
+    // Record1 public properties
+internal:
+    using CppType = ::mylib::Record1;
+    using CsType = Record1^;
+    static CppType ToCpp(CsType cs) { return /* your magic here */; }
+    static CsType FromCpp(const CppType& cs) { return /* your magic here */; }
+private:
+    // Record1 properties' backing fields
+}
+```
+
+For `interface` classes the `CppType` alias is expected to be a `std::shared_ptr<T>`.
+
+Be sure to put the translators into representative and distinct namespaces.
+
+If your type is generic the translator takes the same number of template parameters.
+At usage each is instantiated with the translators of the respective type argument.
+
+```cpp
+template<class A, class B>
+struct Record1
+{
+    using CppType = ::mylib::Record1<typename A::CppType, typename B::CppType>;
+    using ObjcType = MLBRecord1*;
+
+    static CppType toCpp(ObjcType o)
+    {
+        // Use A::toCpp() and B::toCpp() if necessary
+        return /* your magic here */;
+    }
+    static ObjcType fromCpp(CppType c)
+    {
+        // Use A::fromCpp() and B::fromCpp() if necessary
+        return /* your magic here */;
+    }
+
+    using Boxed = Record1;
+};
+```

mkdocs-requirements.txt

@@ -0,0 +1,3 @@
+# dependencies for mkdocs documentation
+mkdocs>=1.1
+mkdocs-material>=7.1.3

mkdocs.yml

@@ -1,7 +1,16 @@
 site_name: djinni-generator
+markdown_extensions:
+  - pymdownx.tabbed
+  - pymdownx.highlight
+  - pymdownx.superfences
+  - pymdownx.details
+  - admonition
+theme:
+  name: material
 nav:
   - setup.md
   - idl.md
   - cli-usage.md
   - generated-code-usage.md
+  - modularization.md
   - developer-guide.md