docs: Configuring of requirements

Revised logic, so now feature=true means that those requirements are
enabled, not disabled.
Added a decision record to lay out why 'hide' was choosen
Adapted README to make it a bit clearer.
Added wrappers for filtering functions used by needpie, etc.

docs/BUILD

@@ -43,8 +43,8 @@ load("@rules_pkg//pkg:mappings.bzl", "pkg_files")
 load("@rules_pkg//pkg:tar.bzl", "pkg_tar")
 load("@rules_python//python:pip.bzl", "compile_pip_requirements")
 load("@rules_python//sphinxdocs:sphinx.bzl", "sphinx_build_binary", "sphinx_docs")
-load("//tools/testing/pytest:defs.bzl", "score_py_pytest")
 load("//tools/feature_flags:feature_flags.bzl", "define_feature_flags")
+load("//tools/testing/pytest:defs.bzl", "score_py_pytest")
 
 define_feature_flags(name = "filter_tags")
 
@@ -80,17 +80,14 @@ sphinx_docs(
         "manual",
     ],
     tools = [
+        ":filter_tags",
         ":plantuml",
-        ":filter_tags"
     ],
 )
 
 sphinx_build_binary(
     name = "sphinx_build",
-    deps = sphinx_requirements + [
-        "//docs/_tooling/sphinx_extensions",
-        "//docs/_tooling/sphinx_extensions/sphinx_extensions/build:modularity",
-    ],
+    deps = sphinx_requirements,
 )
 
 # In order to update the requirements, change the `requirements.txt` file and run:
@@ -129,10 +126,7 @@ py_binary(
     name = "incremental",
     srcs = ["_tooling/incremental.py"],
     data = [":flags_file"],
-    deps = sphinx_requirements + [
-        "//docs/_tooling/sphinx_extensions",
-        "//docs/_tooling/sphinx_extensions/sphinx_extensions/build:modularity",
-    ],
+    deps = sphinx_requirements,
 )
 
 # Virtual python environment for working on the documentation (esbonio).
@@ -184,6 +178,25 @@ score_py_pytest(
     deps = [":score_metamodel"],
 )
 
+py_library(
+    name = "modularity",
+    srcs = glob(["_tooling/extensions/modularity/**/*.py"]),
+    imports = ["_tooling/extensions"],
+    visibility = ["//visibility:public"],
+)
+
+score_py_pytest(
+    name = "modularity_test",
+    size = "small",
+    srcs = glob(["_tooling/extensions/modularity/tests/**/*.py"]),
+    args = [
+        "-s",
+        "-v",
+    ],
+    visibility = ["//visibility:public"],
+    deps = [":modularity"],
+)
+
 filegroup(
     name = "flags_file",
     srcs = [":filter_tags"],

docs/_tooling/sphinx_extensions/sphinx_extensions/build/modularity/README.md → docs/_tooling/extensions/modularity/README.md

@@ -10,14 +10,16 @@ The extension consists of two main components:
 1. `feature_flags.bzl`: A Bazel translation layer that converts commands into tags
 2. `modularity`: A Sphinx extension that filters documentation based on feature flags
 
-It currently functions as a blacklist.  
-**The extension is setup so it only disables requirements where *all* tags are contained within the filtered ones.**  
+The list of tags to filter is built in 'reverse' from the features enabled. In an empty configuration all tags will be added to the 'disable' list. By adding a feature you remove the corresponding tags from the list, therefore enabling requirements with those tags.
+
+**The extension is set up so it only disables requirements where *all* tags are contained within the filtered ones.**  
+
 
 ## Usage
 
 ### Command Line
 
-Disable features when building documentation:
+Configure features when building documentation:
 
 ```bash
 docs build //docs:docs --//docs:feature1=true
@@ -40,7 +42,7 @@ Mapping of features to tags is defined as follows:
 
 ```bzl
 FEATURE_TAG_MAPPING = {
-    "feature1": ["some-ip", "tag2"], # --//docs:feature1=true -> will expand to 'some-ip', 'tag2'
+    "feature1": ["some-ip", "tag2"], # --//docs:feature1=true -> will display requirements with those tags
     "second-feature": ["test-feat", "tag6"],
 }
 ```
@@ -65,16 +67,17 @@ def define_feature_flags(name):
 As can be seen here, each flag needs to be registered as well as have a default value defined. 
 After adding new flags it's possible to call them via the `--//docs:<flag-name>=<value>` command
 
-The `feature_flag.bzl` file will after parsing the flags and translating them write a temporary file with all to be disabled tags.
+If a feature is provided it's corresponding tags are removed from the 'disable' list. Therefore requirements with these tags are now **enabled**.  
+E.g. `bazel build //docs:docs --//docs:feature1=true` -> all requirements with the tags `some-ip` and `tag2` are now enabled and will be rendered in the final HTML output.
 
-### Extension (modularity)
+### Sphinx Extension (modularity)
 
 The extension processes the temporary flag file and disables documentation sections tagged with disabled features.
 
 #### Use in requirements
 
 All requirements which tags are **all** contained within the tags that we look for, will be disabled.  
-Here an eaxmple to illustrate the point.  
+Here an example to illustrate the point.  
 We will disable the `test-feat` tag via feature flags. If we take the following example rst:
 ```rst
 .. tool_req::  Test_TOOL
@@ -99,8 +102,8 @@ We will disable the `test-feat` tag via feature flags. If we take the following
 We can then build it with our feature flag enabled via `bazel build //docs:docs --//docs:second-feature=true` 
 This will expand via our translation layer `feature_flag.bzl` into the tags `test-feat` and `tag5`.  
 
-**The extension is setup so it only disables requirements where *all* tags are contained within the filtered ones.**  
-In the rst above `TEST_TOOL_REQ` has the `test-feat` tag but it also has the `feature1` tag, therefore it wont be disabled.
+**The extension is set up so it only disables requirements where *all* tags are contained within the filtered ones.**  
+In the rst above `TEST_TOOL_REQ` has the `test-feat` tag but it also has the `feature1` tag, therefore it won't be disabled.
 In contrast, TEST_STKH_REQ_20 has only the `test-feat` tag, therefore it will be disabled and removed from any links as well.
 
 If we now look at the rendered HTML:
@@ -110,11 +113,18 @@ We can confirm that the `TEST_STKH_REQ_1` requirement is gone and so is the refe
 
 
 *However, keep in mind that in the source code the actual underlying RST has not changed, it's just the HTML.*  
-This means that one can still see all the requirements there and also if searching for it will still find the document where it is mentioned. 
+This means that one can still see all the requirements there and also if searching for it will still find the document where it was mentioned. 
 
 
 ### How the extension achieves this? 
 
-The extension uses a sphinx-needs buildin option called `hide`. If a need has the `hide=True` it will not be shown in the final HTML.  
-It also gatheres a list of all 'hidden' requirements as it then in a second iteration removes these from any of the possible links.
+The extension uses a sphinx-needs built-in option called `hide`. If a need has the `hide=True` it will not be shown in the final HTML.  
+It also gathers a list of all 'hidden' requirements as it then in a second iteration removes these from any of the possible links.
+
+### Special cases
+
+Needpie, Needtable etc. are special cases as these are not requirements. There are two wrappers inside `filter_overwrite.py` that add the general functionality of hiding all requirements that have `hide==True`. Therefore enabling normal use, without special restrictions needed to be adhered to.
+
+### Decision Record
 
+Please see [Decision Record](/docs/_tooling/extensions/modularity/decision_record.md) for more information.

docs/_tooling/sphinx_extensions/sphinx_extensions/build/modularity/modularity.py → docs/_tooling/extensions/modularity/__init__.py

@@ -1,17 +1,30 @@
+# *******************************************************************************
+# Copyright (c) 2025 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# SPDX-License-Identifier: Apache-2.0
+# *******************************************************************************
 from pprint import pprint
 from sphinx.application import Sphinx
-from sphinx_needs.data import SphinxNeedsData, NeedsInfoType
+from sphinx_needs.data import SphinxNeedsData
 from sphinx_needs.config import NeedsSphinxConfig
 from sphinx.environment import BuildEnvironment
 from sphinx.util import logging
-from copy import deepcopy
+from .filter_overwrite import wrap_filter_common
 import os
 import json
 
+
 logger = logging.getLogger(__name__)
 
 
-def read_filter_tags(app: Sphinx):
+def read_filter_tags(app: Sphinx) -> list:
     """
     Helper function to read in the 'filter_tags' provided by `feature_flag.bzl`.
 
@@ -30,19 +43,21 @@ def read_filter_tags(app: Sphinx):
     assert hasattr(
         app.config, "filter_tags_file_path"
     ), "Config missing filter_tags_file_path, this is mandatory."
-    print(f"Attempting to read filter tags from: {app.config.filter_tags_file_path}")
+    filter_file = app.config.filter_tags_file_path
+    logger.debug(f"Found the following filter_tags_file_path: {filter_file}")
     try:
-        with open(app.config.filter_tags_file_path, "r") as f:
+        with open(filter_file, "r") as f:
             content = f.read().strip()
             filter_tags = [tag.strip() for tag in content.split(",")] if content else []
-            print(f"Successfully read filter tags: {filter_tags}")
+            logger.debug(f"found: {len(filter_tags)} filter tag.")
+            logger.debug(f"filter tags found: {filter_tags}")
             return filter_tags
     except Exception as e:
-        print(f"Could not read filter tags. Error: {e}")
-        return []
+        logger.error(f"could not read file: {filter_file}. Error: {e}")
+        raise e
 
 
-def hide_needs(app: Sphinx, env: BuildEnvironment):
+def hide_needs(app: Sphinx, env: BuildEnvironment) -> None:
     """
     Function that hides needs (requirements) if *all* of their tags are in the specified tags to hide.
     Also deletes any references to hidden requirements, e.g. in 'satisfies' option.
@@ -52,16 +67,18 @@ def hide_needs(app: Sphinx, env: BuildEnvironment):
         env: The current running BuildEnvironment, this will be supplied automatically
     """
     filter_tags = read_filter_tags(app)
-    Need_Data = SphinxNeedsData(env)
-    needs = Need_Data.get_needs_mutable()
+    need_data = SphinxNeedsData(env)
+    needs = need_data.get_needs_mutable()
     extra_links = [x["option"] for x in NeedsSphinxConfig(env.config).extra_links]
-    rm_needs_docs = set()
     rm_needs = []
     for need_id, need in needs.items():
         if need["tags"] and all(tag in filter_tags for tag in need["tags"]):
             rm_needs.append(need_id)
             need["hide"] = True
 
+    logger.debug(f"found {len(rm_needs)} requirements to be disabled.")
+    logger.debug(f"requirements found: {rm_needs}")
+
     # Remove references
     for need_id in needs:
         for opt in extra_links:
@@ -72,14 +89,12 @@ def hide_needs(app: Sphinx, env: BuildEnvironment):
 
 
 def setup(app):
-    app.add_config_value("filter_tags", [], "env", [list])
-    app.add_config_value(
-        "filter_tags_file_path", None, "env"
-    )  # Change default from "" to None
-
-    # Add validation
+    logger.debug("modularity extension loaded")
+    app.add_config_value("filter_tags", [], "env")
+    app.add_config_value("filter_tags_file_path", None, "env")
     app.connect("env-updated", hide_needs)
 
+    wrap_filter_common()
     return {
         "version": "1.0",
         "parallel_read_safe": True,

docs/_tooling/extensions/modularity/decision_record.md

@@ -0,0 +1,47 @@
+# Decision Record: How to disabale requirements
+
+## Context
+[Feature Flags](https://eclipse-score.github.io/score/process/guidance/feature_flags/index.html#feature-flags) require the functionality
+of disable or enabeling requirements based on which feature flags were provided, in order to build relevant documentation only.
+This decision record will contain what implementation was choosen to enable this feature.
+
+## Decision
+Use the 'hide' option from sphinx-needs objects to remove/disable requirements.
+
+## Chosen Solution: 'hide' Option
+The 'hide' option was selected as the primary implementation method.
+
+### Advantages
+- Can be set programmatically with minimal complexity
+- Maintains document structure integrity
+- Provides clear control over element visibility
+- Predictable behavior in programmatic contexts
+
+### Negatives
+- Requirements still appear in the rst via 'view source code' 
+- Information only saved inside the `NeedsInfoType` objects
+
+## Alternatives Considered
+
+### Alternative 1: `:delete:` Option
+**Why Not Chosen:**
+- Lacks programmatic control capabilities. Can only be set hardcoded inside rst files  
+
+More info: [Delete option docs](https://sphinx-needs.readthedocs.io/en/latest/directives/need.html#delete)
+
+### Alternative 2: `del_need`  Method
+**Why Not Chosen:**
+- Introduces complexity in document node management
+- Creates challenges in locating correct manipulation points
+- Risk of document structure corruption during build
+- Increases maintenance overhead due to complex node relationships  
+
+More info: [del_need docs](https://sphinx-needs.readthedocs.io/en/latest/api.html#sphinx_needs.api.need.del_need)
+
+### Alternative 3: `variations` Approach
+**Why Not Chosen:**
+- Filtering behavior appears inconsistent or non functional when used programmatically
+- Requires hardcoding in configuration files for reliable operation  
+
+More info: [variations docs](https://sphinx-needs.readthedocs.io/en/latest/configuration.html#needs-variants)
+

docs/_tooling/extensions/modularity/filter_overwrite.py

@@ -0,0 +1,82 @@
+# *******************************************************************************
+# Copyright (c) 2025 Contributors to the Eclipse Foundation
+#
+# See the NOTICE file(s) distributed with this work for additional
+# information regarding copyright ownership.
+#
+# This program and the accompanying materials are made available under the
+# terms of the Apache License Version 2.0 which is available at
+# https://www.apache.org/licenses/LICENSE-2.0
+#
+# SPDX-License-Identifier: Apache-2.0
+# *******************************************************************************
+from sphinx.application import Sphinx
+from sphinx_needs.views import NeedsView
+from sphinx_needs.data import NeedsInfoType, NeedsFilteredBaseType
+from sphinx_needs.config import NeedsSphinxConfig
+from sphinx_needs.views import NeedsAndPartsListView
+from docutils import nodes
+from sphinx_needs.debug import measure_time
+
+
+#                    ╭──────────────────────────────────────────────────────────────────────────────╮
+#                    │             Wrapping filter funcs to enable 'hide' to be respected           │
+#                    ╰──────────────────────────────────────────────────────────────────────────────╯
+
+
+def wrap_filter_common():
+    """
+    Wrapper function to avoid circular imports by importing filter_common at runtime instead of module level.
+
+    This wraps two functions used by sphinx_needs directives 'needtable' etc. in order to ensure that the
+    hidden requirements are not showing up in the results.
+    """
+    from sphinx_needs import filter_common
+
+    orig_common = filter_common.process_filters
+    orig_parts = filter_common.filter_needs_parts
+
+    @measure_time("filtering")
+    def common_wrapper(
+        app: Sphinx,
+        needs_view: NeedsView,
+        filter_data: NeedsFilteredBaseType,
+        origin: str,
+        location: nodes.Element,
+        include_external: bool = True,
+    ):  # Used by needlist, needtable and needflow.
+        needs_view = needs_view._copy_filtered(
+            i["id"] for i in needs_view.Values() if i["hide"] != True
+        )
+        return orig_common(
+            app, needs_view, filter_data, origin, location, include_external
+        )
+
+    @measure_time("filtering")
+    def parts_wrapper(
+        needs: NeedsAndPartsListView,
+        config: NeedsSphinxConfig,
+        filter_string: None | str = "",
+        current_need: NeedsInfoType | None = None,
+        *,
+        location: tuple[str, int | None] | nodes.Node | None = None,
+        append_warning: str = "",
+        strict_eval: bool = False,
+    ):  # Used by needpie
+        if filter_string is not None and filter_string != "":
+            filter_string += " and hide != True"
+        else:
+            filter_string = "hide != True"
+        return orig_parts(
+            needs,
+            config,
+            filter_string,
+            current_need,
+            location=location,
+            append_warning=append_warning,
+            strict_eval=strict_eval,
+        )
+
+    # Patch the original module
+    filter_common.process_filters = common_wrapper
+    filter_common.filter_needs_parts = parts_wrapper