Incremental anti-join issue

[btheado]

I had trouble with a block Josh provided for automatically binding root elements to external root. According to an email exchange Josh says:

this is definitely a runtime issue. Incremental antijoin is unfortunately pretty complicated, particularly in the context of partial joins.

Josh goes on to say:

In the interim, you can work around the issue by replacing the middle block with:

  widget-root = [#tw-widget-root]
  elem = [#mhtml/element]
  elem != widget-root
  not(
    parent = [#mhtml/element children: elem]
    parent != [#tw-widget-root]
  )

bind
  widget-root.children += elem

which just looks for the widget root separately in the not.

Here is the original code which illustrates the issue:

Use #mhtml/element as a stand-in for #html/element
Create the pretend #tw-widget-root external root
Create parent->child and childless elements

commit
  [#tw-widget-root #mhtml/element text: "#tw-widget-root"]
  [#mhtml/element text: "parent" children: [#mhtml/element text: "child"]]
  [#mhtml/element text: "childless"]

Automatically add parentless elements to #tw-widget-root

search
  widget-root = [#tw-widget-root]
  elem = [#mhtml/element]
  elem != widget-root
  not(
    parent = [#mhtml/element children: elem]
    parent != widget-root
  )

bind
  widget-root.children += elem

Display the children of #tw-widget-root. Expect to see only
'parent' and 'childless', but am also seeing 'child' here

search
  [#tw-widget-root children: child]
  child = [#mhtml/element]
bind
  [#ui/div text: child.text]

[joshuafcole]

I've tracked the issue down. The contents of the antijoin (not) are executed as a separate join node. Joining the results of that join node with the results of the main query happens after that join node has run, and the join node has no knowledge that it's being used as part of an antijoin. Unfortunately, in the case of expressions like !=, this sort of post filtering isn't statically guaranteed to be correct (in fact, != is one of the few cases that could theoretically be special-cased, but doing so isn't fixing the real issue).

The takeaway: This is an architectural issue with 0.3 that I can't think of a trivial solution to. Due to a generally more flexible architecture in 0.4, we've accidentally avoided this issue entirely there. Depending on the 0.4 timeline, it may be better to table this issue and not publicly release 0.3 outside of these previews. If it looks like 0.4 is further off than expected, I'll need to get my hands dirty expanding the regression test suite and then reworking the way antijoin is written.

Unfortunately, this bug (and any real fix) will also impact chooses, since they're implemented using antijoin.