Data plane writable per table entry state v1

[jafingerhut]

No description provided.

[jafingerhut]

I have labeled this PR as experimental, meaning it is not intended for review to be merged into any P4.org specification at this time.

I created this as a PR to the language spec for one purpose only: for review by others interested in this experimental feature, and for anyone who may want to implement or write test cases for this experimental feature. The experimental feature is intended to address this p4c issue: #1177

The keyword rmw can be considered a placeholder at this time, but notice that it is much easier to change it when there is not yet an experimental implementation of this idea, and code using that experimental implementation, so suggest alternative keywords soon if you prefer something else.

I chose to go with an explicit keyword for data-plane-writable action parameters, as opposed to just saying "any directionless parameter modified in the action body", in this PR for two reasons:

• It makes the intent of the P4 developer more explicit. If we said that "any directionless parameter may be written in the action body, even by calling sub-actions, extern functions, or extern methods as an out or inout parameter of that called entity", then it would become fairly easy to do so unintentionally. Data-plane-writable action parameters are likely to "cost more" than action parameters that are read-only in the data plane, so making that cost clear is important, as well as for human reasoning about the possible code behaviour.
• It makes the two cases in the spec (directionless vs. being declared rmw) easier to talk about separately in the specification.

[jafingerhut]

Control plane API notes. Comments/questions/corrections welcome in follow-up comments on this PR, and Andy will try to reflect those by editing this comment in the future.

BEFORE the addition of actions with parameters declared rmw, when a table entry is added in any of the following ways:

• A control plane API call adds an entry to a table
• An action with parameters is specified in a const entries list of table entries
• An action with parameters is specified as a default_action in a table definition
• In the PNA architecture, the extern function add_entry is called to add an entry to the table from within the data plane, without the control plane having to do so.

... it is required that the list of parameters provides the initial value for all directionless action parameters. After that, some control plane APIs, such as TDI and P4Runtime API, allow modifying the values of these directionless action parameters from the control plane. Typically, the expectation is that such a modify action occurs "atomically" relative to data packet processing, i.e. every data packet processed sees either all of the old directionless action parameter values, or all of the new directionless action parameters.

AFTER the addition of actions with parameter declared rmw, it makes a lot of sense if values must be specified for all action parmaeters declared rmw, too. This is so regardless of which of the conditions above causes the entry to be added.

It also makes sense to consider the possibility of defining a control plane API that can modify the values of all directionless and rmw action parameters of an existing table entry, and if this is supported, the simplest definition to understand as a P4 developer is that the control plane operation is atomic relative to data packet processing, i.e. every execution of an action on the modified table entry appears as if it occurs completely before the control plane modification of the entry, or completely after.

Given the complexity of implementing such behavior, and given that modifying rmw action parameters from the control plane might have far fewer desired use cases than modifying directionless action parameters from the control plane, it is understandable if some target implementers might choose not to implement such a modification action from the control plane.

One might also consider the possibility of control plane table entry modify actions that can modify directionless action parameters, but leave the rmw action parameters unchanged. It is not clear if there is a strong use case in providing this operation -- it is mentioned here for target implementers and P4 developers to think about whether it is useful to them, and/or reasonable complexity to implement.

All targets should be able to support deleting table entries, either from the control plane, or via the new PNA capability of defining the table property pna_idle_timeout = PNA_IdleTimeout_t.AUTO_DELETE. Some target implementers might choose NOT to implement control plane table entry deletion for tables with pna_idle_timeout = PNA_IdleTimeout_t.AUTO_DELETE, but to restrict deletion of entries in such tables to be done only by the data plane.

[jnfoster]

I'm sorry, the 5pm ET meetings have conflicted with another, so I haven't been able to participate yet.

As a sanity check to make sure I understand: the state associated with an rmw parameter lives in the data plane, right?

For example, consider the following code:

control(inout bit<8> x) { 
  action a(rmw bit<8> y) { y = y + 1; x = y; }
  table t { actions = { a; } }
  apply {
    t.apply();
  } 
} 

Assuming

• The control plane inserts a single entry with action a where y is initialized to 0
• The control c is executed twice
  Do we know that the final value of x copied out to the caller will be 2?

[jafingerhut]

AI for Andy: Mention something in this PR about atomicity of action execution, and options for increasing performance but perhaps reducing safety by reducing the "size" of the block of code that is atomic. Add link to PNA issue comment that talks about acquire/release per-table-entry lock explicitly here.

Some may suggest the existing @atomic annotation for this purpose. See this issue #1108 proposing that @atomic be replaced with something more explicit than an annotation, especially a comment from 2023-Apr-03 with a test program demonstrating that today's latest p4c might move code from inside of a block marked @atomic to outside of that block during the midend pass LocalCopyPropagation, when that seems to be unsafe.

[jafingerhut]

I'm sorry, the 5pm ET meetings have conflicted with another, so I haven't been able to participate yet.

As a sanity check to make sure I understand: the state associated with an rmw parameter lives in the data plane, right?

For example, consider the following code:

control(inout bit<8> x) { 
  action a(rmw bit<8> y) { y = y + 1; x = y; }
  table t { actions = { a; } }
  apply {
    t.apply();
  } 
} 

Assuming

• The control plane inserts a single entry with action a where y is initialized to 0
• The control c is executed twice
  Do we know that the final value of x copied out to the caller will be 2?

Yes. At least, I cannot think of any exceptions to the definition of this feature I have in mind that would allow any other possible result.

That is, assuming further that each of the two times that the control c is executed, the same entry referred to in that sequence of events is matched each time.

[jafingerhut]

Notes from 2023-Apr-07 meeting discussing this PR:

• Add note about default_action effectively having its own "default entry" containing the values of rmw parameters that can be modified in data plane, and this default entry is separate from any actions of installed keys in that table.

• Add short example of "two sets of parens" action definitions like this:

action(in bit<8> x) (data bit<16> y, state bit<24> z) {
}

where x is existing data-plane-supplied in parameter, data is like current directionless action parameter, and state is like the new proposed rmw parameter. (or should those parameter lists be in the opposite order? but that may be confusing with the current relative order for parsers/controls)

The intent of the two sets of parentheses is to be more like construction of parsers and controls, which have a similar capability that separate construction time parameters, vs. run-time-varying parameters.

• Andy to volunteer to give short heads up notice of this at next P4 API work group meeting if the co-chairs are willing to spend a little time on it. Not a complete proposal, but point them at the comment with some control plane API thoughts, and let people think about it off-line afterwards.

[jfingerh]

Notes from 2023-Apr-14 meeting on this topic:

AI Mario will check whether compiler developers at AMD can make Friday Apr 21 meeting, and also Mihai Budiu, to discuss high level questions on experimental implementation approach, e.g. implement on the main branch OK? (hopefully yes) Changes to IR needed that Mihai would recommend? etc.

AI Andy: Update this PR with generalization suggested by Nate Foster for calling actions with rmw parameters explicitly. Basically in that case rmw parameters are treated like inout parameters.

Control plane API discussion notes:

• At a minimum, the control plane API should be able to provide values for all directionless and rmw action parameters when a new table entry is added.

• Optionally, some targets may also choose to implement the ability for control plane to modify an existing entry, and such a modify allows control plane to provide new values to all directionless and rmw action parameters. Such writes happen "between" packet processing, i.e. the control plane write can be serialized with packets that execute the action on that entry in some order.

  • TODO: Does today's P4RUntime API spec say whether modify of action parameters is atomic relative to data packet processing? Andy's guess: Even if it doesn't, control plane writers probably assume or hope this.

• Optionally, some target mal also choose to implement the ability for control plane to read an existing entry, and the read result will contain some "consistent snapshot" of all rmw parameter values as they were "between processing packets". Again, control plane reads are serialized relative to data packet processing, in some order.

• Is it desirable to have control plane API writes that ONLY modify directionless action parameters, but leave the rmw parameter values as they currently are? Or write arbitrary subsets of directionless and rmw action parameters? That seems to get complicated for an implementation pretty quickly, although possible for some kinds of targets perhaps.

• Note the workaround for enabling control plane to read-modify-write from the control plane of rmw action parameters is for the control to inject "special data packets" that trigger the execution of the action, and the P4 developer writes cases in the action that cover the desired update behavior.

• Are there use cases where developers might not care that writes are "atomic" relative to data packet processing?

[jafingerhut]

A few notes from 2023-Apr-21 meeting on this topic:

Q: Why not use inout instead of rmw for these action parameters that are data-plane writable?

A: Mainly because that would be a backwards-incompatible change. Direction in, out, and inout action parameters already have a defined meaning in the language spec: they are parameters that must be specified in the P4 source code at the time the action is called (e.g. if the action is called from a control, or from the body of another action), or when the action is placed into the actions list of a table definition. Such parameters never come from the control plane when a table entry is added.

Thus by defining a new keyword qualifier like rmw for data-plane writable action parameters, existing P4 programs that use direction inout continue to mean the same thing that they have meant before this enhancement is added to the language.

Q: Is it reasonable for such an experimental change to p4c to be merged into the main branch "soon" (e.g. this year), and even if the feature remains experimental for a year or two after that time, the feature is present in the main branch of the p4c repository?

A: Yes, that should be fine, as long as it does not change the meaning of P4 programs that do not use this new feature. There was some question of whether there should be a command line option to enable (or disable?) this experimental feature, and/or a warning message whenever a new rmw action parameter is declared, which some P4 developers could use to trigger a build failure if they wanted to avoid using this new feature.

[jafingerhut]

@mariobaldi Just a reminder if you could check on any expected time frame in which the prototype of this feature might be implemented in p4c

[mariobaldi]

The AMD Pensando compiler team expects to start working on the feature at the beginning of 2024.