NTTable fields are sorted alphabetically instead of by insertion order

[ericonr]

Describe the bug

I have a table with "Timestamp" and "Event" columns, and originally used "value.A" and "value.B" for its columns. However, after taking a look at the spec, which mentions that

There is no normative requirement that the field names of value match the strings in labels.

I thought it might make sense to get more descriptive value names: "value.timestamp" and "value.event". However, that broke my NTTable.

My database declares the timestamp waveform before the event one, but pvinfo now lists the fields in the wrong order, and the values are indeed switched in pvget/pvmonitor output:

Type:
    epics:nt/NTTable:1.0
        structure record
            structure _options
                int queueSize
                boolean atomic
        string[] labels
        structure value
            double[] event
            double[] timestamp

To Reproduce

Seems trivial, but I can provide a minimal example if necessary.

Expected behavior

I would have expected the field ordering to respect my declaration order. I don't know if that's actually viable, though, given that users probably expect declarations using value.[A-Z] to be properly sorted...

I tried using epics-base's implementation to compare, but it simply tells me "value.timestamp" is an invalid field name. So it seems to require value.[A-Z].

If sorting is required, I think reasonable fixes could be simply rejecting more complex value field names, or documenting that sorting always happens, so custom field names have limited usefulness.

Information (please complete the following):

• PVXS Version or Git commit ID: 1.3.1
• EPICS Base Version: 7.0.7
• EPICS_HOST_ARCH: linux-x86_64

[mdavidsaver]

To Reproduce

Seems trivial, but I can provide a minimal example if necessary.

I think an example, or better a unittest case, will be necessary.

I extended the NTTable case with 87abad5, and think that it shows the order being preserved.

[ericonr]

I'm having some trouble with expressing it as a test, but I have a reproducer:

record(aai, "$(N)LabelsOrder_"){
	field(FTVL, STRING)
	field(NELM, 2)
	field(INP, {const:["Label $(VALA)", "Label $(VALB)"]})
	info(Q:group, {
		"$(N)TblOrder:$(VALA)-$(VALB)": {
			+id: "epics:nt/NTTable:1.0",
			labels: {+type: "plain", +channel: "VAL"},
		}
	})
}

record(aai, "$(N)DataFirst$(VALA)"){
	field(NELM, 1)
	field(INP, {const:[0]})
	info(Q:group, {
		"$(N)TblOrder:$(VALA)-$(VALB)": {
			"value.$(VALA)": {+type: "plain", +channel: "VAL"}
		}
	})
}

record(aai, "$(N)DataSecond$(VALB)"){
	field(NELM, 1)
	field(INP, {const:[1]})
	info(Q:group, {
		"$(N)TblOrder:$(VALA)-$(VALB)": {
			"":{+type:"meta", +channel:"VAL"},
			"value.$(VALB)": {+type: "plain", +channel: "VAL"}
		}
	})
}

Macros VALA=A,VALB=B (definition order matches alphabetic)

~ ➜ pvget tbl:TblOrder:A-B
tbl:TblOrder:A-B 2025-01-07 18:13:58.625
"Label A" "Label B"
        0         1
~ ➜ pvinfo tbl:TblOrder:A-B
tbl:TblOrder:A-B
Server: 10.15.1.133:38603
Type:
    epics:nt/NTTable:1.0
        structure record
            structure _options
                int queueSize
                boolean atomic
        alarm_t alarm
            int severity
            int status
            string message
        time_t timeStamp
            long secondsPastEpoch
            int nanoseconds
            int userTag
        string[] labels
        structure value
            int[] A
            int[] B

Macros VALA=B,VALB=A (definition order different from alphabetic)

~ ➜ pvget tbl:TblOrder:B-A
tbl:TblOrder:B-A 2025-01-07 18:16:17.592
"Label B" "Label A"
        1         0
~ ➜ pvinfo tbl:TblOrder:B-A
tbl:TblOrder:B-A
Server: 10.15.1.133:45647
Type:
    epics:nt/NTTable:1.0
        structure record
            structure _options
                int queueSize
                boolean atomic
        alarm_t alarm
            int severity
            int status
            string message
        time_t timeStamp
            long secondsPastEpoch
            int nanoseconds
            int userTag
        string[] labels
        structure value
            int[] A
            int[] B

The behavior is the same if you switch A for "event" and B for "timestamp", which were the values I had trouble with when opening this issue.

As you can see, in the first case, everything is coherent: the first column is "value.A", it's shown that way in pvget (value 0) and in pvinfo.

In the second case, that's no longer true: the first column is still "value.A", even though it was defined afterwards. pvget shows that by having value 1 in the first column, and pvinfo has that order in value.

EDIT: I reread the commit adding more testing and managed to write out my own in #95

[ericonr]

Aside: I realized I was required to have the data records NELM>1 for pvget to show the table with table formatting, because otherwise pvinfo showed value.A and value.B as int instead of int[], which I assume doesn't match the normative type fully.

Is this a limitation related to #69 ? Or a separate bug?