-
Notifications
You must be signed in to change notification settings - Fork 2
/
Copy pathdraft-ietf-opsawg-ipfix-on-path-telemetry-08.txt
1624 lines (1025 loc) · 60.4 KB
/
draft-ietf-opsawg-ipfix-on-path-telemetry-08.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Network Working Group T. Graf
Internet-Draft Swisscom
Intended status: Standards Track B. Claise
Expires: 9 January 2025 Huawei
A. Huang Feng
INSA-Lyon
8 July 2024
Export of On-Path Delay in IPFIX
draft-ietf-opsawg-ipfix-on-path-telemetry-08
Abstract
This document introduces new IP Flow Information Export (IPFIX)
information elements to expose the On-Path Telemetry measured delay
on the IOAM transit and decapsulation nodes.
Status of This Memo
This Internet-Draft is submitted in full conformance with the
provisions of BCP 78 and BCP 79.
Internet-Drafts are working documents of the Internet Engineering
Task Force (IETF). Note that other groups may also distribute
working documents as Internet-Drafts. The list of current Internet-
Drafts is at https://datatracker.ietf.org/drafts/current/.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."
This Internet-Draft will expire on 9 January 2025.
Copyright Notice
Copyright (c) 2024 IETF Trust and the persons identified as the
document authors. All rights reserved.
This document is subject to BCP 78 and the IETF Trust's Legal
Provisions Relating to IETF Documents (https://trustee.ietf.org/
license-info) in effect on the date of publication of this document.
Please review these documents carefully, as they describe your rights
and restrictions with respect to this document. Code Components
extracted from this document must include Revised BSD License text as
described in Section 4.e of the Trust Legal Provisions and are
provided without warranty as described in the Revised BSD License.
Graf, et al. Expires 9 January 2025 [Page 1]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . 5
3. Performance Metrics . . . . . . . . . . . . . . . . . . . . . 6
3.1. IP One-Way Delay Hybrid Type I Passive Performance
Metrics . . . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.1. Summary . . . . . . . . . . . . . . . . . . . . . . . 6
3.1.2. Description . . . . . . . . . . . . . . . . . . . . . 7
3.1.3. Change Controller . . . . . . . . . . . . . . . . . . 7
3.1.4. Version of Registry Format . . . . . . . . . . . . . 7
3.2. Metric Definition . . . . . . . . . . . . . . . . . . . . 8
3.2.1. Reference Definition . . . . . . . . . . . . . . . . 8
3.2.2. Fixed Parameters . . . . . . . . . . . . . . . . . . 8
3.3. Method of Measurement . . . . . . . . . . . . . . . . . . 8
3.3.1. Reference Methods . . . . . . . . . . . . . . . . . . 8
3.3.2. Packet Stream Generation . . . . . . . . . . . . . . 9
3.3.3. Traffic Filtering (Observation) Details . . . . . . . 9
3.3.4. Sampling Distribution . . . . . . . . . . . . . . . . 9
3.3.5. Runtime Parameters and Data Format . . . . . . . . . 9
3.3.6. Roles . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4. Output . . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.1. Type . . . . . . . . . . . . . . . . . . . . . . . . 10
3.4.2. Reference Definition . . . . . . . . . . . . . . . . 10
3.4.3. Administrative Items . . . . . . . . . . . . . . . . 13
3.4.4. Comments and Remarks . . . . . . . . . . . . . . . . 13
4. IPFIX Information Elements . . . . . . . . . . . . . . . . . 13
5. Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . 14
6. IANA Considerations . . . . . . . . . . . . . . . . . . . . . 15
6.1. Performance Metrics . . . . . . . . . . . . . . . . . . . 15
6.2. IPFIX Entities . . . . . . . . . . . . . . . . . . . . . 15
6.2.1. PathDelayMeanDeltaMicroseconds . . . . . . . . . . . 16
6.2.2. PathDelayMinDeltaMicroseconds . . . . . . . . . . . . 17
6.2.3. PathDelayMaxDeltaMicroseconds . . . . . . . . . . . . 17
6.2.4. PathDelaySumDeltaMicroseconds . . . . . . . . . . . . 17
7. Operational Considerations . . . . . . . . . . . . . . . . . 18
7.1. Time Accuracy . . . . . . . . . . . . . . . . . . . . . . 18
7.2. Mean Delay . . . . . . . . . . . . . . . . . . . . . . . 18
7.3. Reduced-size encoding . . . . . . . . . . . . . . . . . . 18
7.4. IOAM Application . . . . . . . . . . . . . . . . . . . . 18
8. Security Considerations . . . . . . . . . . . . . . . . . . . 19
9. Implementation Status . . . . . . . . . . . . . . . . . . . . 19
9.1. FD.io VPP . . . . . . . . . . . . . . . . . . . . . . . . 19
9.2. Huawei VRP . . . . . . . . . . . . . . . . . . . . . . . 19
9.3. Fluvia . . . . . . . . . . . . . . . . . . . . . . . . . 20
9.4. Pmacct Data Collection . . . . . . . . . . . . . . . . . 20
10. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . 20
11. References . . . . . . . . . . . . . . . . . . . . . . . . . 20
Graf, et al. Expires 9 January 2025 [Page 2]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
11.1. Normative References . . . . . . . . . . . . . . . . . . 20
11.2. Informative References . . . . . . . . . . . . . . . . . 20
Appendix A. IPFIX Encoding Examples . . . . . . . . . . . . . . 24
A.1. Aggregated On-Path Dealay Examples . . . . . . . . . . . 24
A.1.1. Template Record and Data Set with Mean Delta . . . . 24
A.1.2. Template Record and Data Set with Sum Delta . . . . . 26
Authors' Addresses . . . . . . . . . . . . . . . . . . . . . . . 28
1. Introduction
Network operators want a statistical delay view of their networks.
They want to understand where in the network, for which customer
traffic, how much and why delay is being accummlated. In order to
answer why and where, delay needs to be reported into device and
control-plane context. In order to understand which customer traffic
is affected, delay needs to be reported into customer data-plane
context. That enables network operators to quickly identify when the
control-plane updates the current path with a different next-hop and
therefore the forwarding path changes to different nodes and
interfaces, how the path delay changes for which customer traffic.
With On-Path Telemetry, described in the Network Telemetry Framework
[RFC9232] and applied in In-situ OAM [I-D.ietf-ippm-ioam-deployment]
and Alternate Marking Deployment Framework
[I-D.ietf-ippm-alt-mark-deployment], the path delay between two
endpoints is measured by inserting a timestamp in the packet.
On-Path Telemetry can be distinguished between two modes. Passport
mode, [RFC9197], where only the last hop in the forwarding path of
the On-Path Telemetry domain exposes all the metrics, and postcard
mode, [I-D.song-ippm-postcard-based-telemetry], where the metrics are
also exposed in the transit nodes. In both modes the forwarding path
exposes performance metrics allowing to determine how much delay has
been accumulated on which hop.
This document defines four new IPFIX Information Elements (IEs),
exposing the On-Path delay on IOAM transit and decapsulation nodes,
following the postcard mode principles. Since these IPFIX IEs are
performance metrics [RFC8911], they must be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
Following the guidelines for Registered Performance Metric requesters
and reviewers [RFC8911], the different characteristics of the
performance metrics (Identifier, Name, URI, Status, Requester,
Revision, Revision Date, Description, etc) must be clearly specified
in the "IANA Performance Metric Registry [IANA-PERF-METRIC] in order
for the results of measurements using the Performance Metrics to be
comparable even if they are performed by different implementations
Graf, et al. Expires 9 January 2025 [Page 3]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
and in different networks. These characteristics start by selecting
a meaningful name, following the "MetricType_Method_SubTypeMethod_...
Spec_Units_Output" naming convention (See Section 7.1.2 of
[RFC8911]).
+------------------------------------+-------------------------------+
| Performance Metric | IPFIX Information Element |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMeanDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Mean (TBD1)|(TBD5) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMinDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Min (TBD2) |(TBD6) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelayMaxDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Max (TBD3) |(TBD7) |
+------------------------------------+-------------------------------+
|OWDelay_HybridType1_Passive_I |PathDelaySumDeltaMicroseconds |
|P_RFC[RFC-to-be]_Seconds_Sum (TBD4) |(TBD8) |
+------------------------------------+-------------------------------+
Table 1: Correspondance between IPFIX IE and Performance Metric
The delay is measured by calculating the difference between the
timestamp imposed with On-Path Telemetry in the packet at the IOAM
encapsulation node and the timestamp exported in the IPFIX flow
record from the IOAM transit and decapsulation nodes. The lowest,
highest, mean, and/or the sum of measured path delay can be exported,
thanks to the different IPFIX IE specifications.
On-Path Telemetry Domain
.........................................
. .
. D1 .
. <------> .
. .
. D2 .
. <--------------------> .
. .
. D3 .
. <-----------------------------------> .
. .
(H1) ------ (R1) ------- (R2) ------- (R3) -------- (R4) ------ (H2)
Host 1 Encapsulation Transit Transit Decapsulation Host 2
Node Node 1 Node 2 Node
. .
. .
.........................................
Graf, et al. Expires 9 January 2025 [Page 4]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
Figure 1: Delay use case. Packets flow from host 1 to host 2.
On the usecase showed in Figure 1 using On-path Telemetry to export
the delay metrics, the node R2 exports the delay D1, the node R3
exports the delay D2 and the decapsulation node R4 exports the total
delay D3 using IPFIX.
2. Terminology
The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and
"OPTIONAL" in this document are to be interpreted as described in BCP
14 [RFC2119] [RFC8174] when, and only when, they appear in all
capitals, as shown here.
This document makes use of the terms defined in [RFC7011] and
[I-D.ietf-ippm-ioam-deployment].
The following terms are used as defined in [RFC7011].
* IPFIX
* IPFIX Information Elements (IEs)
* Flow
* Flow Record
* Exporter
The following terms are used as defined in [RFC8911].
* Performance Metric
* Registered Performance Metric
* Performance Metrics Registry
The following terms are used as defined in
[I-D.ietf-ippm-ioam-deployment].
* IOAM encapsulation node
* IOAM transit node
* IOAM decapsulation node
Graf, et al. Expires 9 January 2025 [Page 5]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
3. Performance Metrics
This section defines and describes the new performance metrics by
applying the template defined in Section 11 of [RFC8911].
IANA Note (to be removed): RFC 8192 section 4 was taken a guiding
example.
3.1. IP One-Way Delay Hybrid Type I Passive Performance Metrics
This section specifies four performance metrics for the Hybrid Type I
Passive assessment of IP One-Way Delay, to be registered in the "IANA
Performance Metric Registry [IANA-PERF-METRIC].
All column entries besides the ID, Name, Description, and Output
Reference Method categories are the same; thus, this section defines
four closely related performance metrics. As a result, IANA has
assigned corresponding URLs to each of the four registered
performance metrics.
3.1.1. Summary
This category includes multiple indexes of the registered performance
metrics: the element ID and Metric Name.
3.1.1.1. ID (Identifier)
IANA has allocated the numeric Identifiers TBD1, TBD2, TBD3, and TBD4
for the four Named Metric Entries in the following section.
RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4.
3.1.1.2. Name
TBD1: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
TBD2: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
TBD3: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
TBD4: OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace [RFC-to-be].
3.1.1.3. URI
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
Graf, et al. Expires 9 January 2025 [Page 6]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
URL: https://www.iana.org/assignments/performance-metrics/
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
RFC EDITOR NOTE: please replace TBD1, TBD2, TBD3, and TBD4.
3.1.2. Description
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean: This
metric assesses the mean of one-way delays of all successfully
forwarded IP packets constituting a single Flow between two hosts.
We consider the measurement of one-way delay based on a single
Observation Point (OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min: This
metric assesses the minimum of one-way delays of all successfully
forwarded IP packets constituting a single Flow between two hosts.
We consider the measurement of one-way delay based on a single
Observation Point (OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max: This
metric assesses the maximum of one-way delays of all successfully
forwarded IP packets constituting a single Flow between two hosts.
We consider the measurement of one-way delay based on a single
Observation Point (OP) [RFC7011] somewhere in the network.
* OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum: This
metric assesses the sum of one-way delays of all successfully
forwarded IP packets constituting a single Flow between two hosts.
We consider the measurement of one-way delay based on a single
Observation Point (OP) [RFC7011] somewhere in the network.
3.1.3. Change Controller
IETF
3.1.4. Version of Registry Format
1.0
Graf, et al. Expires 9 January 2025 [Page 7]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
3.2. Metric Definition
This category includes columns to prompt the entry of all necessary
details related to the metric definition, including the immutable
document reference and values of input factors, called "Fixed
Parameters".
3.2.1. Reference Definition
Almes, G., Kalidindi, S., Zekauskas, M., and A. Morton, Ed., "A One-
Way Delay Metric for IP Performance Metrics (IPPM)", STD 81, RFC
7679, DOI 10.17487/RFC7679, January 2016, <https://www.rfc-
editor.org/info/rfc7679>. [RFC7679]
Morton, A. and E. Stephan, "Spatial Composition of Metrics", RFC
6049, DOI 10.17487/RFC6049, January 2011, <https://www.rfc-
editor.org/info/rfc6049>. [RFC6049]
Section 3.4 of [RFC7679] provides the reference definition of the
singleton (single value) one-way delay metric. Section 4.4 of
[RFC7679] provides the reference definition expanded to cover a
multi-value sample. Note that terms such as "singleton" and "sample"
are defined in section 2 of [RFC2330].
With the OP [RFC7011] typically located between the hosts
participating in the IP Flow, the one-way delay metric requires one
individual measurement between the OP and sourcing host, such that
the Spatial Composition [RFC6049] of the measurements yields a one-
way delay singleton.
This document specifies how to export the performance metric with
IPFIX.
3.2.2. Fixed Parameters
None
3.3. Method of Measurement
This category includes columns for references to relevant sections of
the RFC(s) and any supplemental information needed to ensure an
unambiguous method for implementations.
3.3.1. Reference Methods
The foundational methodology for this metric is defined in section 4
of [RFC7323] using the Timestamps option with modifications that
allow application at a mid-path OP [RFC7011].
Graf, et al. Expires 9 January 2025 [Page 8]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
3.3.2. Packet Stream Generation
The timestamp when the packet is being received at IOAM encapsulation
node. Format depends on On-Path Telemetry implementation. For IOAM,
Section 4.4.1 of [RFC9197] describes what kind of timestamps are
supported. Section 4.4.2.3 and 4.4.2.4 describe where the timestamp
is being inserted. For the Enhanced Alternate Marking Method,
Section 2 of [I-D.zhou-ippm-enhanced-alternate-marking] describes
timestamp encoding and granularity.
3.3.3. Traffic Filtering (Observation) Details
Runtime Parameters (below) may be used for Traffic Filtering.
3.3.4. Sampling Distribution
This metric requires a partial sample of all packets that qualify
according to the Traffic Filter criteria.
3.3.5. Runtime Parameters and Data Format
Runtime Parameters are input factors that must be determined,
configured into the measurement system, and reported with the results
for the context to be complete.
The hybrid type I metering parameters must be reported to provide the
complete measurement context. As an example, if the IPFIX Metering
Process is used, then the IPFIX Metering Process parameters (IPFIX
Template Record, potential traffic filters, and potential sampling
method and parameters) that generate the Flow Records must be
reported to provide the complete measurement context. At a minimum,
the following fields are required:
Src: The IP address of the host in the host A Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see section 4 of [RFC6991].
Dst: The IP address of the host in the host B Role (format
ipv4-address-no-zone value for IPv4 or ipv6-address-no-zone value
for IPv6; see section 4 of [RFC6991].
T0: T time, the start of a measurement interval (format "date/time"
as specified in Section 5.6 of [RFC3339]; see also "date-and-time"
in Section 3 of [RFC6991]). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", a start time is
unspecified and Tf is to be interpreted as the duration of the
measurement interval. The start time is controlled through other
means.
Graf, et al. Expires 9 January 2025 [Page 9]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
Tf: A time, the end of a measurement interval (format "date/time" as
specified in Section 5.6 of [RFC3339]; see also "date-and-time" in
Section 3 of [RFC6991])). The UTC Time Zone is required by
Section 6.1 of [RFC2330]. When T0 is "all-zeros", an ending time
and date is ignored and Tf is interpreted as the duration of the
measurement interval.
3.3.6. Roles
host A: Launches the IP packet to start the Flow. The Role of "host
A" is synonymous with the IP address used at host A.
host B: Receives the IP packet to start the Flow. The Role of "host
B" is synonymous with the IP address used at host B.
Encapsulation Node: Receives the IP Flow packets and encapsulates
the timestamp into the packet. The Role of "Encapsulation Node"
is synonymous with the timestamp inserted in the packet.
Transit Node: Receives the IP Flow packets and measures the delay
between the timestamp in the packet and the timestamp when the
packet was received.
Decapsulation Node: Receives the IP Flow packets and measures the
delay between the timestamp in the packet and the timestamp when
the packet was received and removes the IOAM header from the
packet.
3.4. Output
This category specifies all details of the output of measurements
using the metric.
3.4.1. Type
OWDelay Types are discussed in the subsections below.
3.4.2. Reference Definition
For all output types:
OWDelay_HybridType1_Passive_IP: The one-trip delay of one IP packet
is a Singleton
For each <statistic> Singleton one of the following subsections
applies.
Graf, et al. Expires 9 January 2025 [Page 10]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
3.4.2.1. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean
The mean SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.2.2 of [RFC6049] for details on calculating this
statistic; see also section 4.2.3 of [RFC6049].
Mean: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.2. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Min
The minimum SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.3.2 of [RFC6049] for details on calculating this
statistic; see also section 4.3.3 of [RFC6049].
Min: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.3. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Max
The maximum SHALL be calculated using the conditional distribution of
all packets with a finite value of one-way delay (undefined delays
are excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
Graf, et al. Expires 9 January 2025 [Page 11]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
See section 4.3.2 of [RFC6049] for a closely related method for
calculating this statistic; see also section 4.3.3 of [RFC6049]. The
formula is as follows:
Max = (FiniteDelay[j])
such that for some index, j, where 1 <= j <= N
FiniteDelay[j] >= FiniteDelay[n] for all n
where all packets n = 1 through N have finite singleton delays.
Max: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.4. OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Sum
The sum SHALL be calculated using the conditional distribution of all
packets with a finite value of one-way delay (undefined delays are
excluded) -- a single value, as follows:
See section 4.1 of [RFC3393] for details on the conditional
distribution to exclude undefined values of delay, and see section 5
of [RFC6703] for background on this analysis choice.
See section 4.3.5 of [RFC6049] for details on calculating this
statistic. However in this case FiniteDelay or MaxDelay MAY be used.
Sum: The time value of the result is expressed in units of seconds,
as a positive value of type decimal64 with fraction digits = 9
(see section 9.3 of [RFC6020]) with a resolution of
0.000000001 seconds (1.0 ns), and with lossless conversion to/from
the 64-bit NTP timestamp as per section 6 of [RFC5905].
3.4.2.5. Metric Units
* Mean
* Min
* Max
* Sum
The one-way delay of the IP Flow singleton is expressed in seconds.
Graf, et al. Expires 9 January 2025 [Page 12]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
3.4.2.6. Calibration
Passive Measurements at an OP could be calibrated against an Active
Measurement at host A where the Active Measurement represents the
ground truth.
3.4.3. Administrative Items
3.4.3.1. Status
Current
3.4.3.2. Requester
This RFC
RFC EDITOR NOTE: please replace This RFC text by the RFC issued from
this document
3.4.3.3. Revision
1.0
3.4.3.4. Revision Date
RFC Date
3.4.4. Comments and Remarks
none
4. IPFIX Information Elements
This section defines and describes the new IPFIX IEs.
PathDelayMeanDeltaMicroseconds
32-bit unsigned integer that identifies the mean path delay of all
packets in the Flow, in microseconds, between the IOAM
encapsulation node and the local node with the IOAM domain (either
an IOAM transit node or an IOAM decapsulation node).
PathDelayMinDeltaMicroseconds
32-bit unsigned integer that identifies the lowest path delay of
all packets in the Flow, in microseconds, between the IOAM
encapsulation node and the local node with the IOAM domain (either
an IOAM transit node or an IOAM decapsulation node).
Graf, et al. Expires 9 January 2025 [Page 13]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
PathDelayMaxDeltaMicroseconds
32-bit unsigned integer that identifies the highest path delay of
all packets in the Flow, in microseconds, between the IOAM
encapsulation node and the local node with the IOAM domain (either
an IOAM transit node or an IOAM decapsulation node).
PathDelaySumDeltaMicroseconds
64-bit unsigned integer that identifies the sum of the path delay
of all packets in the Flow, in microseconds, between the IOAM
encapsulation node and the local node with the IOAM domain (either
an IOAM transit node or an IOAM decapsulation node).
5. Use Cases
The measured On-Path delay can be aggregated with Flow Aggregation as
defined in [RFC7015] to the following device and control-plane
dimensions to determine:
* With node id and egressInterface(IE14), on which node which
logical egress interfaces have been contributing to how much
delay.
* With node id and egressPhysicalInterface(253), on which node which
physical egress interfaces have been contributing to how much
delay.
* With ipNextHopIPv4Address(IE15) or ipNextHopIPv6Address(IE62), the
forwarding path to which next-hop IP contributed to how much
delay.
* With mplsTopLabelIPv4Address(IE47) or destinationIPv6Address and
srhActiveSegmentIPv6 from [RFC9487], the forwarding path to which
MPLS top label IPv4 address or IPv6 destination address and SRv6
active segment contributed to how much delay.
* BGP communities are often used for setting a path priority or
service selection. With bgpDestinationExtendedCommunityList(488)
or bgpDestinationCommunityList(485) or
bgpDestinationLargeCommunityList(491) which group of prefixes
accumulated at which node how much delay.
* With destinationIPv4Address(13), destinationTransportPort(11),
protocolIdentifier (4) and sourceIPv4Address(IE8), the forwarding
path delay on each node from each IPv4 source address to a
specific application in the network.
Graf, et al. Expires 9 January 2025 [Page 14]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
Taking figure 1 from section 1 as topology example. Below example
table shows the aggregated delay per each node, ingressInterface,
egressInterface, destinationIPv6Address and srhActiveSegmentIPv6.
+-----------+-----------+------+-------------+-------------+------------+
| ingress | egress | Node | destination | srhActive | Path Delay |
| Interface | Interface | | IPv6Address | SegmentIPv6 | |
+-----------+-----------+------+-------------+-------------+------------+
| 271 | 276 | R1 | 2001:db8::2 | 2001:db8::4 | 0 us |
+-----------+-----------+------+-------------+-------------+------------+
| 301 | 312 | R2 | 2001:db8::3 | 2001:db8::4 | 22 us |
+-----------+-----------+------+-------------+-------------+------------+
| 22 | 27 | R3 | 2001:db8::4 | 2001:db8::4 | 42 us |
+-----------+-----------+------+-------------+-------------+------------+
| 852 | 854 | R4 | 2001:db8::4 | 2001:db8::4 | 122 us |
+-----------+-----------+------+-------------+-------------+------------+
Table 2: Example table of measured delay. Ascending by delay.
6. IANA Considerations
6.1. Performance Metrics
This document requests IANA to create four new performance metrics
under the "Performance Metrics" registry [RFC8911] with the four
templates defined in section 3.
6.2. IPFIX Entities
This document requests IANA to create new IPFIX IEs (see table 3)
under the "IPFIX Information Elements" registry [RFC7012] available
at "IANA Performance Metric Registry [IANA-PERF-METRIC] and assign
the following initial code points.
Graf, et al. Expires 9 January 2025 [Page 15]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
+-------+--------------------------------+
|Element| Name |
| ID | |
+-------+--------------------------------+
| TBD5 | PathDelayMeanDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD6 | PathDelayMinDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD7 | PathDelayMaxDeltaMicroseconds |
| | |
+-------+--------------------------------+
| TBD8 | PathDelaySumDeltaMicroseconds |
| | |
+-------+--------------------------------+
Table 3: Creates IPFIX IEs in the "IPFIX Information Elements" registry
Note to the RFC-Editor:
* Please replace TBD5 - TBD8 with the values allocated by IANA
* Please replace all instances of [RFC-to-be] in this section with
the RFC number assigned to this document
6.2.1. PathDelayMeanDeltaMicroseconds
Name: PathDelayMeanDeltaMicroseconds
ElementID: TBD5
Description: This Information Element identifies the mean path delay
of all packets in the Flow, in microseconds, between the IOAM
encapsulation node and the local node with the IOAM domain (either
an IOAM transit node or an IOAM decapsulation node), according to
OWDelay_HybridType1_Passive_IP_RFC[RFC-to-be]_Seconds_Mean in the
IANA Performance Metric Registry
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Mean in the IANA Performance Metric Registry.
Graf, et al. Expires 9 January 2025 [Page 16]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
6.2.2. PathDelayMinDeltaMicroseconds
Name: PathDelayMinDeltaMicroseconds
ElementID: TBD6
Description: This Information Element identifies the lowest path
delay of all packets in the Flow, in microseconds, between the
IOAM encapsulation node and the local node with the IOAM domain
(either an IOAM transit node or an IOAM decapsulation node),
according to the OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Min in the IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Min in the IANA Performance Metric Registry.
6.2.3. PathDelayMaxDeltaMicroseconds
Name: PathDelayMaxDeltaMicroseconds
ElementID: TBD7
Description: This Information Element identifies the highest path
delay of all packets in the Flow, in microseconds, between the
IOAM encapsulation node and the local node with the IOAM domain
(either an IOAM transit node or an IOAM decapsulation node),
according to OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Max in the IANA Performance Metric Registry.
Abstract Data Type: unsigned32
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Max in the IANA Performance Metric Registry.
6.2.4. PathDelaySumDeltaMicroseconds
Name: PathDelaySumDeltaMicroseconds
ElementID: TBD8
Description: This Information Element identifies the sum of the path
Graf, et al. Expires 9 January 2025 [Page 17]
Internet-Draft Export of On-Path Delay in IPFIX July 2024
delay of all packets in the Flow, in microseconds, between the
IOAM encapsulation node and the local node with the IOAM domain
(either an IOAM transit node or an IOAM decapsulation node),
according to OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Sum in the IANA Performance Metric Registry.
Abstract Data Type: unsigned64
Data Type Semantics: deltaCounter
Reference: [RFC-to-be], OWDelay_HybridType1_Passive_IP_RFC[RFC-to-
be]_Seconds_Sum in the IANA Performance Metric Registry.
7. Operational Considerations
7.1. Time Accuracy
The same recommendation as defined in section 4.5 of [RFC5153] for
IPFIX applies in terms of clock precision to this document as well.
7.2. Mean Delay
The mean (average) path delay can be calculated by dividing the
PathDelaySumDeltaMicroseconds(TBD8) by the packetDeltaCount(2) at the
IPFIX data collection in order to offload the IPFIX Exporter from
calculating the mean for every Flow at export time.
7.3. Reduced-size encoding
Unsigned64 has been chosen as type for PathDelaySumDeltaMicroseconds
to support cases with large delay numbers and where many packets are
being accounted. As an example, a specific flow record with path
delay of 100 milliseconds can not observe more than 42949 packets
without overflowing the unsigned32 counter. The procedure described
in Section 6.2 of [RFC7011] could be applied to reduce network
bandwidth between the IPFIX Exporter and Collector if unsigned32
would be large enough without wrapping around.
7.4. IOAM Application
This document is applicable in IOAM to the Edge-to-Edge and Direct
Exporting Option-Type.
In case of Edge-to-Edge Option-Type, as described in Section 4.6 of