Merge pull request #104640 from openshift-cherrypick-robot/cherry-pick-104418-to-enterprise-4.21

[enterprise-4.21] TELCODOCS-2495: Moving holdover feature to GA and improving docs

modules/nw-ptp-t-bc-t-tsc-holdover.adoc

@@ -4,24 +4,28 @@
 
 :_mod-docs-content-type: PROCEDURE
 [id="nw-ptp-t-bc-t-tsc-holdover_{context}"]
-= Applying unassisted holdover for boundary clocks and time slave clocks
+= Applying unassisted holdover for boundary clocks and time synchronous clocks
 
-The unassisted holdover feature enables an Intel E810-XXVDA4T Network Interface Card (NIC), configured as either a PTP boundary clock (T-BC) or a PTP time slave clock (T-TSC), to maintain highly accurate time synchronization even when the upstream timing signal is lost. This is achieved by relying on the NIC's internal oscillator to enter a stable, controlled drift state.
+The unassisted holdover feature enables an Intel E810-XXVDA4T NIC, configured as a PTP boundary clock (T-BC) or telecom time synchronous clock (T-TSC), to maintain time synchronization when the upstream timing source becomes unavailable.
 
-The `ts2phc` service monitors the `ptp4l` instance bound to the timing receiver (TR) port. If, for example, the TR port stops operating as the time receiver, the upstream grandmaster clock (T-GM) deteriorates in quality or the link disconnects, the system enters holdover mode and reconfigures itself dynamically.
+When the upstream source degrades, disconnects, or fails, the NIC enters the holdover state. In this state, the NIC relies on its internal oscillator to maintain accurate time autonomously. On nodes with multiple NICs, one NIC acts as the leading NIC. Other NICs on the node synchronize to the leading NIC through SMA cable connections managed by the `ts2phc` service.
 
-:FeatureName: Applying unassisted holdover for T-BC and T-TSC
-include::snippets/technology-preview.adoc[leveloffset=+1]
+In T-BC configurations, the time transmitter (TT) ports continue transmitting timing to downstream devices, signaling a degraded but usable clock quality. Timing remains stable on both the local node and the downstream network until the upstream source recovers or the holdover timeout expires.
+
+This example describes how to configure unassisted holdover for a multi-card T-BC configuration with three NICs.
 
 .Prerequisites
 * Install the OpenShift CLI (`oc`).
 * Log in as a user with `cluster-admin` privileges.
 * Install the PTP Operator.
-* An Intel E810-XXVDA4T NIC.
+* One or more Intel E810-XXVDA4T NICs.
+* For multi-card configurations, SMA cables connecting the NICs.
 
 .Procedure
 
-. Configure the triple port T-BC NIC. See the example below where the `PtpConfig` resource contains two profiles, one for time transmitter ports (00-tbc-tt) and one to configure all the hardware, the TR port, and `ts2phc` and `phc2sys` processes:
+. Create a `PtpConfig` CR with two profiles: one for time transmitter (TT) ports and one for the time receiver (TR) port with hardware configuration.
+
+.. Define the TT profile that configures downstream-facing ports:
 +
 [source,yaml]
 ----
@@ -35,11 +39,366 @@ spec:
   - name: 00-tbc-tt
     ptp4lConf: |
       [ens4f0]
-      masterOnly 1 <1>
+      masterOnly 1
       [ens8f0]
-      masterOnly 1 <1>
+      masterOnly 1
       [ens1f0]
-      masterOnly 1 <1>
+      masterOnly 1
+      [global]
+      #
+      # Default Data Set
+      #
+      twoStepFlag 1
+      slaveOnly 0
+      priority1 128
+      priority2 128
+      domainNumber 25
+      clockClass 248
+      clockAccuracy 0xFE
+      offsetScaledLogVariance 0xFFFF
+      free_running 0
+      freq_est_interval 1
+      dscp_event 0
+      dscp_general 0
+      dataset_comparison G.8275.x
+      G.8275.defaultDS.localPriority 128
+      #
+      # Port Data Set
+      #
+      logAnnounceInterval -3
+      logSyncInterval -4
+      logMinDelayReqInterval -4
+      logMinPdelayReqInterval -4
+      announceReceiptTimeout 3
+      syncReceiptTimeout 0
+      delayAsymmetry 0
+      fault_reset_interval -4
+      neighborPropDelayThresh 20000000
+      masterOnly 0
+      G.8275.portDS.localPriority 128
+      #
+      # Run time options
+      #
+      assume_two_step 0
+      logging_level 6
+      path_trace_enabled 0
+      follow_up_info 0
+      hybrid_e2e 0
+      inhibit_multicast_service 0
+      net_sync_monitor 0
+      tc_spanning_tree 0
+      tx_timestamp_timeout 50
+      unicast_listen 0
+      unicast_master_table 0
+      unicast_req_duration 3600
+      use_syslog 1
+      verbose 0
+      summary_interval 0
+      kernel_leap 1
+      check_fup_sync 0
+      clock_class_threshold 135
+      #
+      # Servo Options
+      #
+      pi_proportional_const 0.60
+      pi_integral_const 0.001
+      pi_proportional_scale 0.0
+      pi_proportional_exponent -0.3
+      pi_proportional_norm_max 0.7
+      pi_integral_scale 0.0
+      pi_integral_exponent 0.4
+      pi_integral_norm_max 0.3
+      step_threshold 2.0
+      first_step_threshold 0.00002
+      max_frequency 900000000
+      clock_servo pi
+      sanity_freq_limit 200000000
+      ntpshm_segment 0
+      #
+      # Transport options
+      #
+      transportSpecific 0x0
+      ptp_dst_mac AA:BB:CC:DD:EE:FF
+      p2p_dst_mac BB:CC:DD:EE:FF:GG
+      udp_ttl 1
+      udp6_scope 0x0E
+      uds_address /var/run/ptp4l
+      #
+      # Default interface options
+      #
+      clock_type BC
+      network_transport L2
+      delay_mechanism E2E
+      time_stamping hardware
+      tsproc_mode filter
+      delay_filter moving_median
+      delay_filter_length 10
+      egressLatency 0
+      ingressLatency 0
+      boundary_clock_jbod 1
+      #
+      # Clock description
+      #
+      productDescription ;;
+      revisionData ;;
+      manufacturerIdentity 00:00:00
+      userDescription ;
+      timeSource 0xA0
+    ptp4lOpts: -2 --summary_interval -4
+    ptpSchedulingPolicy: SCHED_FIFO
+    ptpSchedulingPriority: 10
+    ptpSettings:
+      controllingProfile: 01-tbc-tr
+      logReduce: "false"
+# ...
+----
+* `spec.profile[].ptp4lConf` defines the `ptp4l` configuration. In the `[ens4f0]` interface section, the `masterOnly 1` setting configures TT ports to transmit timing downstream.
+* `clock_type BC` sets the clock to boundary clock operation.
+* `boundary_clock_jbod 1` enables multi-NIC configurations.
+* `spec.profile[].ptpSettings.controllingProfile` references the TR profile that controls holdover behavior.
+
+.. Define the TR profile with the hardware plugin configuration:
++
+[source,yaml]
+----
+# ...
+  - name: 01-tbc-tr
+    phc2sysOpts: -r -n 25 -N 8 -R 16 -u 0 -m -s ens4f1
+    plugins:
+      e810:
+        enableDefaultConfig: false
+        interconnections:
+        - gnssInput: false
+          id: ens4f0
+          part: E810-XXVDA4T
+          phaseOutputConnectors:
+          - SMA1
+          - SMA2
+          upstreamPort: ens4f1
+        - id: ens1f0
+          inputConnector:
+            connector: SMA1
+          part: E810-XXVDA4T
+        - id: ens8f0
+          inputConnector:
+            connector: SMA1
+          part: E810-XXVDA4T
+        pins:
+          ens4f0:
+            SMA1: 2 1
+            SMA2: 2 2
+            U.FL1: 0 1
+            U.FL2: 0 2
+          ens1f0:
+            SMA1: 1 1
+            SMA2: 0 2
+            U.FL1: 0 1
+            U.FL2: 0 2
+          ens8f0:
+            SMA1: 1 1
+            SMA2: 0 2
+            U.FL1: 0 1
+            U.FL2: 0 2
+        settings:
+          LocalHoldoverTimeout: 14400
+          LocalMaxHoldoverOffSet: 1500
+          MaxInSpecOffset: 100
+# ...
+----
+* `phc2sysOpts` specifies the upstream port (`ens4f1`) as the source for system clock synchronization.
+* `plugins.e810.interconnections` defines how NICs are connected. The leading NIC (`ens4f0`) outputs phase to other NICs through SMA cables.
+* `plugins.e810.interconnections[].phaseOutputConnectors` lists SMA connectors used for cable connections to downstream NICs.
+* `plugins.e810.interconnections[].upstreamPort` specifies the TR port that receives timing from upstream. Set this for both T-BC and T-TSC configurations.
+* `plugins.e810.pins` configures SMA and U.FL pin directions for each NIC.
+* `plugins.e810.settings` configures holdover behavior. For details about these parameters, see "Holdover in a grandmaster clock with GNSS as the source".
+
+.. Configure the `ptp4lConf` for the TR port:
++
+[source,yaml]
+----
+# ...
+    ptp4lConf: |
+      [ens4f1]
+      masterOnly 0
+      [global]
+      #
+      # Default Data Set
+      #
+      twoStepFlag 1
+      slaveOnly 0
+      priority1 128
+      priority2 128
+      domainNumber 25
+      clockClass 248
+      clockAccuracy 0xFE
+      offsetScaledLogVariance 0xFFFF
+      free_running 0
+      freq_est_interval 1
+      dscp_event 0
+      dscp_general 0
+      dataset_comparison G.8275.x
+      G.8275.defaultDS.localPriority 128
+      #
+      # Port Data Set
+      #
+      logAnnounceInterval -3
+      logSyncInterval -4
+      logMinDelayReqInterval -4
+      logMinPdelayReqInterval -4
+      announceReceiptTimeout 3
+      syncReceiptTimeout 0
+      delayAsymmetry 0
+      fault_reset_interval -4
+      neighborPropDelayThresh 20000000
+      masterOnly 0
+      G.8275.portDS.localPriority 128
+      #
+      # Run time options
+      #
+      assume_two_step 0
+      logging_level 6
+      path_trace_enabled 0
+      follow_up_info 0
+      hybrid_e2e 0
+      inhibit_multicast_service 0
+      net_sync_monitor 0
+      tc_spanning_tree 0
+      tx_timestamp_timeout 50
+      unicast_listen 0
+      unicast_master_table 0
+      unicast_req_duration 3600
+      use_syslog 1
+      verbose 0
+      summary_interval 0
+      kernel_leap 1
+      check_fup_sync 0
+      clock_class_threshold 135
+      #
+      # Servo Options
+      #
+      pi_proportional_const 0.60
+      pi_integral_const 0.001
+      pi_proportional_scale 0.0
+      pi_proportional_exponent -0.3
+      pi_proportional_norm_max 0.7
+      pi_integral_scale 0.0
+      pi_integral_exponent 0.4
+      pi_integral_norm_max 0.3
+      step_threshold 2.0
+      first_step_threshold 0.00002
+      max_frequency 900000000
+      clock_servo pi
+      sanity_freq_limit 200000000
+      ntpshm_segment 0
+      #
+      # Transport options
+      #
+      transportSpecific 0x0
+      ptp_dst_mac AA:BB:CC:DD:EE:HH
+      p2p_dst_mac BB:CC:DD:EE:FF:II
+      udp_ttl 1
+      udp6_scope 0x0E
+      uds_address /var/run/ptp4l
+      #
+      # Default interface options
+      #
+      clock_type OC
+      network_transport L2
+      delay_mechanism E2E
+      time_stamping hardware
+      tsproc_mode filter
+      delay_filter moving_median
+      delay_filter_length 10
+      egressLatency 0
+      ingressLatency 0
+      boundary_clock_jbod 1
+      #
+      # Clock description
+      #
+      productDescription ;;
+      revisionData ;;
+      manufacturerIdentity 00:00:00
+      userDescription ;
+      timeSource 0xA0
+    ptp4lOpts: -2 --summary_interval -4
+    ptpSchedulingPolicy: SCHED_FIFO
+    ptpSchedulingPriority: 10
+    ptpSettings:
+      inSyncConditionThreshold: "10"
+      inSyncConditionTimes: "12"
+      logReduce: "false"
+# ...
+----
+* `spec.profile[].ptp4lConf` defines the `ptp4l` configuration. In the `[ens4f1]` interface section, the `masterOnly 0` setting configures the TR port to receive timing from upstream.
+* `clock_type OC` sets the TR profile to ordinary clock because it handles only the upstream-facing port.
+
+.. Configure `ts2phc` for all participating NICs:
++
+[source,yaml]
+----
+# ...
+    ts2phcConf: |
+      [global]
+      use_syslog
+      verbose 1
+      logging_level 7
+      ts2phc.pulsewidth 100000000
+      leapfile  /usr/share/zoneinfo/leap-seconds.list
+      domainNumber 25
+      uds_address /var/run/ptp4l.0.socket
+      [ens4f0]
+      ts2phc.extts_polarity rising
+      ts2phc.extts_correction -10
+      ts2phc.master 0
+      [ens1f0]
+      ts2phc.extts_polarity rising
+      ts2phc.extts_correction -27
+      ts2phc.master 0
+      [ens8f0]
+      ts2phc.extts_polarity rising
+      ts2phc.extts_correction -27
+      ts2phc.master 0
+    ts2phcOpts: -s generic -a --ts2phc.rh_external_pps 1
+# ...
+----
+* The `domainNumber` must match the upstream PTP domain.
+* Interface sections like `[ens4f0]` list all NICs participating in the configuration. The `ts2phc.extts_correction` values compensate for cable and hardware delays.
+
+.. Define the `recommend` section to apply profiles to nodes:
++
+[source,yaml]
+----
+# ...
+  recommend:
+  - match:
+    - nodeLabel: node-role.kubernetes.io/master
+    priority: 4
+    profile: 00-tbc-tt
+  - match:
+    - nodeLabel: node-role.kubernetes.io/master
+    priority: 4
+    profile: 01-tbc-tr
+----
+
+. Review the full configuration:
++
+[source,yaml]
+----
+apiVersion: ptp.openshift.io/v1
+kind: PtpConfig
+metadata:
+  name: t-bc
+  namespace: openshift-ptp
+spec:
+  profile:
+  - name: 00-tbc-tt
+    ptp4lConf: |
+      [ens4f0]
+      masterOnly 1
+      [ens8f0]
+      masterOnly 1
+      [ens1f0]
+      masterOnly 1
       [global]
       #
       # Default Data Set
@@ -147,11 +506,11 @@ spec:
       controllingProfile: 01-tbc-tr
       logReduce: "false"
   - name: 01-tbc-tr
-    phc2sysOpts: -r -n 25 -N 8 -R 16 -u 0 -m -s ens4f1 <2>
-    plugins: <3>
+    phc2sysOpts: -r -n 25 -N 8 -R 16 -u 0 -m -s ens4f1
+    plugins:
       e810:
         enableDefaultConfig: false
-        interconnections: <4>
+        interconnections:
         - gnssInput: false
           id: ens4f0
           part: E810-XXVDA4T
@@ -188,7 +547,6 @@ spec:
           LocalMaxHoldoverOffSet: 1500
           MaxInSpecOffset: 100
     ptp4lConf: |
-      # The interface name is hardware-specific
       [ens4f1]
       masterOnly 0
       [global]
@@ -305,9 +663,9 @@ spec:
       logging_level 7
       ts2phc.pulsewidth 100000000
       leapfile  /usr/share/zoneinfo/leap-seconds.list
-      domainNumber 25 <5>
-      uds_address /var/run/ptp4l.0.socket <6>
-      [ens4f0] <7>
+      domainNumber 25
+      uds_address /var/run/ptp4l.0.socket
+      [ens4f0]
       ts2phc.extts_polarity rising
       ts2phc.extts_correction -10
       ts2phc.master 0
@@ -319,7 +677,7 @@ spec:
       ts2phc.extts_polarity rising
       ts2phc.extts_correction -27
       ts2phc.master 0
-    ts2phcOpts: -s generic -a --ts2phc.rh_external_pps 1 <8>
+    ts2phcOpts: -s generic -a --ts2phc.rh_external_pps 1
   recommend:
   - match:
     - nodeLabel: node-role.kubernetes.io/master
@@ -330,34 +688,41 @@ spec:
     priority: 4
     profile: 01-tbc-tr
 ----
-<1> All TT ports have the `masterOnly` set to 1.
-<2> The `phc2sysOpts` setting in the TR profile specifies the upstream port `ens4f1` as the source of the node time synchronization.
-<3> The TR profile contains the hardware plugin section.
-<4> The interconnections section in the hardware plugin has three NICs: `ens4f0`, `ens1f0`, and `ens8f0`. The leading NIC, `ens4f0`, is the only one with the `gnnsInput` field, set to `false`, and the `upstreamPort` field that specifies the TR port. It also has a list of `phaseOutputConnectors`, `SMA1` and `SMA2`. The following NICs have the `inputConnector` field. Set the time receiver NIC `ens4f0` and the specific TR port. that is `upstreamPort: ens4f1`, for both T-BC and T-TSC configurations.
-<5> The `ts2phc` configuration contains the `domainNumber` of the upstream PTP domain.
-<6> The `ts2phc` configuration contains the `uds_address`. Its value is not important because the daemon patches it with the correct address.
-<7> The `ts2phc` configuration must include all NICs participating in this setup (`ens4f0`, `ens1f0`, and `ens8f0`).
-<8> `ts2phcOpts` sets the source as generic with `-s generic` and automatic with `-a`. The last option, `--ts2phc.rh_external_pps 1`, configures it to operate with external phase source, the digital phase-locked loop (DPLL).
 +
 [NOTE]
 ====
-In the single-NIC case, disable all pins or enable outputs if using for 1PPS measurements.
+This example shows a multi-card T-BC configuration with three NICs. To adapt for other scenarios:
+
+T-TSC:
+
+* Remove the `00-tbc-tt` profile entirely
+* Set `clock_type` to `OC` in the TR profile
+* In `ts2phcConf`, list only the TR NIC
+
+Single-card T-BC:
+
+* In the TT profile, list only ports from the same NIC as the TR port
+* Remove additional NIC entries from the `interconnections` and `pins` sections
+* In `ts2phcConf`, list only the single NIC
+* Set `phaseOutputConnectors` to an empty array or disable all pins if not using SMA outputs
 ====
 
-[NOTE]
-====
-To render this configuration for T-TSC operation, remove the `00-tbc-tt` profile and adjust the `ts2phcConf` section to list only the TR NIC.
-====
+. Save the configuration to a file, for example `t-bc-holdover-config.yaml`, and apply it:
++
+[source,terminal]
+----
+$ oc apply -f t-bc-holdover-config.yaml
+----
 
 .Verification
 
-To get the T-BC status, run the following command:
-
+* Get the status of the T-BC by running the following command:
++
 [source,terminal]
 ----
-$ oc -linuxptp-daemon-container logs ds/linuxptp-daemon --since=1s -f |grep T-BC
+$ oc logs ds/linuxptp-daemon -n openshift-ptp -c linuxptp-daemon-container --since=1s -f | grep T-BC
 ----
-
++
 .Example output
 [source,terminal]
 ----
@@ -365,4 +730,9 @@ T-BC[1760525446]:[ts2phc.1.config] ens4f0 offset 1 T-BC-STATUS s2
 T-BC[1760525447]:[ts2phc.1.config] ens4f0 offset 1 T-BC-STATUS s2
 T-BC[1760525448]:[ts2phc.1.config] ens4f0 offset -1 T-BC-STATUS s2
 ----
-This is reported every second, where `s2` indicates it is locked, `s1` indicates holdover is activated, and `s0`, unlocked.
++
+The status is reported every second:
++
+* `s2`: Locked - synchronized with the upstream clock
+* `s1`: Holdover - upstream source lost, internal oscillator maintaining timing
+* `s0`: Unlocked - holdover limits exceeded or no valid holdover data