mirror of
https://github.com/openshift/openshift-docs.git
synced 2026-02-05 12:46:18 +01:00
Dual NIC HA PTP T-BC docs
Aneesh's review comments Updates for Aneesh May 2 Jeana's review comments dual-NIC is a compound adj Update for Hen
This commit is contained in:
Aidan Reilly
committed by
openshift-cherrypick-robot
openshift-cherrypick-robot
parent
6c2ea790ee
commit
b271f8b869
@@ -0,0 +1,87 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/ptp/configuring-ptp.adoc
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="cnf-getting-the-dpll-firmware-version-for-intel-800-series-nics_{context}"]
|
||||
= Getting the DPLL firmware version for the CGU in an Intel 800 series NIC
|
||||
|
||||
You can get the digital phase-locked loop (DPLL) firmware version for the Clock Generation Unit (CGU) in an Intel 800 series NIC by opening a debug shell to the cluster node and querying the NIC hardware.
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* You have installed the OpenShift CLI (`oc`).
|
||||
|
||||
* You have logged in as a user with `cluster-admin` privileges.
|
||||
|
||||
* You have installed an Intel 800 series NIC in the cluster host.
|
||||
|
||||
* You have installed the PTP Operator on a bare-metal cluster with hosts that support PTP.
|
||||
|
||||
.Procedure
|
||||
|
||||
. Start a debug pod by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc debug node/<node_name>
|
||||
----
|
||||
+
|
||||
where:
|
||||
+
|
||||
--
|
||||
<node_name>:: Is the node where you have installed the Intel 800 series NIC.
|
||||
--
|
||||
|
||||
. Check the CGU firmware version in the NIC by using the `devlink` tool and the bus and device name where the NIC is installed.
|
||||
For example, run the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
sh-4.4# devlink dev info <bus_name>/<device_name> | grep cgu
|
||||
----
|
||||
+
|
||||
where:
|
||||
+
|
||||
--
|
||||
<bus_name> :: Is the bus where the NIC is installed. For example, `pci`.
|
||||
<device_name> :: Is the NIC device name. For example, `0000:51:00.0`.
|
||||
--
|
||||
+
|
||||
.Example output
|
||||
[source,terminal]
|
||||
----
|
||||
cgu.id 36 <1>
|
||||
fw.cgu 8032.16973825.6021 <2>
|
||||
----
|
||||
<1> CGU hardware revision number
|
||||
<2> The DPLL firmware version running in the CGU, where the DPLL firmware version is `6201`, and the DPLL model is `8032`.
|
||||
The string `16973825` is a shorthand representation of the binary version of the DPLL firmware version (`1.3.0.1`).
|
||||
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
The firmware version has a leading nibble and 3 octets for each part of the version number.
|
||||
The number `16973825` in binary is `0001 0000 0011 0000 0000 0000 0001`.
|
||||
Use the binary value to decode the firmware version.
|
||||
For example:
|
||||
|
||||
.DPLL firmware version
|
||||
[cols="1,2", width="90%", options="header"]
|
||||
|====
|
||||
|Binary part
|
||||
|Decimal value
|
||||
|
||||
|`0001`
|
||||
|1
|
||||
|
||||
|`0000 0011`
|
||||
|3
|
||||
|
||||
|`0000 0000`
|
||||
|0
|
||||
|
||||
|`0000 0001`
|
||||
|1
|
||||
|====
|
||||
====
|
||||
@@ -8,12 +8,12 @@
|
||||
|
||||
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`, `ts2phc`) as grandmaster clock (T-GM) for dual E810 Westport Channel NICs by creating a `PtpConfig` custom resource (CR) that configures the host NICs.
|
||||
|
||||
For distributed RAN (D-RAN) use cases, you can configure PTP for dual NICs as follows:
|
||||
For distributed RAN (D-RAN) use cases, you can configure PTP for dual-NICs as follows:
|
||||
|
||||
* NIC one is synced to the global navigation satellite system (GNSS) time source.
|
||||
* NIC two is synced to the 1PPS timing output provided by NIC one. This configuration is provided by the PTP hardware plugin in the `PtpConfig` CR.
|
||||
|
||||
The dual NIC PTP T-GM configuration uses a single instance of `ptp4l` and one `ts2phc` process reporting two `ts2phc` instances, one for each NIC.
|
||||
The dual-NIC PTP T-GM configuration uses a single instance of `ptp4l` and one `ts2phc` process reporting two `ts2phc` instances, one for each NIC.
|
||||
The host system clock is synchronized from the NIC that is connected to the GNSS time source.
|
||||
|
||||
[NOTE]
|
||||
|
||||
@@ -8,13 +8,13 @@
|
||||
|
||||
Use this information to understand how to use the link:https://github.com/openshift/linuxptp-daemon/blob/release-4.14/addons/intel/e810.go[Intel E810-XXVDA4T hardware plugin] to configure a pair of E810 network interfaces as PTP grandmaster clock (T-GM).
|
||||
|
||||
Before you configure the dual NIC cluster host, you must connect the two NICs with an SMA1 cable using the 1PPS faceplace connections.
|
||||
Before you configure the dual-NIC cluster host, you must connect the two NICs with an SMA1 cable using the 1PPS faceplace connections.
|
||||
|
||||
When you configure a dual NIC T-GM, you need to compensate for the 1PPS signal delay that occurs when you connect the NICs using the SMA1 connection ports.
|
||||
When you configure a dual-NIC T-GM, you need to compensate for the 1PPS signal delay that occurs when you connect the NICs using the SMA1 connection ports.
|
||||
Various factors such as cable length, ambient temperature, and component and manufacturing tolerances can affect the signal delay.
|
||||
To compensate for the delay, you must calculate the specific value that you use to offset the signal delay.
|
||||
|
||||
.E810 dual NIC T-GM PtpConfig CR reference
|
||||
.E810 dual-NIC T-GM PtpConfig CR reference
|
||||
[cols="1,2" width="90%", options="header"]
|
||||
|====
|
||||
|PtpConfig field
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/ptp/about-ptp.adoc
|
||||
// * networking/ptp/configuring-ptp.adoc
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="install-ptp-operator-cli_{context}"]
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/ptp/about-ptp.adoc
|
||||
// * networking/ptp/configuring-ptp.adoc
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="install-ptp-operator-web-console_{context}"]
|
||||
|
||||
@@ -99,4 +99,10 @@ Possible values are 0 (`UNAVAILABLE`) and 1 (`AVAILABLE`).
|
||||
GNSS provides satellite-based positioning, navigation, and timing services globally.
|
||||
Possible values are 0 (`NOFIX`), 1 (`DEAD RECKONING ONLY`), 2 (`2D-FIX`), 3 (`3D-FIX`), 4 (`GPS+DEAD RECKONING FIX`), 5, (`TIME ONLY FIX`).
|
||||
|`{from="gnss",iface="ens2fx",node="compute-1.example.com",process="gnss"} 3`
|
||||
|
||||
|`openshift_ptp_ha_profile_status`
|
||||
|Returns the current status of the highly available system clock when there are multiple time sources on different NICs.
|
||||
Possible values are 0 (`INACTIVE`), and 1 (`ACTIVE`).
|
||||
|`{node="node1",process="phc2sys",profile="profile1"} 1`
|
||||
`{node="node1",process="phc2sys",profile="profile2"} 0`
|
||||
|====
|
||||
|
||||
@@ -4,12 +4,12 @@
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="ptp-configuring-linuxptp-services-as-bc-for-dual-nic_{context}"]
|
||||
= Configuring linuxptp services as boundary clocks for dual NIC hardware
|
||||
= Configuring linuxptp services as boundary clocks for dual-NIC hardware
|
||||
|
||||
:FeatureName: Precision Time Protocol (PTP) hardware with dual NIC configured as boundary clocks
|
||||
:FeatureName: Precision Time Protocol (PTP) hardware with dual-NIC configured as boundary clocks
|
||||
include::snippets/technology-preview.adoc[leveloffset=+1]
|
||||
|
||||
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`) as boundary clocks for dual NIC hardware by creating a `PtpConfig` custom resource (CR) object for each NIC.
|
||||
You can configure the `linuxptp` services (`ptp4l`, `phc2sys`) as boundary clocks for dual-NIC hardware by creating a `PtpConfig` custom resource (CR) object for each NIC.
|
||||
|
||||
Dual NIC hardware allows you to connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.
|
||||
|
||||
@@ -76,7 +76,7 @@ spec:
|
||||
You must completely remove the `phc2sysOpts` field from the second `PtpConfig` CR to disable the `phc2sys` service on the second NIC.
|
||||
====
|
||||
|
||||
. Create the dual NIC `PtpConfig` CRs by running the following commands:
|
||||
. Create the dual-NIC `PtpConfig` CRs by running the following commands:
|
||||
|
||||
.. Create the CR that configures PTP for the first NIC:
|
||||
+
|
||||
@@ -94,7 +94,7 @@ $ oc create -f boundary-clock-ptp-config-nic2.yaml
|
||||
|
||||
.Verification
|
||||
|
||||
* Check that the PTP Operator has applied the `PtpConfig` CRs for both NICs. Examine the logs for the `linuxptp` daemon corresponding to the node that has the dual NIC hardware installed. For example, run the following command:
|
||||
* Check that the PTP Operator has applied the `PtpConfig` CRs for both NICs. Examine the logs for the `linuxptp` daemon corresponding to the node that has the dual-NIC hardware installed. For example, run the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
|
||||
@@ -0,0 +1,171 @@
|
||||
// Module included in the following assemblies:
|
||||
//
|
||||
// * networking/ptp/configuring-ptp.adoc
|
||||
|
||||
:_mod-docs-content-type: PROCEDURE
|
||||
[id="ptp-configuring-linuxptp-services-as-ha-bc-for-dual-nic_{context}"]
|
||||
= Configuring linuxptp as a highly available system clock for dual-NIC Intel E810 PTP boundary clocks
|
||||
|
||||
You can configure the `linuxptp` services `ptp4l` and `phc2sys` as a highly available (HA) system clock for dual PTP boundary clocks (T-BC).
|
||||
|
||||
The highly available system clock uses multiple time sources from dual-NIC Intel E810 Salem channel hardware configured as two boundary clocks.
|
||||
Two boundary clocks instances participate in the HA setup, each with its own configuration profile.
|
||||
You connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.
|
||||
|
||||
Create two `PtpConfig` custom resource (CR) objects that configure the NICs as T-BC and a third `PtpConfig` CR that configures high availability between the two NICs.
|
||||
|
||||
[IMPORTANT]
|
||||
====
|
||||
When you create the `PtpConfig` CRs, ensure that the `phc2sysOpts` field is an empty string to prevent setting up the `phc2sys` processes on these two profiles.
|
||||
====
|
||||
|
||||
The third `PtpConfig` CR configures a highly available system clock service.
|
||||
The CR sets the `ptp4lOpts` field to an empty string to prevent the `ptp4l` process from running.
|
||||
The CR adds profiles for the `ptp4l` configurations under the `spec.profile.ptpSettings.haProfiles` key and passes the kernel socket path of those profiles to the `phc2sys` service.
|
||||
When a `ptp4l` failure occurs, the `phc2sys` service switches to the backup `ptp4l` configuration.
|
||||
When the primary profile becomes active again, the `phc2sys` service reverts to the original state.
|
||||
|
||||
.Prerequisites
|
||||
|
||||
* Install the {oc-first}.
|
||||
|
||||
* Log in as a user with `cluster-admin` privileges.
|
||||
|
||||
* Install the PTP Operator.
|
||||
|
||||
* Configure a cluster node with Intel E810 Salem channel dual-NIC.
|
||||
|
||||
.Procedure
|
||||
|
||||
. Create two separate `PtpConfig` CRs, one for each NIC, using the CRs in "Configuring linuxptp services as boundary clocks for dual-NIC hardware" as a reference for each CR.
|
||||
|
||||
.. Create the `ha-ptp-config-nic1.yaml` file, specifying an empty string for the `phc2sysOpts` field.
|
||||
For example:
|
||||
+
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: ptp.openshift.io/v1
|
||||
kind: PtpConfig
|
||||
metadata:
|
||||
name: ha-ptp-config-nic1
|
||||
namespace: openshift-ptp
|
||||
spec:
|
||||
profile:
|
||||
- name: "ha-ptp-config-profile1"
|
||||
ptp4lOpts: "-2 --summary_interval -4"
|
||||
ptp4lConf: | <1>
|
||||
[ens5f1]
|
||||
masterOnly 1
|
||||
[ens5f0]
|
||||
masterOnly 0
|
||||
#...
|
||||
phc2sysOpts: "" <2>
|
||||
----
|
||||
<1> Specify the required interfaces to start `ptp4l` as a boundary clock. For example, `ens5f0` synchronizes from a grandmaster clock and `ens5f1` synchronizes connected devices.
|
||||
<2> Set `phc2sysOpts` with an empty string.
|
||||
These values are populated from the `spec.profile.ptpSettings.haProfiles` field of the `PtpConfig` CR that configures high availability.
|
||||
|
||||
.. Apply the `PtpConfig` CR for NIC 1 by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc create -f ha-ptp-config-nic1.yaml
|
||||
----
|
||||
|
||||
.. Create the `ha-ptp-config-nic2.yaml` file, specifying an empty string for the `phc2sysOpts` field.
|
||||
For example:
|
||||
+
|
||||
[source,yaml]
|
||||
----
|
||||
apiVersion: ptp.openshift.io/v1
|
||||
kind: PtpConfig
|
||||
metadata:
|
||||
name: ha-ptp-config-nic2
|
||||
namespace: openshift-ptp
|
||||
spec:
|
||||
profile:
|
||||
- name: "ha-ptp-config-profile2"
|
||||
ptp4lOpts: "-2 --summary_interval -4"
|
||||
ptp4lConf: |
|
||||
[ens7f1]
|
||||
masterOnly 1
|
||||
[ens7f0]
|
||||
masterOnly 0
|
||||
#...
|
||||
phc2sysOpts: ""
|
||||
----
|
||||
|
||||
.. Apply the `PtpConfig` CR for NIC 2 by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc create -f ha-ptp-config-nic2.yaml
|
||||
----
|
||||
|
||||
. Create the `PtpConfig` CR that configures the HA system clock.
|
||||
For example:
|
||||
|
||||
.. Create the `ptp-config-for-ha.yaml` file:
|
||||
+
|
||||
[source,yaml]
|
||||
----
|
||||
include::snippets/ztp_PtpConfigForHA.yaml[]
|
||||
----
|
||||
+
|
||||
[IMPORTANT]
|
||||
====
|
||||
Do not apply the high availability `PtpConfig` CR before the `PtpConfig` CRs that configure the individual NICs.
|
||||
====
|
||||
|
||||
.. Apply the HA `PtpConfig` CR by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc create -f ptp-config-for-ha.yaml
|
||||
----
|
||||
|
||||
.Verification
|
||||
|
||||
* Verify that the PTP Operator has applied the `PtpConfig` CRs correctly.
|
||||
Perform the following steps:
|
||||
|
||||
.. Get the list of pods in the `openshift-ptp` namespace by running the following command:
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc get pods -n openshift-ptp -o wide
|
||||
----
|
||||
+
|
||||
.Example output
|
||||
[source,terminal]
|
||||
----
|
||||
NAME READY STATUS RESTARTS AGE IP NODE
|
||||
linuxptp-daemon-4xkrb 1/1 Running 0 43m 10.1.196.24 compute-0.example.com
|
||||
ptp-operator-657bbq64c8-2f8sj 1/1 Running 0 43m 10.129.0.61 control-plane-1.example.com
|
||||
----
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
There should be only one `linuxptp-daemon` pod.
|
||||
====
|
||||
|
||||
.. Check that the profile is correct by running the following command.
|
||||
Examine the logs of the `linuxptp` daemon that corresponds to the node you specified in the `PtpConfig` profile.
|
||||
+
|
||||
[source,terminal]
|
||||
----
|
||||
$ oc logs linuxptp-daemon-4xkrb -n openshift-ptp -c linuxptp-daemon-container
|
||||
----
|
||||
+
|
||||
.Example output
|
||||
[source,terminal]
|
||||
----
|
||||
I1115 09:41:17.117596 4143292 daemon.go:107] in applyNodePTPProfile
|
||||
I1115 09:41:17.117604 4143292 daemon.go:109] updating NodePTPProfile to:
|
||||
I1115 09:41:17.117607 4143292 daemon.go:110] ------------------------------------
|
||||
I1115 09:41:17.117612 4143292 daemon.go:102] Profile Name: ha-ptp-config-profile1
|
||||
I1115 09:41:17.117616 4143292 daemon.go:102] Interface:
|
||||
I1115 09:41:17.117620 4143292 daemon.go:102] Ptp4lOpts: -2
|
||||
I1115 09:41:17.117623 4143292 daemon.go:102] Phc2sysOpts: -a -r -n 24
|
||||
I1115 09:41:17.117626 4143292 daemon.go:116] ------------------------------------
|
||||
----
|
||||
@@ -4,12 +4,12 @@
|
||||
|
||||
:_mod-docs-content-type: CONCEPT
|
||||
[id="ptp-dual-nics_{context}"]
|
||||
= Using dual Intel E810 NIC hardware with PTP
|
||||
= Using dual-NIC Intel E810 hardware with PTP
|
||||
|
||||
{product-title} supports single and dual NIC Intel E810 hardware for precision PTP timing in grandmaster clocks (T-GM) and boundary clocks (T-BC).
|
||||
{product-title} supports single and dual-NIC Intel E810 hardware for precision PTP timing in grandmaster clocks (T-GM) and boundary clocks (T-BC).
|
||||
|
||||
Dual NIC grandmaster clock::
|
||||
You can use a cluster host that has dual NIC hardware as PTP grandmaster clock.
|
||||
You can use a cluster host that has dual-NIC hardware as PTP grandmaster clock.
|
||||
One NIC receives timing information from the global navigation satellite system (GNSS).
|
||||
The second NIC receives the timing information from the first using the SMA1 Tx/Rx connections on the E810 NIC faceplate.
|
||||
The system clock on the cluster host is synchronized from the NIC that is connected to the GNSS satellite.
|
||||
@@ -22,10 +22,22 @@ image::561_OpenShift_Using_PTP_network_0124.png[Dual NIC PTP grandmaster clock c
|
||||
+
|
||||
[NOTE]
|
||||
====
|
||||
In a dual NIC T-GM configuration, a single `ts2phc` process reports as two `ts2phc` instances in the system.
|
||||
In a dual-NIC T-GM configuration, a single `ts2phc` process reports as two `ts2phc` instances in the system.
|
||||
====
|
||||
|
||||
Dual NIC boundary clock::
|
||||
For 5G telco networks that deliver mid-band spectrum coverage, each virtual distributed unit (vDU) requires connections to 6 radio units (RUs). To make these connections, each vDU host requires 2 NICs configured as boundary clocks.
|
||||
+
|
||||
Dual NIC hardware allows you to connect each NIC to the same upstream leader clock with separate `ptp4l` instances for each NIC feeding the downstream clocks.
|
||||
|
||||
Highly available system clock with dual-NIC boundary clocks::
|
||||
You can configure Intel E810-XXVDA4 Salem channel dual-NIC hardware as dual PTP boundary clocks that provide timing for a highly available system clock.
|
||||
This is useful when you have multiple time sources on different NICs.
|
||||
High availability ensures that the node does not lose timing synchronisation if one of the two timing sources is lost or disconnected.
|
||||
+
|
||||
Each NIC is connected to the same upstream leader clock.
|
||||
Highly available boundary clocks use multiple PTP domains to synchronize with the target system clock.
|
||||
When a T-BC is highly available, the host system clock can maintain the correct offset even if one or more `ptp4l` instances syncing the NIC PHC clock fails.
|
||||
If any single SFP port or cable failure occurs, the boundary clock stays in sync with the leader clock.
|
||||
+
|
||||
Boundary clock leader source selection is done using the A-BMCA algorithm. For more information, see link:https://www.itu.int/rec/T-REC-G.8275.1/en[ITU-T recommendation G.8275.1].
|
||||
|
||||
@@ -30,7 +30,7 @@ Optional: subscribe applications to PTP events that happen on the node that the
|
||||
You subscribe the application to events via HTTP.
|
||||
|
||||
Limits and requirements::
|
||||
* High availability is not supported with dual NIC configurations.
|
||||
* High availability is not supported with dual-NIC configurations.
|
||||
|
||||
* GPS offsets are not reported.
|
||||
Use a default offset of less than or equal to 5.
|
||||
|
||||
@@ -46,6 +46,8 @@ include::modules/nw-ptp-configuring-linuxptp-services-as-boundary-clock.adoc[lev
|
||||
|
||||
include::modules/ptp-configuring-linuxptp-services-as-boundary-clock-dual-nic.adoc[leveloffset=+2]
|
||||
|
||||
include::modules/ptp-configuring-linuxptp-services-as-ha-bc-for-dual-nic.adoc[leveloffset=+2]
|
||||
|
||||
include::modules/nw-ptp-configuring-linuxptp-services-as-ordinary-clock.adoc[leveloffset=+1]
|
||||
|
||||
[role="_additional-resources"]
|
||||
@@ -68,4 +70,6 @@ include::modules/cnf-configuring-log-filtering-for-linuxptp.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/cnf-troubleshooting-common-ptp-operator-issues.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/cnf-getting-the-dpll-firmware-version-for-intel-800-series-nics.adoc[leveloffset=+1]
|
||||
|
||||
include::modules/cnf-about-collecting-ptp-data.adoc[leveloffset=+1]
|
||||
|
||||
22
snippets/ztp_PtpConfigForHA.yaml
Normal file
22
snippets/ztp_PtpConfigForHA.yaml
Normal file
@@ -0,0 +1,22 @@
|
||||
apiVersion: ptp.openshift.io/v1
|
||||
kind: PtpConfig
|
||||
metadata:
|
||||
name: boundary-ha
|
||||
namespace: openshift-ptp
|
||||
annotations:
|
||||
ran.openshift.io/ztp-deploy-wave: "10"
|
||||
spec:
|
||||
profile:
|
||||
- name: "boundary-ha"
|
||||
ptp4lOpts: " "
|
||||
phc2sysOpts: "-a -r -n 24"
|
||||
ptpSchedulingPolicy: SCHED_FIFO
|
||||
ptpSchedulingPriority: 10
|
||||
ptpSettings:
|
||||
logReduce: "true"
|
||||
haProfiles: "ha-ptp-config-nic1,ha-ptp-config-nic2"
|
||||
recommend:
|
||||
- profile: "boundary-ha"
|
||||
priority: 4
|
||||
match:
|
||||
- nodeLabel: "node-role.kubernetes.io/$mcp"
|
||||
Reference in New Issue
Block a user