Update content for multiple networks

- OSDOCS-475
2026-02-05 12:46:18 +01:00 · 2019-08-21 16:56:46 -04:00
parent c98644542e
commit 8df1fb5848
19 changed files with 1002 additions and 87 deletions
--- a/_topic_map.yml
+++ b/_topic_map.yml
@@ -285,12 +285,25 @@ Topics:
  File: dns-operator
 - Name: Understanding the Ingress Operator
  File: ingress-operator
- Name: Multiple networks
-  File: managing-multinetworking
-  Distros: openshift-enterprise,openshift-origin
 - Name: Configuring network policy
  File: configuring-networkpolicy
  Distros: openshift-origin,openshift-enterprise
+- Name: Multiple networks
+  Dir: multiple-networks
+  Distros: openshift-enterprise,openshift-origin
+  Topics:
+  - Name: Understanding multiple networks
+    File: understanding-multiple-networks
+  - Name: Configuring a bridge network
+    File: configuring-bridge
+  - Name: Configuring a macvlan network
+    File: configuring-macvlan
+  - Name: Configuring an ipvlan network
+    File: configuring-ipvlan
+  - Name: Configuring a host-device network
+    File: configuring-host-device
+  - Name: Configuring SR-IOV
+    File: configuring-sr-iov
 - Name: OpenShift SDN
  Dir: openshift-sdn
  Topics:
--- a/modules/nw-multus-add-pod.adoc
+++ b/modules/nw-multus-add-pod.adoc
@@ -0,0 +1,66 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-add-pod_{context}"]
+= Adding a Pod to an additional network
+
+You can add a Pod to an additional network. {product-title} will continue using
+the primary network for normal cluster related network traffic.
+
+.Prerequisites
+
+* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
+* You must log in to the cluster.
+
+.Procedure
+
+To add a Pod to an additional network, complete the following steps:
+
+. Edit the Pod resource definition. If you are editing an existing Pod, run the
+following command to edit its definition in the default editor. Replace `<name>`
+with the name of the Pod to edit.
+
+----
+$ oc edit pod <name>
+----
+
+. In the Pod resource definition, add the `k8s.v1.cni.cncf.io/networks`
+parameter to the Pod `metadata` mapping. The `k8s.v1.cni.cncf.io/networks`
+accepts a comma separated string of one or more `AdditionalNetworkDefinition`
+names:
+
+[source,yaml] 
+----
+metadata:
+  annotations:
+    k8s.v1.cni.cncf.io/networks: <network>[,<network>,...] <1>
+----
+<1> Replace `<network>` with the name of the additional network to associate
+with the Pod. To specify more than one additional network, separate each network
+with a comma. Do not include whitespace between the comma. If you specify
+the same additional network multiple times, that Pod will have multiple network
+interfaces attached to that network.
+
+In the following example, two additional networks are attached to the Pod:
+
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: example-pod
+  annotations:
+    k8s.v1.cni.cncf.io/networks: net1,net2
+spec:
+  containers:
+  - name: example-pod
+    command: ["/bin/bash", "-c", "sleep 2000000000000"]
+    image: centos/tools
+----
+
+. Optional: Confirm that the annotation exists in the Pod CR by running the
+following command:
+
+----
+$ oc describe pod <name>
+----
--- a/modules/nw-multus-bridge-object.adoc
+++ b/modules/nw-multus-bridge-object.adoc
@@ -0,0 +1,84 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-bridge-object_{context}"]
+= Configuration for bridge
+
+// TODO - duplicated in ipvlan, copy changes from there to here.
+
+.bridge CNI plug-in YAML configuration
+[source,yaml]
+----
+name: <name> <1>
+namespace: <namespace> <2>
+rawCNIConfig: '' <3>
+type: Raw
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from
+the `rawCNIConfig` JSON object.
+
+<2> `namespace`: Specify the namespace to create the network attachment in. If
+a value is not specified, the `default` namespace is used.
+
+<3> `rawCNIConfig`: Specify the CNI plug-in configuration.
+
+.bridge CNI plug-in JSON configuration object
+[source,json]
+----
+{
+  "cniVersion": "0.3.1",
+  "name": "<name>", <1>
+  "type": "bridge",
+  "bridge": "<bridge>", <2>
+  "ipam": { <3>
+    ...
+  },
+  "ipMasq": false, <4>
+  "isGateway": false, <5>
+  "isDefaultGateway": false, <6>
+  "forceAddress": false, <7>
+  "hairpinMode": false, <8>
+  "promiscMode": false, <9>
+  "vlan": <vlan>, <10>
+  "mtu": <mtu> <11>
+}
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition`.
+
+<2> `bridge`: Specify the name of the virtual bridge to use. If the bridge
+interface does not exist on the host, it is created. The default value is
+`cni0`.
+
+<3> `ipam`: Specify a configuration object for the ipam CNI plug-in. The plug-in
+manages IP address assignment for the network attachment definition.
+
+<4> `ipMasq`: Set to `true` to enable IP masquerading for traffic that leaves the
+virtual network. The source IP address for all traffic is rewritten to the
+bridge's IP address. If the bridge does not have an IP address, this setting has
+no effect. The default value is `false`.
+
+<5> `isGateway`: Set to `true` to assign an IP address to the bridge. The
+default value is `false`.
+
+<6> `isDefaultGateway`: Set to `true` to configure the bridge as the default
+gateway for the virtual network. The default value is `false`. If
+`isDefaultGateway` is set to `true`, then `isGateway` is also set to `true`
+automatically. 
+
+<7> `forceAddress`: Set to `true` to allow assignment of a previously assigned
+IP address to the virtual bridge. When set to `false`, if an IPv4 address or an
+IPv6 address from overlapping subsets is assigned to the virtual bridge, an
+error occurs. The default value is `false`.
+
+<8> `hairpinMode`: Set to `true` to allow the virtual bridge to send an ethernet
+frame back through the virtual port it was received on. This mode is also known
+as _reflective relay_. The default value is `false`.
+
+<9> `promiscMode`: Set to `true` to enable promiscuous mode on the bridge. The
+default value is `false`.
+
+<10> `vlan`: Specify a virtual LAN (VLAN) tag as an integer value. By default,
+no VLAN tag is assigned.
+
+<11> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The
+default value is automatically chosen by the kernel.
--- a/modules/nw-multus-create-network.adoc
+++ b/modules/nw-multus-create-network.adoc
@@ -0,0 +1,153 @@
+// Module included in the following assemblies:
+//
+
+// Configuring Multus plug-ins using the Cluster Network Operator
+// is nearly identical in each case.
+
+ifeval::["{context}" == "configuring-macvlan"]
+:plugin: macvlan
+:yaml:
+endif::[]
+ifeval::["{context}" == "configuring-ipvlan"]
+:plugin: ipvlan
+:json:
+endif::[]
+ifeval::["{context}" == "configuring-bridge"]
+:plugin: bridge
+:json:
+endif::[]
+ifeval::["{context}" == "configuring-host-device"]
+:plugin: host-device
+:json:
+endif::[]
+
+[id="nw-multus-create-network_{context}"]
+= Creating an additional network attachment with the {plugin} CNI plug-in
+
+The Cluster Network Operator (CNO) manages additional network definitions. When
+you specify an additional network to create, the CNO creates the
+`NetworkAttachmentDefinition` Custom Resource (CR) automatically.
+
+[IMPORTANT]
+====
+Do not edit the `NetworkAttachmentDefinition` CRs that the Cluster Network
+Operator manages. Doing so might disrupt network traffic on your additional
+network.
+====
+
+.Prerequisites
+
+* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
+* Log in as a user with `cluster-admin` privileges.
+
+.Procedure
+
+To create an additional network for your cluster, complete the following steps:
+
+. Run the following command to edit the CNO CR:
+
+----
+$ oc edit networks.operator.openshift.io cluster
+----
+
+. Modify the CR that you are creating by adding the configuration for the
+additional network you are creating.
+
+ifdef::yaml[]
+The following YAML configures the {plugin} CNI plug-in:
+
+[source,yaml]
+----
+apiVersion: operator.openshift.io/v1
+kind: Network
+metadata:
+  name: cluster
+spec:
+  additionalNetworks: <1>
+  - name: example-addn-network
+    type: SimpleMacvlan
+    simpleMacvlanConfig:
+      ipamConfig:
+        type: DHCP
+----
+endif::yaml[]
+ifdef::json[]
+The following YAML configures the {plugin} CNI plug-in:
+endif::json[]
+
+ifeval::["{plugin}" == "bridge"]
+[source,yaml,subs="attributes+"]
+----
+apiVersion: operator.openshift.io/v1
+kind: Network
+metadata:
+  name: cluster
+spec:
+  additionalNetworks: <1>
+  - name: test-network-1
+    type: Raw
+    rawCNIConfig: '{
+      "cniVersion": "0.3.1",
+      "type": "{plugin}",
+      "master": "eth1",
+      "ipam": {
+        "type": "dhcp"
+        }
+    }'
+----
+endif::[]
+ifeval::["{plugin}" == "host-device"]
+[source,yaml,subs="attributes+"]
+----
+apiVersion: operator.openshift.io/v1
+kind: Network
+metadata:
+  name: cluster
+spec:
+  additionalNetworks: <1>
+  - name: test-network-1
+    type: Raw
+    rawCNIConfig: '{
+      "cniVersion": "0.3.1",
+      "type": "{plugin}",
+      "device": "eth1"
+    }'
+----
+endif::[]
+ifeval::["{plugin}" == "ipvlan"]
+[source,yaml,subs="attributes+"]
+----
+apiVersion: operator.openshift.io/v1
+kind: Network
+metadata:
+  name: cluster
+spec:
+  additionalNetworks: <1>
+  - name: test-network-1
+    type: Raw
+    rawCNIConfig: '{
+      "cniVersion": "0.3.1",
+      "type": "{plugin}",
+      "master": "eth1",
+      "mode": "l2",
+      "ipam": {
+        "type": "dhcp"
+        }
+    }'
+----
+endif::[]
+<1> `additionalNetworks`: Specify the configuration for the additional network
+attachment definition.
+
+. Save your changes and quit the text editor to commit your changes.
+
+. Optional: Confirm that the CNO created the `NetworkAttachmentDefinition` CR by
+running the following command. There might be a delay before the CNO creates the
+CR.
+
+----
+$ oc get network-attachment-definitions
+NAME                 AGE
+example-network      14m
+example-macvlan      21m
+----
--- a/modules/nw-multus-delete-network.adoc
+++ b/modules/nw-multus-delete-network.adoc
@@ -0,0 +1,43 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-delete-network_{context}"]
+= Removing an additional network attachment definition
+
+As a cluster administrator, you can remove an additional network from your
+{product-title} cluster. The additional network is not removed from any Pods it
+is attached to.
+
+.Prerequisites
+
+* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
+* Log in as a user with `cluster-admin` privileges.
+
+.Procedure
+
+To remove an additional network from your cluster, complete the following steps:
+
+. Run the following command to edit the Cluster Network Operator (CNO) in your
+default text editor:
+
+----
+$ oc edit networks.operator.openshift.io cluster
+----
+
+. Modify the CR by removing the configuration from the `additionalNetworks`
+collection for the network attachment definition you are removing.
+
+[source,yaml]
+----
+apiVersion: operator.openshift.io/v1
+kind: Network
+metadata:
+  name: cluster
+spec:
+  additionalNetworks: [] <1>
+----
+<1>  If you are removing the configuration mapping for the only additional
+network attachment definition in the `additionalNetworks` collection, you must
+specify an empty collection.
+
+. Save your changes and quit the text editor to commit your changes.
--- a/modules/nw-multus-edit-network.adoc
+++ b/modules/nw-multus-edit-network.adoc
@@ -0,0 +1,49 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-edit-network_{context}"]
+= Modifying an additional network attachment definition
+
+As a cluster administrator, you can make changes to an existing additional
+network.
+
+.Prerequisites
+
+* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
+* Log in as a user with `cluster-admin` privileges.
+
+.Procedure
+
+To edit an additional network for your cluster, complete the following steps:
+
+. Run the following command to edit the Cluster Network Operator (CNO) CR in
+your default text editor:
+
+----
+$ oc edit networks.operator.openshift.io cluster
+----
+
+. In the `additionalNetworks` collection, update the additional network with
+your changes.
+
+. Save your changes and quit the text editor to commit your changes.
+
+. Optional: Confirm that the CNO updated the `NetworkAttachmentDefinition` CR by
+running the following command. Replace `<network-name>` with the name of the
+additional network to display. There might be a delay before the CNO updates the
+`NetworkAttachmentDefinition` CR to reflect your changes.
+
+----
+$ oc get network-attachment-definitions <network-name> -o yaml
+----
+
+For example, the following console output displays a
+`NetworkAttachmentDefinition` that is named `net1`:
+
+----
+$ oc get network-attachment-definitions net1 -o go-template='{{printf "%s\n" .spec.config}}'
+{ "cniVersion": "0.3.1", "type": "macvlan",
+"master": "ens5",
+"mode": "bridge",
+"ipam":       {"type":"static","routes":[{"dst":"0.0.0.0/0","gw":"10.128.2.1"}],"addresses":[{"address":"10.128.2.100/23","gateway":"10.128.2.1"}],"dns":{"nameservers":["172.30.0.10"],"domain":"us-west-2.compute.internal","search":["us-west-2.compute.internal"]}} }
+----
--- a/modules/nw-multus-host-device-object.adoc
+++ b/modules/nw-multus-host-device-object.adoc
@@ -0,0 +1,52 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-host-device-object_{context}"]
+= Configuration for host-device
+
+// TODO - consider refactoring this into its own module as it is used
+// by both ipvlan and bridge CNI plug-ins.
+
+.host-device CNI plug-in YAML configuration
+[source,yaml]
+----
+name: <name> <1>
+namespace: <namespace> <2>
+rawCNIConfig: '' <3>
+type: Raw
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from
+the `rawCNIConfig` JSON object.
+
+<2> `namespace`: Specify the namespace to create the network attachment in. If
+a value is not specified, the `default` namespace is used.
+
+<3> `rawCNIConfig`: Specify the CNI plug-in configuration.
+
+IMPORTANT: You can specify your network device using only one of the following
+parameters: `device`, `hwaddr`, `kernelpath`, or `pciBusID`.
+
+.host-device CNI plug-in JSON configuration object
+[source,json]
+----
+{
+  "cniVersion": "0.3.1",
+  "name": "<name>", <1>
+  "type": "host-device",
+  "device": "<device>", <2>
+  "hwaddr": "<hwaddr>", <3>
+  "kernelpath": "<kernelpath>", <4>
+  "pciBusID": "<pciBusID>" <5>
+}
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition`.
+
+<2> `device`: Specify the name of the device, such as `eth0`.
+
+<3> `hwaddr`: Specify the device hardware MAC address.
+
+<4> `kernelpath`: Specify the Linux kernel device path, such as
+`/sys/devices/pci0000:00/0000:00:1f.6`.
+
+<5> `pciBusID`: Specify the PCI address of the network device, such as
+`0000:00:1f.6`.
--- a/modules/nw-multus-ipam-object.adoc
+++ b/modules/nw-multus-ipam-object.adoc
@@ -0,0 +1,199 @@
+// Module included in the following assemblies:
+//
+
+// Because the Cluster Network Operator abstracts the configuration for
+// Macvlan, including IPAM configuration, this must be provided as YAML
+// for the Macvlan CNI plug-in only. In the future other Multus plug-ins
+// might be managed the same way by the CNO.
+
+ifeval::["{context}" == "configuring-macvlan"]
+:yaml:
+endif::[]
+ifeval::["{context}" != "configuring-macvlan"]
+:json:
+endif::[]
+
+:type: pass:q[Specify `static` to configure the plug-in to manage IP address \
+assignment. Specify `DHCP` to allow a DHCP server to manage IP \
+address assignment. You cannot specify any additional parameters if you \
+specify a value of `DHCP`.]
+
+:addresses: pass:q[that define IP addresses to assign to the virtual \
+interface. Both IPv4 and IPv6 IP addresses are supported.]
+
+:address: pass:q[A block of IP addresses specified in CIDR format to assign \
+to Pods on a worker node.]
+
+:address-gateway: pass:q[The default gateway to route egress network traffic to.]
+
+:routes: pass:q[describing routes to configure inside the Pod.]
+
+:route: pass:q[The IP address range in CIDR format.]
+
+:route-gateway: pass:q[The gateway to use to route network traffic to.]
+
+:dns: pass:q[The DNS configuration. Optional.]
+
+:nameservers: pass:q[one or more IP addresses for to send DNS queries to.]
+
+:domain: pass:q[The default domain to append to a host name. For example, if the \
+domain is set to `example.com`, a DNS lookup query for `example-host` will be \
+rewritten as `example-host.example.com`.]
+
+:search: pass:q[An array of domain names to append to an unqualified host name, \
+such as `example-host`, during a DNS lookup query.]
+
+[id="nw-multus-ipam-object_{context}"]
+= Configuration for ipam CNI plug-in
+
+The IP address management (IPAM) CNI plug-in manages IP address assignment for
+other CNI plug-ins.
+
+ifdef::json[]
+The following JSON configuration object describes the parameters that you can set.
+endif::json[]
+
+ifdef::yaml[]
+The following YAML configuration describes the parameters that you can set.
+endif::yaml[]
+
+IMPORTANT: If you set the `type` parameter to the `DHCP` value, you cannot set
+any other parameters.
+
+ifdef::json[]
+.IPAM CNI plug-in JSON configuration object
+[source,json]
+----
+{
+    "type": "<type>", <1>
+    "addresses": [ <2>
+      {
+        "address": "<address>", <3>
+        "gateway": "<gateway>" <4>
+      }
+    ],
+    "routes": [ <5>
+      {
+        "dst": "<dst>" <6>
+        "gw": "<gw>" <7>
+      }
+    ],
+    "dns": { <8>
+      "nameservers": ["<nameserver>"], <9>
+      "domain": "<domain>", <10>
+      "search": ["<search_domain>"] <11>
+    }
+}
+----
+<1> `type`: {type}
+
+<2> `addresses`: An array of {addresses}
+
+<3> `address`: {address}
+
+<4> `gateway`: {address-gateway}
+
+<5> `routes`: An array {routes}
+
+<6> `dst`: {route}
+
+<7> `gw`: {route-gateway}
+
+<8> `dns`: {dns}
+
+<9> `nameservers`: An of array of {nameservers}
+
+<10> `domain`: {domain}
+
+<11> `search`: {search}
+endif::json[]
+
+// YAML uses collection and mapping to describe arrays and objects
+
+ifdef::yaml[]
+.ipam CNI plug-in YAML configuration object
+[source,yaml]
+----
+ipamConfig: <1>
+  type: <type> <2>
+  ... <3>
+----
+<1> `ipamConfig`: The ipam configuration.
+
+<2> `type`: {type}
+
+<3> If you set the `type` parameter to `static`, then provide the
+`staticIPAMConfig` parameter.
+
+== Static ipam configuration YAML
+
+.Static ipam configuration YAML
+[source,yaml]
+----
+staticIPAMConfig:
+  addresses: <1>
+  - address: <address> <2>
+    gateway: <gateway> <3>
+  routes: <4>
+  - destination: <destination> <5>
+    gateway: <gateway> <6>
+  dns: <7>
+    nameservers: <8>
+    - <nameserver>
+    domain: <domain> <9>
+    search: <10>
+    - <search_domain>
+----
+<1> `addresses`: A collection of mappings {addresses}
+
+<2> `address`: {address}
+
+<3> `gateway`: {address-gateway}
+
+<4> `routes`: A collection of mappings {routes}
+
+<5> `destination`: {route}
+
+<6> `gateway`: {route-gateway}
+
+<7> `dns`: {dns}
+
+<8> `nameservers`: A collection of {nameservers}
+
+<9> `domain`: {domain}
+
+<10> `search`: {search}
+
+== Example ipam configuration YAML
+
+The following example shows an ipam configuration for static IP addresses:
+
+[source,yaml]
+----
+ipamConfig:
+  type: static
+  staticIPAMConfig:
+    addresses:
+    - address: 198.51.100.11/24
+      gateway: 198.51.100.10
+    routes:
+    - destination: 0.0.0.0/0
+      gateway: 198.51.100.1
+    dns:
+      nameservers:
+      - 198.51.100.1
+      - 198.51.100.2
+      domain: testDNS.example
+      search:
+      - testdomain1.example
+      - testdomain2.example
+----
+
+The following example shows an ipam configuration for DHCP:
+
+[source,yaml]
+----
+ipamConfig:
+  type: DHCP
+----
+endif::yaml[]
--- a/modules/nw-multus-ipvlan-object.adoc
+++ b/modules/nw-multus-ipvlan-object.adoc
@@ -0,0 +1,60 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-ipvlan-object_{context}"]
+= Configuration for ipvlan
+
+// TODO - consider refactoring this into its own module as it is used
+// by both ipvlan and bridge CNI plug-ins.
+
+.ipvlan CNI plug-in YAML configuration
+[source,yaml]
+----
+name: <name> <1>
+namespace: <namespace> <2>
+rawCNIConfig: '' <3>
+type: Raw
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition` created from
+the `rawCNIConfig` JSON object.
+
+<2> `namespace`: Specify the namespace to create the network attachment in. If
+a value is not specified, the `default` namespace is used.
+
+<3> `rawCNIConfig`: Specify the CNI plug-in configuration.
+
+////
+TODO -
+
+- `mode` needs an extended discussion; l2 might not work on all cloud providers
+////
+
+.ipvlan CNI plug-in JSON configuration object
+[source,json]
+----
+{
+  "cniVersion": "0.3.1",
+  "name": "<name>", <1>
+  "type": "ipvlan",
+  "mode": "<mode>", <2>
+  "master": "<master>", <3>
+  "mtu": <mtu>, <4>
+  "ipam": { <5>
+    ...
+  }
+}
+----
+<1> `name`: Specify the name of the `NetworkAttachmentDefinition`.
+
+<2> `mode`: Specify the operating mode for the virtual network. The value must
+be `l2`, `l3`, or `l3s`. The default value is `l2`.
+
+<3> `master`: Specify the ethernet interface to associate with the network
+attachment. If a `master` is not specified, the interface for the default
+network route is used.
+
+<4> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The
+default value is automatically chosen by the kernel.
+
+<5> `ipam`: Specify a configuration object for the ipam CNI plug-in. The plug-in
+manages IP address assignment for the attachment definition.
--- a/modules/nw-multus-macvlan-object.adoc
+++ b/modules/nw-multus-macvlan-object.adoc
@@ -0,0 +1,44 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-macvlan-object_{context}"]
+= Configuration for macvlan CNI plug-in
+
+The following YAML describes the configuration parameters for the macvlan
+Container Network Interface (CNI) plug-in:
+
+.macvlan YAML configuration
+[source,yaml]
+----
+name: <name> <1>
+namespace: <namespace> <2>
+type: SimpleMacvlan
+simpleMacvlanConfig: <3>
+  master: <master> <4>
+  mode: <mode> <5>
+  mtu: <mtu> <6>
+  ipamConfig: <7>
+    ...
+----
+<1> `name`: Specify a unique name for the additional network attachment. The
+associated `NetworkAttachmentDefinition` will share this name.
+
+<2> `namespace`: Specify the namespace to create the network attachment in. If
+a value is not specified, the `default` namespace is used.
+
+<3> `simpleMacvlanConfig`: Provide the configuration for the macvlan CNI
+plug-in.
+
+<4> `master`: The ethernet interface to associate with the virtual interface. If
+a value for `master` is not specified, then the host system's primary ethernet
+interface is used.
+
+<5> `mode`: Configures traffic visibility on the virtual network. Must be either
+`bridge`, `passthru`, `private`, or `vepa`. If a value for `mode` is not
+provided, the default value is `bridge`.
+
+<6> `mtu`: Set the maximum transmission unit (MTU) to the specified value. The
+default value is automatically chosen by the kernel.
+
+<7> `ipamConfig`: Specify a configuration object for the ipam CNI plug-in. The
+plug-in manages IP address assignment for the attachment definition.
--- a/modules/nw-multus-remove-pod.adoc
+++ b/modules/nw-multus-remove-pod.adoc
@@ -0,0 +1,66 @@
+// Module included in the following assemblies:
+//
+
+[id="nw-multus-remove-pod_{context}"]
+= Removing a Pod from an additional network
+
+You can remove a Pod from an additional network.
+
+.Prerequisites
+
+* Install the OpenShift Command-line Interface (CLI), commonly known as `oc`.
+* You must log in to the cluster.
+* You must have configured an additional network for your cluster.
+* You must have a Pod attached to an additional network.
+
+.Procedure
+
+To remove a Pod from an additional network, complete the following steps:
+
+. Edit the Pod resource definition by running the following command. Replace
+`<name>` with the name of the Pod to edit.
+
+----
+$ oc edit pod <name>
+----
+
+. Update the `annotations` mapping to remove the additional network from the
+Pod:
+
+* To remove all additional networks from a Pod, remove the
+`k8s.v1.cni.cncf.io/networks` parameter from the Pod resource definition as in
+the following example:
+
+[source,yaml] 
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: example-pod
+  annotations: {}
+spec:
+  containers:
+  - name: example-pod
+    command: ["/bin/bash", "-c", "sleep 2000000000000"]
+    image: centos/tools
+----
+
+* To remove a specific additional network from a Pod, update the
+`k8s.v1.cni.cncf.io/networks` parameter by removing the name of the
+`AdditionalNetworkAttachment` for the additional network. In the following
+example, the additional network `net1` was removed:
+
+[source,yaml]
+----
+apiVersion: v1
+kind: Pod
+metadata:
+  name: example-pod
+  annotations:
+    k8s.v1.cni.cncf.io/networks: net2
+spec:
+  containers:
+  - name: example-pod
+    command: ["/bin/bash", "-c", "sleep 2000000000000"]
+    image: centos/tools
+----
--- a/networking/managing-multinetworking.adoc
+++ b/networking/managing-multinetworking.adoc
@@ -1,82 +0,0 @@
-[id="managing-multiple-networks"]
-= Managing multiple networks
-include::modules/common-attributes.adoc[]
-:context: managing-multiple-networks
-
-toc::[]
-
-== Overview
-
-Multus CNI provides the capability to attach multiple network interfaces to Pods
-in {product-title}. This gives you flexibility when you must configure Pods that
-deliver network functionality, such as switching or routing.
-
-Multus CNI is useful in situations where network isolation is needed, including
-data plane and control plane separation. Isolating network traffic is useful for
-the following performance and security reasons:
-
-Performance:: You can send traffic along two different planes in order to manage
-how much traffic is along each plane.
-Security:: You can send sensitive traffic onto a network plane that is managed
-specifically for security considerations, and you can separate private data that
-must not be shared between tenants or customers.
-
-All of the Pods in the cluster will still use the cluster-wide default network
-to maintain connectivity across the cluster. Every Pod has an `eth0` interface
-which is attached to the cluster-wide Pod network. You can view the interfaces
-for a Pod using the `oc exec -it <pod_name> \-- ip a` command. If you
-add additional network interfaces using Multus CNI, they will be named `net1`,
-`net2`, ..., `netN`.
-
-To attach additional network interfaces to a Pod, you must create configurations
-which define how the interfaces are attached. Each interface is specified using
-a Custom Resource (CR) that has a `NetworkAttachmentDefinition` type. A CNI
-configuration inside each of these CRs defines how that interface will be
-created. Multus CNI is a CNI plug-in that can call other CNI plug-ins. This
-allows the use of other CNI plug-ins to create additional network interfaces.
-For high performance networking, use the SR-IOV Device Plugin with Multus CNI.
-
-Execute the following steps to attach additional network interfaces to Pods:
-
-. Create a CNI configuration as a custom resource.
-. Annotate the Pod with the configuration name.
-. Verify that the attachment was successful by viewing the status annotation.
-
-== CNI configurations
-
-CNI configurations are JSON data with only a single required field, `type`. The
-configuration in the `additional` field is free-form JSON data, which allows CNI
-plug-ins to make the configurations in the form that they require. Different CNI
-plug-ins use different configurations. See the documentation specific to the CNI
-plug-in that you want to use.
-
-An example CNI configuration:
-
-[source,json]
----
-{
-  "cniVersion": "0.3.0", <1>
-  "type": "loopback", <2>
-  "additional": "<plugin-specific-json-data>" <3>
-}
----
-
-<1> `cniVersion`: Specifies the CNI version that is used. The CNI plug-in uses
-this information to check whether it is using a valid version.
-
-<2> `type`: Specifies which CNI plug-in binary to call on disk. In this example,
-the loopback binary is specified, Therefore, it creates a loopback-type network
-interface.
-
-<3> `additional`: The `<information>` value provided in the code above is an
-example. Each CNI plug-in specifies the configuration parameters it needs in
-JSON. These are specific to the CNI plug-in binary that is named in the `type`
-field.
-
-include::modules/nw-multinetwork-creating-first-attachments.adoc[leveloffset=+1]
-include::modules/nw-multinetwork-host-device.adoc[leveloffset=+1]
-
-:FeatureName: SR-IOV multinetwork support
-include::modules/technology-preview.adoc[leveloffset=+1]
-
-include::modules/nw-multinetwork-sriov.adoc[leveloffset=+1]
--- a/networking/multiple-networks/configuring-bridge.adoc
+++ b/networking/multiple-networks/configuring-bridge.adoc
@@ -0,0 +1,22 @@
+[id="configuring-bridge"]
+= Configuring a bridge network
+include::modules/common-attributes.adoc[]
+:context: configuring-bridge
+
+toc::[]
+
+As a cluster administrator, you can configure an additional network for your
+cluster using the bridge Container Network Interface (CNI) plug-in. When
+configured, all Pods on a node are connected to a virtual switch. Each Pod is
+assigned an IP address on the additional network.
+
+// TODO - include specifics of virtual bridge
+
+include::modules/nw-multus-create-network.adoc[leveloffset=+1]
+include::modules/nw-multus-bridge-object.adoc[leveloffset=+2]
+include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
+include::modules/nw-multus-add-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-remove-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-edit-network.adoc[leveloffset=+1]
+include::modules/nw-multus-delete-network.adoc[leveloffset=+1]
+
--- a/networking/multiple-networks/configuring-host-device.adoc
+++ b/networking/multiple-networks/configuring-host-device.adoc
@@ -0,0 +1,19 @@
+[id="configuring-host-device"]
+= Configuring a host-device network
+include::modules/common-attributes.adoc[]
+:context: configuring-host-device
+
+toc::[]
+
+As a cluster administrator, you can configure an additional network for your
+cluster by using the host-device Container Network Interface (CNI) plug-in. The
+plug-in allows you to move the specified network device from the host's network
+namespace into the Pod's network namespace.
+
+include::modules/nw-multus-create-network.adoc[leveloffset=+1]
+include::modules/nw-multus-host-device-object.adoc[leveloffset=+2]
+include::modules/nw-multus-add-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-remove-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-edit-network.adoc[leveloffset=+1]
+include::modules/nw-multus-delete-network.adoc[leveloffset=+1]
+
--- a/networking/multiple-networks/configuring-ipvlan.adoc
+++ b/networking/multiple-networks/configuring-ipvlan.adoc
@@ -0,0 +1,24 @@
+[id="configuring-ipvlan"]
+= Configuring an ipvlan network
+include::modules/common-attributes.adoc[]
+:context: configuring-ipvlan
+
+toc::[]
+
+As a cluster administrator, you can configure an additional network for your
+cluster by using the ipvlan Container Network Interface (CNI) plug-in. The virtual
+network created by this plug-in is associated with a physical interface that you
+specify.
+
+// Each virtual interface shares the same MAC address as the physical interface.
+
+// TODO - Discuss layer 2, layer 3, and layer 3s.
+
+include::modules/nw-multus-create-network.adoc[leveloffset=+1]
+include::modules/nw-multus-ipvlan-object.adoc[leveloffset=+2]
+include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
+include::modules/nw-multus-add-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-remove-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-edit-network.adoc[leveloffset=+1]
+include::modules/nw-multus-delete-network.adoc[leveloffset=+1]
+
--- a/networking/multiple-networks/configuring-macvlan.adoc
+++ b/networking/multiple-networks/configuring-macvlan.adoc
@@ -0,0 +1,26 @@
+[id="configuring-macvlan"]
+= Configuring a macvlan network
+include::modules/common-attributes.adoc[]
+:context: configuring-macvlan
+
+toc::[]
+
+As a cluster administrator, you can configure an additional network for your
+cluster using the macvlan CNI plug-in. When a Pod is attached to the network,
+the plug-in creates a sub-interface from the parent interface on the host. A
+unique hardware mac address is generated for each sub-device.
+
+[IMPORTANT]
+====
+The unique MAC addresses this plug-in generates for sub-interfaces might not be
+compatible with the security polices of your cloud provider.
+====
+
+include::modules/nw-multus-create-network.adoc[leveloffset=+1]
+include::modules/nw-multus-macvlan-object.adoc[leveloffset=+2]
+include::modules/nw-multus-ipam-object.adoc[leveloffset=+2]
+include::modules/nw-multus-add-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-remove-pod.adoc[leveloffset=+1]
+include::modules/nw-multus-edit-network.adoc[leveloffset=+1]
+include::modules/nw-multus-delete-network.adoc[leveloffset=+1]
+
--- a/networking/multiple-networks/configuring-sr-iov.adoc
+++ b/networking/multiple-networks/configuring-sr-iov.adoc
@@ -0,0 +1,12 @@
+[id="configuring-sr-iov"]
+= Configuring an additional network for SR-IOV
+include::modules/common-attributes.adoc[]
+:context: configuring-sr-iov
+
+toc::[]
+
+:FeatureName: SR-IOV multinetwork support
+include::modules/technology-preview.adoc[leveloffset=+1]
+
+include::modules/nw-multinetwork-sriov.adoc[leveloffset=+1]
+
--- a/networking/multiple-networks/understanding-multiple-networks.adoc
+++ b/networking/multiple-networks/understanding-multiple-networks.adoc
@@ -0,0 +1,65 @@
+[id="understanding-multiple-networks"]
+= Understanding multiple networks
+include::modules/common-attributes.adoc[]
+:context: understanding-multiple-networks
+
+toc::[]
+
+In Kubernetes, container networking is delegated to networking plug-ins that
+implement the Container Network Interface (CNI).
+
+{product-title} uses the Multus CNI plug-in to allow chaining of CNI plug-ins.
+During cluster installation, you configure your _default_ Pod network. The
+default network handles all routine network traffic for the cluster. You can
+define an _additional network_ based on the available CNI plug-ins and attach
+one or more of these networks to your Pods. You can define more than one
+additional network for your cluster, depending on your needs. This gives you
+flexibility when you configure Pods that deliver network functionality, such as
+switching or routing.
+
+You can use an additional network in situations where network isolation is
+needed, including data plane and control plane separation. Isolating network
+traffic is useful for the following performance and security reasons:
+
+Performance:: You can send traffic on two different planes in order to manage
+how much traffic is along each plane.
+Security:: You can send sensitive traffic onto a network plane that is managed
+specifically for security considerations, and you can separate private data that
+must not be shared between tenants or customers.
+
+{product-title} provides the following CNI plug-ins for creating additional
+networks in your cluster:
+
+ * *bridge*: xref:../../networking/multiple-networks/configuring-bridge.adoc#configuring-bridge[Creating a bridge-based additional network]
+ allows Pods on the same host to communicate with each other and the host.
+
+ * *macvlan*: xref:../../networking/multiple-networks/configuring-macvlan.adoc#configuring-macvlan[Creating a macvlan-based additional network]
+ allows Pods on a host to communicate with other hosts and Pods on those hosts
+ by using a physical network interface. Each Pod that is attached to a macvlan-based
+ additional network is provided a unique MAC address.
+
+ * *ipvlan*: xref:../../networking/multiple-networks/configuring-ipvlan.adoc#configuring-ipvlan[Creating an ipvlan-based additional network]
+ allows Pods on a host to communicate with other hosts and Pods on those hosts,
+ similar to a macvlan-based additional network. Unlike a macvlan-based
+ additional network, each Pod shares the same MAC address as the parent physical
+ network interface. 
+
+ * *SR-IOV*: xref:../../networking/multiple-networks/configuring-sr-iov.adoc#configuring-sr-iov[Creating a SR-IOV based additional network]
+ allows Pods to attach to a virtual function (VF) interface on SR-IOV capable
+ hardware on the host system.
+
+////
+
+All of the Pods in the cluster still use the cluster-wide default network
+to maintain connectivity across the cluster. Every Pod has an `eth0` interface
+that is attached to the cluster-wide Pod network. You can view the interfaces
+for a Pod by using the `oc exec -it <pod_name> \-- ip a` command. If you
+add additional network interfaces that use Multus CNI, they are named `net1`,
+`net2`, ..., `netN`.
+
+To attach additional network interfaces to a Pod, you must create configurations
+that define how the interfaces are attached. You specify each interface by using
+a Custom Resource (CR) that has a `NetworkAttachmentDefinition` type. A CNI
+configuration inside each of these CRs defines how that interface is created.
+
+////
--- a/welcome/index.adoc
+++ b/welcome/index.adoc
@@ -107,8 +107,8 @@ rotate these certificates.
 the xref:../networking/cluster-network-operator.adoc[Cluster Network Operator] (CNO).
 The CNO uses iptables rules in xref:../networking/openshift-sdn/configuring-kube-proxy.adoc[kube-proxy] to
 direct traffic between nodes and pods running on those nodes.
-The Multis Container Network interface
-adds the capability to attach xref:../networking/managing-multinetworking.adoc#managing-multiple-networks[multiple network interfaces] to a Pod. Using
+The Multis Container Network Interface
+adds the capability to attach xref:../networking/multiple-networks/understanding-multiple-networks.adoc#understanding-multiple-networks[multiple network interfaces] to a Pod. Using
 xref:../networking/configuring-networkpolicy.adoc#configuring-networkpolicy-plugin[NetworkPolicy] features, you can isolate your networks or permit selected traffic.

 - **xref:../storage/understanding-persistent-storage.adoc[Manage storage]**: {product-title} allows cluster administrators to