openshift-docs/modules/nw-multus-advanced-annotations.adoc

// Module included in the following assemblies:
//
// * networking/multiple_networks/attaching-pod.adoc

:_mod-docs-content-type: PROCEDURE
[id="nw-multus-advanced-annotations_{context}"]
= Specifying pod-specific addressing and routing options

[role="_abstract"]
To set static IP addresses, MAC addresses, and default routes for a pod in {product-title}, you can configure pod-specific addressing and routing options using JSON-formatted annotations. With these annotations, you can customize network behavior for individual pods on secondary networks.

.Prerequisites

* The pod must be in the same namespace as the secondary network.
* Install the OpenShift CLI (`oc`).
* You must log in to the cluster.
ifdef::sriov[]
* You must have the SR-IOV Operator installed and a `SriovNetwork` object defined.
endif::sriov[]

.Procedure

. Edit the `Pod` resource definition. If you are editing an existing `Pod` resource, run the
following command to edit its definition in the default editor. Replace `<name>`
with the name of the `Pod` resource to edit.
+
[source,terminal]
----
$ oc edit pod <name>
----

. In the `Pod` resource definition, add the `k8s.v1.cni.cncf.io/networks` parameter to the pod `metadata` mapping. The `k8s.v1.cni.cncf.io/networks` accepts a JSON string of a list of objects that reference the name of `NetworkAttachmentDefinition` custom resource (CR) names in addition to specifying additional properties.
+
[source,yaml]
----
metadata:
  annotations:
    k8s.v1.cni.cncf.io/networks: '[<network>[,<network>,...]]'
# ...
----
+
--
where:

`<network>`:: Replace with a JSON object as shown in the following examples. The single quotes are required.
--
+
In the following example the annotation specifies which network attachment will have the default route, using the `default-route` parameter.
+
[source,yaml]
----
apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
    {
      "name": "net1"
    },
    {
      "name": "net2",
      "default-route": ["192.0.2.1"]
    }]'
spec:
  containers:
  - name: example-pod
    command: ["/bin/bash", "-c", "sleep 2000000000000"]
    image: centos/tools
----
+
--
where:

`name`:: Specifies the name of the secondary network to associate
with the pod.
`default-route`:: Specifies a value of a gateway for traffic to be routed over if no other routing entry is present in the routing table. If more than one `default-route` key is specified, this will cause the pod to fail to become active.
--
+
The default route will cause any traffic that is not specified in other routes to be routed to the gateway.
+
[IMPORTANT]
====
Setting the default route to an interface other than the default network interface for {product-title}
may cause traffic that is anticipated for pod-to-pod traffic to be routed over another interface.
====
+
To verify the routing properties of a pod, the `oc` command may be used to execute the `ip` command within a pod.
+
[source,terminal]
----
$ oc exec -it <pod_name> -- ip route
----
+
[NOTE]
====
You may also reference the pod's `k8s.v1.cni.cncf.io/network-status` to see which secondary network has been
assigned the default route, by the presence of the `default-route` key in the JSON-formatted list of objects.
====
+
To set a static IP address or MAC address for a pod you can use the JSON formatted annotations. This requires you create networks that specifically allow for this functionality. This can be specified in a rawCNIConfig for the CNO.

. Edit the CNO CR by running the following command:
+
[source,terminal]
----
$ oc edit networks.operator.openshift.io cluster
----
+
The following YAML describes the configuration parameters for the CNO:
+
.Cluster Network Operator YAML configuration
[source,terminal,subs="attributes+"]
----
name: <name>
namespace: <namespace>
rawCNIConfig: '{
  ...
}'
type: Raw
----
+
--
where:

`name`:: Specifies a name for the secondary network attachment that you are creating. The name must be unique within the specified `namespace`.
`namespace`:: Specifies the namespace to create the network attachment in. If you do not specify a value, then the `default` namespace is used.
`rawCNIConfig`:: Specifies the CNI plugin configuration in JSON format, which is based on the following template.
--
+
The following object describes the configuration parameters for utilizing static MAC address and IP address using the macvlan CNI plugin:
+
.macvlan CNI plugin JSON configuration object using static IP and MAC address
[source,json]
----
{
  "cniVersion": "0.3.1",
  "name": "<name>",
  "plugins": [{
      "type": "macvlan",
      "capabilities": { "ips": true },
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "static"
      }
    }, {
      "capabilities": { "mac": true },
      "type": "tuning"
    }]
}
----
+
--
where:

`name`:: Specifies the name for the secondary network attachment to create. The name must be unique within the specified `namespace`.
`plugins`:: Specifies an array of CNI plugin configurations. The first object specifies a macvlan plugin configuration and the second object specifies a tuning plugin configuration.
`ips`:: Specifies that a request is made to enable the static IP address functionality of the CNI plugin runtime configuration capabilities.
`master`:: Specifies the interface that the macvlan plugin uses.
`mac`:: Specifies that a request is made to enable the static MAC address functionality of a CNI plugin.
--
+
The above network attachment can be referenced in a JSON formatted annotation, along with keys to specify which static IP and MAC address will be assigned to a given pod.


. Edit the pod by entering the following command:
+
[source,terminal]
----
$ oc edit pod <name>
----
+
.macvlan CNI plugin JSON configuration object using static IP and MAC address
+
[source,yaml]
----
apiVersion: v1
kind: Pod
metadata:
  name: example-pod
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
      {
        "name": "<name>",
        "ips": [ "192.0.2.205/24" ],
        "mac": "CA:FE:C0:FF:EE:00"
      }
    ]'
----
+
--
where:

`name`:: Specifies the name for the secondary network attachment to create. The name must be unique within the specified `namespace`. 
`ips`:: Specifies an IP address including the subnet mask.
`mac`:: Specifies the MAC address.
--
+
[NOTE]
====
Static IP addresses and MAC addresses do not have to be used at the same time. You can use them individually, or together.
====