Service Preview Reference

The following is a reference for the various components of Service Preview.

See Service Preview Quick Start for detailed installation instructions.

Traffic Manager

The Traffic Manager is the central point of communication between Traffic Agents in the cluster and Edge Control Daemons on developer workstations.

The following YAML is the basic Traffic Manager installation manifests that is available for download at https://getambassador.io/yaml/traffic-manager.yaml.

# This is traffic-manager.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traffic-manager
  namespace: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: traffic-manager
rules:
  - apiGroups: [""]
    resources: ["namespaces", "services", "pods", "secrets"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: traffic-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traffic-manager
subjects:
  - kind: ServiceAccount
    name: traffic-manager
    namespace: ambassador
---
apiVersion: v1
kind: Service
metadata:
  name: telepresence-proxy
  namespace: ambassador
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app: telepresence-proxy
  ports:
    - name: sshd
      protocol: TCP
      port: 8022
    - name: api
      protocol: TCP
      port: 8081
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: telepresence-proxy
  namespace: ambassador
  labels:
    app: telepresence-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: telepresence-proxy
  template:
    metadata:
      annotations:
        consul.hashicorp.com/connect-inject: 'false'
        sidecar.istio.io/inject: 'false'
      labels:
        app: telepresence-proxy
    spec:
      containers:
      - name: telepresence-proxy
        image: docker.io/datawire/aes:1.8.1
        command: [ "traffic-manager" ]
        ports:
          - name: sshd
            containerPort: 8022
        env:
          - name: AMBASSADOR_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: REDIS_URL
            value: ambassador-redis:6379
        volumeMounts:
          - mountPath: /tmp/ambassador-pod-info
            name: ambassador-pod-info
      restartPolicy: Always
      serviceAccountName: traffic-manager
      terminationGracePeriodSeconds: 0
      volumes:
      - downwardAPI:
          items:
          - fieldRef:
              fieldPath: metadata.labels
            path: labels
        name: ambassador-pod-info

The Traffic Manager needs to be able to watch resources in the cluster so it is aware of what services are interceptable by Service Preview. The default is to provide a cluster-wide scope for this as shown above so you can run Service Preview in any namespace.

It also requires the ability to read your Ambassador Edge Stack license key from the ambassador-edge-stack Secret.

Traffic Manager Options

Remove permission to read Secrets

If you do not wish to grant read privileges on Secrets to the traffic-manager ServiceAccount, you may mount the ambassador-edge-stack secret containing the license key in an extra volume and reference it using the AMBASSADOR_LICENSE_FILE environment variable:

# [...]
    env:
    - name: AMBASSADOR_LICENSE_FILE
      value: /.config/ambassador/license-key
    # [...]
    volumeMounts:
    - mountPath: /.config/ambassador
      name: ambassador-edge-stack-secrets
      readOnly: true
  # [...]
  volumes:
  - name: ambassador-edge-stack-secrets
    secret:
      secretName: ambassador-edge-stack

Run with namespace scope

You can run the Traffic Agent without cluster-wide permissions if you only want to use service preview in a single namespace.

To do so, you will need use the following manifest which modifies the deployment to run only in the ambassador namespace.

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traffic-manager
  namespace: ambassador
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: traffic-manager
rules:
  - apiGroups: [""]
    resources: ["namespaces", "services", "pods", "secrets"]
    verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: traffic-manager
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: traffic-manager
subjects:
  - kind: ServiceAccount
    name: traffic-manager
    namespace: ambassador
---
apiVersion: v1
kind: Service
metadata:
  name: telepresence-proxy
  namespace: ambassador
spec:
  type: ClusterIP
  clusterIP: None
  selector:
    app: telepresence-proxy
  ports:
    - name: sshd
      protocol: TCP
      port: 8022
    - name: api
      protocol: TCP
      port: 8081
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: telepresence-proxy
  namespace: ambassador
  labels:
    app: telepresence-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      app: telepresence-proxy
  template:
    metadata:
      annotations:
        consul.hashicorp.com/connect-inject: 'false'
        sidecar.istio.io/inject: 'false'
      labels:
        app: telepresence-proxy
    spec:
      containers:
      - name: telepresence-proxy
        image: docker.io/datawire/aes:1.8.1
        command: [ "traffic-manager" ]
        ports:
          - name: sshd
            containerPort: 8022
        env:
          - name: AMBASSADOR_SINGLE_NAMESPACE
            value: "true"
          - name: AMBASSADOR_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: REDIS_URL
            value: ambassador-redis:6379
        volumeMounts:
          - mountPath: /tmp/ambassador-pod-info
            name: ambassador-pod-info
      restartPolicy: Always
      serviceAccountName: traffic-manager
      terminationGracePeriodSeconds: 0
      volumes:
      - downwardAPI:
          items:
          - fieldRef:
              fieldPath: metadata.labels
            path: labels
        name: ambassador-pod-info

Traffic Agent

Any pod running in a cluster with a Traffic Manager can opt in to intercept functionality by including the Traffic Agent container.

Configuring RBAC

Since the Traffic Agent is built on Ambassador Edge Stack, it needs a subset of the same RBAC permissions that Ambassador does. The easiest way to provide this is to create a ServiceAccount in your service's namespace, bound to the traffic-agent Role or ClusterRole.

The following YAML is the basic Traffic Agent RBAC configuration manifests that is available for download at https://getambassador.io/yaml/traffic-agent-rbac.yaml.

# This is traffic-agent-rbac.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traffic-agent
  namespace: default
  labels:
    product: aes
---
## After creating the ServiceAccount, create a service-account-token for traffic-agent with a matching name.
## Since the ambassador-injector will use this token name, it must be deterministic and not auto-generated.
apiVersion: v1
kind: Secret
metadata:
  name: traffic-agent
  namespace: default
  annotations:
    kubernetes.io/service-account.name: traffic-agent
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: traffic-agent
rules:
  - apiGroups: [""]
    resources: [ "namespaces", "services", "secrets" ]
    verbs: ["get", "list", "watch"]
  - apiGroups: [ "getambassador.io" ]
    resources: [ "*" ]
    verbs: ["get", "list", "watch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: traffic-agent
  labels:
    product: aes
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traffic-agent
subjects:
  - name: traffic-agent
    namespace: default
    kind: ServiceAccount

If you want to include the Traffic Agent with multiple services, they can all use the same ServiceAccount name, as long as it exists in every namespace.

RBAC Options

Run with namespace scope

You can reduce the scope of the Traffic Agent if you only want to run Service Preview in a single namespace.

To do so, create the following RBAC roles instead of the cluster-scoped ones above:

# This is traffic-agent-rbac.yaml
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: traffic-agent
  namespace: default
  labels:
    product: aes
---
## After creating the ServiceAccount, create a service-account-token for traffic-agent with a matching name.
## Since the ambassador-injector will use this token name, it must be deterministic and not auto-generated.
apiVersion: v1
kind: Secret
metadata:
  name: traffic-agent
  namespace: default
  annotations:
    kubernetes.io/service-account.name: traffic-agent
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: traffic-agent
rules:
  - apiGroups: [""]
    resources: [ "namespaces", "services", "secrets" ]
    verbs: ["get", "list", "watch"]
  - apiGroups: [ "getambassador.io" ]
    resources: [ "*" ]
    verbs: ["get", "list", "watch", "update"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: traffic-agent
  labels:
    product: aes
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: traffic-agent
subjects:
  - name: traffic-agent
    namespace: default
    kind: ServiceAccount

Give permission to all ServiceAccounts in the Cluster

Alternatively, if you already have specific ServiceAccounts defined for each of your pod, you may grant all of them the additional traffic-agent permissions:

---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: traffic-agent
  labels:
    product: aes
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traffic-agent
subjects:
  - name: system:serviceaccounts
    kind: Group
    apiGroup: rbac.authorization.k8s.io

Automatic Traffic Agent Sidecar Injection with Ambassador Injector

The Ambassador Injector automatically injects the Traffic Agent sidecar into services that you want to use Service Preview with.

It does this with a Mutating Admission Webhook that runs when pods are created in the cluster.

The Ambassador Injector can be installed in your cluster in the Ambassador namespace from this YAML manifest: https://getambassador.io/yaml/ambassador-injector.yaml.

This works well for most usecase but there are a couple of important points to make sure the Ambassador Injector is able to function properly.

TLS certificates are required for the Ambassador Injector to authenticate with Kubernetes. The ambassador-injector.yaml provides some default certificates that can be used in development environments but this should be replaced when running in production.

The port the container is listening on must be defined in the Pod template. The Injector will automatically detect container ports with the name http or https and use those ports to know how to route to the container.

spec:
  containers:                   # Application container
    - name: hello
      image: docker.io/datawire/hello-world:latest
      ports:
        - name: http
          containerPort: 8000   # Application port

Take a look at the following for a more detailed look at what is included in https://getambassador.io/yaml/ambassador-injector.yaml:

# This is ambassador-injector.yaml
---
kind: Secret
apiVersion: v1
metadata:
  name: ambassador-injector-tls
  namespace: ambassador
type: Opaque
data:
  crt.pem: $CRT_PEM_BASE64$
  key.pem: $KEY_PEM_BASE64$
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: ambassador-injector
  namespace: ambassador
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ambassador-injector
      app.kubernetes.io/instance: ambassador
  template:
    metadata:
      annotations:
        consul.hashicorp.com/connect-inject: 'false'
        sidecar.istio.io/inject: 'false'
      labels:
        app.kubernetes.io/name: ambassador-injector
        app.kubernetes.io/instance: ambassador
    spec:
      containers:
        - name: webhook
          image: docker.io/datawire/aes:1.8.1
          command: [ "aes-injector" ]
          env:
            - name: TRAFFIC_AGENT_IMAGE                # Mandatory. The Traffic Agent is included in the AES image.
              value: docker.io/datawire/aes:1.8.1
            - name: TRAFFIC_AGENT_SERVICE_ACCOUNT_NAME # Optional. The Injector can configure the sidecar to use a specific ServiceAccount and service-account-token. if unspecified, the original Pod ServiceAccount is used.
              value: traffic-agent
            - name: TRAFFIC_AGENT_AGENT_LISTEN_PORT    # Optional. The port on which the Traffic Agent will listen. Defaults to "9900".
              value: "9900"
            - name: AGENT_MANAGER_NAMESPACE            # Optional. Namespace for contacting the Traffic Manager. Defaults to "ambassador".
              value: ambassador
          ports:
            - containerPort: 8443
              name: https
          livenessProbe:
            httpGet:
              path: /healthz
              port: https
              scheme: HTTPS
          volumeMounts:
            - mountPath: /var/run/secrets/tls
              name: tls
              readOnly: true
      volumes:
        - name: tls
          secret:
            secretName: ambassador-injector-tls
---
apiVersion: v1
kind: Service
metadata:
  name: ambassador-injector
  namespace: ambassador
spec:
  type: ClusterIP
  selector:
    app.kubernetes.io/name: ambassador-injector
    app.kubernetes.io/instance: ambassador
  ports:
    - name: ambassador-injector
      port: 443
      targetPort: https
---
apiVersion: admissionregistration.k8s.io/v1beta1
kind: MutatingWebhookConfiguration
metadata:
  name: ambassador-injector-webhook-config
webhooks:
  - name: ambassador-injector.getambassador.io
    clientConfig:
      service:
        name: ambassador-injector
        namespace: ambassador
        path: "/traffic-agent"
      caBundle: $CA_BUNDLE_BASE64$
    failurePolicy: Ignore
    rules:
      - operations: ["CREATE"]
        apiGroups: [""]
        apiVersions: ["v1"]
        resources: ["pods"]

Manual Traffic Agent Sidecar Configuration

Each service that you want to work with Service Preview requires the Traffic Agent sidecar. This is typically managed by the Ambassador Injector.

The following is information on how to manually configure the Traffic Agent as a sidecar to your service.

You'll need to modify the YAML for each microservice to include the Traffic Agent. We'll start with a set of manifests for a simple microservice:

# This is hello.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: hello
  namespace: default
  labels:
    app: hello
spec:
  selector:
    app: hello
  ports:
    - protocol: TCP
      port: 80
      targetPort: http              # Application port
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  namespace: default
  labels:
    app: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      labels:
        app: hello
    spec:
      containers:                   # Application container
        - name: hello
          image: docker.io/datawire/hello-world:latest
          ports:
            - name: http
              containerPort: 8000   # Application port

In order to run the sidecar:

you need to include the Traffic Agent container in the microservice pod;
you need to switch the microservice's Service definition to point to the Traffic Agent's listening port (using named ports such as http or https allow us to change the Pod definition without changing the Service definition); and
you need to tell the Traffic Agent how to set up for the microservice, using environment variables.

Here is a modified set of manifests that includes the Traffic Agent (with notes below):

# This is hello-intercept.yaml
---
apiVersion: v1
kind: Service
metadata:
  name: hello
  namespace: default
  labels:
    app: hello
spec:
  selector:
    app: hello
  ports:
    - protocol: TCP
      port: 80
      targetPort: http              # Traffic Agent listen port (note 1)
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: hello
  namespace: default
  labels:
    app: hello
spec:
  replicas: 1
  selector:
    matchLabels:
      app: hello
  template:
    metadata:
      annotations:
        consul.hashicorp.com/connect-inject: 'false'
        sidecar.istio.io/inject: 'false'
      labels:
        app: hello
    spec:
      containers:
        - name: hello               # Application container (note 2)
          image: docker.io/datawire/hello-world:latest
          ports:
            - containerPort: 8000   # Application port
        - name: traffic-agent       # Traffic Agent container (note 3)
          image: docker.io/datawire/aes:1.8.1 # (note 4)
          ports:
            - name: http
              containerPort: 9900   # Traffic Agent listen port
          env:
          - name: AGENT_SERVICE     # Name to use for intercepting (note 5)
            value: hello
          - name: AGENT_PORT        # Port on which to talk to the microservice (note 6)
            value: "8000"
          - name: AGENT_MANAGER_NAMESPACE # Namespace for contacting the Traffic Manager (note 7)
            value: ambassador
          - name: AMBASSADOR_NAMESPACE # Namespace in which this microservice is running (note 8)
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
          - name: AMBASSADOR_SINGLE_NAMESPACE # Traffic Agent container can run in a single-namespace (note 9)
            value: "true"
          - name: AMBASSADOR_ENVOY_BASE_ID  # Allows Traffic Agent to run alongside other Envoy proxies
            value: "1"
          - name: AGENT_LISTEN_PORT # Port on which to listen for connections (note 10)
            value: "9900"
      serviceAccountName: traffic-agent # The pod runs with traffic-agent RBAC

Key points include:

Note 1: The Service now points to the Traffic Agent’s listen port (named http, 9900) instead of the application’s port (8000).
Note 2: The microservice's application container is actually unchanged.
Note 3: The Traffic Agent's container has been added.
Note 4: The Traffic Agent is included in the AES image.
Note 5: The AGENT_SERVICE environment variable is mandatory. It sets the name that the Traffic Agent will report to the Traffic Manager for this microservice: you will have to provide this name to intercept this microservice.
Note 6: The AGENT_PORT environment variable is mandatory. It tells the Traffic Agent the local port on which the microservice is listening.
Note 7: The AGENT_MANAGER_NAMESPACE environment variable tells the Traffic Agent the namespace in which it will be able to find the Traffic Manager. If not present, it defaults to the ambassador namespace.
Note 8: The AMBASSADOR_NAMESPACE environment variable is mandatory. It lets the Traffic Agent tell the Traffic Manager the namespace in which the microservice is running.
Note 9: The AMBASSADOR_SINGLE_NAMESPACE environment variable tells the Traffic Agent to watch resources only in its current namespace. This allows the traffic-agent ServiceAccount to only have Role permissions instead of a cluster-wide ClusterRole.
Note 10: The AGENT_LISTEN_PORT environment variable tells the Traffic Agent the port on which to listen for incoming connections. The Service must point to this port (see Note 1). If not present, it defaults to port 9900.

gRPC Support

The Traffic-Agent can inspect and intercept gRPC traffic for deployments exposing gRPC endpoints instead of plain HTTP:

Set the getambassador.io/inject-traffic-agent-grpc: "true" pod annotation, or the AGENT_ENABLE_GRPC: "true" environment variable if injecting the sidecar manually.
The Traffic Agent will instruct Envoy to use HTTP/2 on its listen port (named grpc instead of http; 9900 by default)

TLS Support

If other microservices in the cluster expect to speak TLS to this microservice, tell the Traffic Agent to terminate TLS:

Set the getambassador.io/inject-terminating-tls-secret pod annotation, or the AGENT_TLS_TERM_SECRET environment variable if injecting the sidecar manually, to the name of a Kubernetes Secret that contains a TLS certificate
The Traffic Agent will terminate TLS on its listen port (named https instead of http; 9900 by default) using the named certificate
The Traffic Agent will not accept cleartext communication when configured to terminate TLS

If this microservice expects incoming requests to speak TLS, tell the Traffic Agent to originate TLS:

Set the getambassador.io/inject-originating-tls-secret pod annotation, or the AGENT_TLS_ORIG_SECRET environment variable if injecting the sidecar manually, to the name of a Kubernetes Secret that contains a TLS certificate
The Traffic Agent will use that certificate originate HTTPS requests to the application

Ambassador Edge Stack

To enable Preview URLs, you must first enable preview URL processing in one or more Host resources. Ambassador Edge Stack uses Host resources to configure various aspects of a given host. Enabling preview URLs is as simple as adding the previewUrl section and setting enabled to true:

# This is minimal-host-preview-url.yaml
apiVersion: getambassador.io/v2
kind: Host
metadata:
  name: minimal-host
spec:
  hostname: host.example.com
  acmeProvider:
    email: julian@example.com
  previewUrl:
    enabled: true
    type: Path

Note: If you already had an active Edge Control Daemon connection to the cluster, you must reconnect to the cluster for the Edge Control Daemon to detect the change to the Host resource. This limitation will be removed in the future.

What's Next?

See how Edge Control commands can be used in action to establish outbound connectivity with a remote Kubernetes cluster and intercept inbound requests.

Questions?

We’re here to help. If you have questions, join our Slack or contact us.