Skip to main content

Deploying an Application

This guide takes you through the happy path: from an empty folder in your team's manifest repository to a running application in GAP, using a Gappynator Application resource.

It focuses on the manifest itself. The surrounding CI/CD setup — the Dockerfile, pushing to ACR, the build workflow and the Argo CD Application — is covered in Deploy Your First App using Gappynator and Argo CD.

Automate this with gap-api

gap-api generates all of the files below and opens the pull requests for you. Use it if you can, and come back here when you need to understand or change what it produced.

Prerequisites​

Before your application can be deployed you need:

  • An onboarded namespace — see Namespace Onboarding
  • A container image in ACR — see Pushing a Container Image
  • Your team's manifest repository, named <your-team>-kubernetes-manifests
  • Your team's gap.io/owner, gap.io/cost-center and gap.io/service-code values

Repository layout​

Manifests live under apps/<app-name>/, with one directory per environment. This is the layout used across the manifest repositories today, and the one Argo CD expects:

apps/my-app/
├── common.jsonnet      # values shared by every environment
├── README.md           # a link back to the source repository
├── test/
│   ├── app.jsonnet     # the test environment
│   └── params.json     # written by CI — never edit by hand
└── prod/
    ├── app.jsonnet     # the prod environment
    └── params.json     # written by CI — never edit by hand

Keep the directory name, the Application name, the image repository name and the source repository name identical. Every team follows this convention and a lot of tooling depends on it.

We use jsonnet for manifests so that the two environments can share values without duplicating the whole resource.

Shared values​

Put everything that does not change between environments in common.jsonnet:

apps/my-app/common.jsonnet
{
  name:: 'my-app',
  namespace:: 'team-example',
  port:: 8080,

  // Replace with your team's values
  labels:: {
    'gap.io/owner': 'team-example',
    'gap.io/cost-center': '19-00000',
    'gap.io/service-code': 'BSN0000000',
  },

  // Key Vault object name -> environment variable name
  secrets:: [
    { name: 'my-app-db-username', envVar: 'DB_USERNAME' },
    { name: 'my-app-db-password', envVar: 'DB_PASSWORD' },
  ],
}

The application manifest​

app.jsonnet builds the Application resource for one environment. Everything not listed here falls back to a Gappynator default.

apps/my-app/test/app.jsonnet
local common = import '../common.jsonnet';
local params = import 'params.json';

local environment = 'test';
local keyVaultName = 'gtm-example-shr-en-test-kv';
local hostSuffix = 'apps-int.testgjensidige.io';

local application = {
  apiVersion: 'gap.io/v1',
  kind: 'Application',
  metadata: {
    name: common.name,
    namespace: common.namespace,
    labels: common.labels + {
      environment: environment,
    } + {
      // Deployment provenance, filled in by CI
      ['github.gap.io/' + key]: params.github[key]
      for key in std.objectFields(params.github)
    },
  },
  spec: {
    image: params.container_image,  // [1]
    port: common.port,

    ingress: {
      host: common.name + '.' + hostSuffix,  // [2]
    },

    resources: {  // [3]
      requests: { cpu: '100m', memory: '384Mi' },
      limits: { cpu: '800m', memory: '512Mi' },
    },

    autoscaling: {  // [4]
      minReplicas: 2,
      maxReplicas: 4,
    },

    env: [
      { name: 'ENVIRONMENT', value: environment },
    ],

    azure: {  // [5]
      secretProviderClass: [
        {
          name: common.name,
          keyVaultName: keyVaultName,
          secrets: common.secrets,
        },
      ],
    },
  },
};

{
  apiVersion: 'v1',
  kind: 'List',
  items: [application],
}
  1. Always take the image from params.json. Never hardcode a tag — CI writes a tag pinned to a digest here after every build.
  2. Setting host is enough — Gappynator creates the route for you. By default this is a Gateway API HTTPRoute.
  3. The defaults are intentionally small. Set values that match your application.
  4. Two to four replicas is the default and a good starting point. Note that setting minReplicas equal to maxReplicas disables autoscaling entirely.
  5. Key Vault secrets become environment variables in your container. Gappynator creates the SecretProviderClass and the managed identity, and keeps the secrets in sync.

The prod variant is the same file with three values changed — the environment, the Key Vault and the host suffix:

apps/my-app/prod/app.jsonnet
local environment = 'prod';
local keyVaultName = 'gpm-example-shr-en-prod-kv';
local hostSuffix = 'apps-int.gjensidige.io';

Deployment parameters​

params.json carries the image and the provenance of the build that produced it. Commit it once with empty values, then leave it alone — the deploy workflow rewrites it on every release.

apps/my-app/test/params.json
{
  "container_image": "",
  "container_image_tag": "",
  "github": {
    "workflow_actor_username": "",
    "repo_name": "",
    "repo_commit_sha": "",
    "manifest_repo_name": "",
    "manifest_repo_commit_sha": ""
  }
}

Because the github block is copied onto the resource as labels, you can always tell which commit of which repository, built by whom, produced what is running in the cluster.

Wire up delivery​

Two things remain, both covered elsewhere:

  1. Add the GAP Workflow Dispatch step to your build workflow so it patches params.json after each successful build — see Deploy Your First App using Gappynator and Argo CD.
  2. Create an Argo CD Application pointing at apps/my-app/test so the cluster is kept in sync — see Deploying with Argo CD.

Verify​

Render the manifest locally before pushing:

jsonnet -J vendor apps/my-app/test/app.jsonnet

Once Argo CD has synced, check the resource and its aggregated status:

kubectl get application my-app -n team-example
kubectl describe application my-app -n team-example

The Ready condition summarises every resource Gappynator owns for your application, and is what Argo CD reports as the health status. If something is wrong, describe names the sub-resource that failed.

# What was actually generated
kubectl get deployment,service,hpa,pdb,servicemonitor -n team-example -l gap.io/name=my-app
Defaults are not visible on the resource

kubectl get application my-app -o yaml shows what you committed, not the effective configuration. To see the defaults that were applied, inspect the generated Deployment.

Commonly added configuration​

The manifest above is deliberately minimal. These are the additions teams make next, roughly in the order they need them.

Health probes. Gappynator does not add probes for you. For a Spring Boot application exposing Actuator on the metrics port — see Runtime-specific configuration for how to expose it:

livenessProbe: {
  httpGet: { path: '/actuator/health/liveness', port: 'metrics' },
  initialDelaySeconds: 45,
  periodSeconds: 10,
  failureThreshold: 3,
},
readinessProbe: {
  httpGet: { path: '/actuator/health/readiness', port: 'metrics' },
  initialDelaySeconds: 45,
  periodSeconds: 10,
  failureThreshold: 3,
},

Network access. Namespaces deny all traffic by default. Use accessPolicy to describe intent and let Gappynator translate it into a Cilium policy:

accessPolicy: {
  outbound: {
    rules: [
      { application: 'other-service', namespace: 'team-other' },
    ],
    external: [
      { host: 'example-redis.redis.cache.windows.net', ports: [{ port: 6380 }] },
    ],
  },
},

See Network Access Policies for the details, including the lower-level networkPolicies alternative.

Configuration files. Mount a ConfigMap, a secret or an Azure file share with filesFrom. The referenced resource must already exist:

filesFrom: [
  {
    configMap: 'my-app-config',
    mountPath: '/application/config/application.yml',
    subPath: 'application.yml',
  },
],

Runtime-specific configuration​

spec.runtime selects which OpenTelemetry auto-instrumentation is injected into your pod. It accepts java, dotnet, python, nodejs, go and other, and defaults to java. It does not affect resource sizing, probes or any other setting.

spec: {
  runtime: 'dotnet',
}

Gappynator always adds a second container port named metrics on 8081, and by default scrapes Prometheus metrics from /actuator/prometheus on that port. Your application has to hold up its end of that contract — the sections below describe what each runtime needs to expose.

Access to metrics must be restricted

End users should never be able to reach your application metrics. Metrics endpoints must only be available to internal systems. Serving them on the separate metrics port, which is never exposed externally, is how we achieve that.

Java and Spring Boot​

Spring Boot is the default runtime and needs no spec.runtime at all. Add the Actuator and Micrometer dependencies:

pom.xml
<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-actuator</artifactId>
  </dependency>
  <dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-core</artifactId>
  </dependency>
  <dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
  </dependency>
</dependencies>

Then expose health probes and metrics on port 8081, and enable graceful shutdown:

src/main/resources/application.yaml
spring:
  application:
    name: my-app # Change this

management:
  endpoints:
    web:
      exposure:
        include: health,prometheus
  endpoint:
    health:
      probes:
        enabled: true
  metrics:
    tags:
      application: ${spring.application.name}
  server:
    port: 8081 # Must match the port Gappynator names "metrics"

server:
  port: 8080
  shutdown: graceful

This gives you the three endpoints that Gappynator's defaults expect:

  • Liveness probe — /actuator/health/liveness on port 8081
  • Readiness probe — /actuator/health/readiness on port 8081
  • Prometheus metrics — /actuator/prometheus on port 8081

Spring Security must permit the Actuator endpoints. Normally you would not use .permitAll() here, but because these are served on port 8081, which is never exposed externally, it is safe:

src/SecurityConfig.kt
@EnableWebSecurity
class SecurityConfig : GjeWebSecurityConfigurerAdapter() {
  override fun authorizeRequests(http: HttpSecurity) {
    http
      .authorizeRequests()
      .requestMatchers(
        EndpointRequest.to("health", "prometheus")
      ).permitAll() /* OK to permit all, these are served on the internal port 8081 */
      .antMatchers(
        HttpMethod.GET, "/v1/your-endpoint/**"
      ).hasAuthority(GjensidigeRole.PRIVATPERSON.value()) /* Always authorize requests */
      .anyRequest().denyAll() /* Deny non permitted requests */
  }
}

Finally, add the gap.io/spring-boot annotation so that Gappynator adds a preStop hook. Together with server.shutdown: graceful above, this lets Tomcat drain in-flight requests and gives you zero-downtime deployments:

metadata: {
  annotations: {
    'gap.io/spring-boot': 'true',
  },
},

.NET​

Set dotnetRuntime to match your base image — linux-musl-x64 for Alpine-based images, linux-x64 otherwise. .NET applications usually expose health on /health rather than the Actuator paths:

spec: {
  runtime: 'dotnet',
  observability: {
    opentelemetry: {
      autoInstrumentation: {
        enabled: true,
        dotnetRuntime: 'linux-musl-x64',
      },
    },
  },
  env: [
    { name: 'ASPNETCORE_URLS', value: 'http://*:8080;http://*:8081' },
  ],
  livenessProbe: { httpGet: { path: '/health', port: 'metrics' } },
  readinessProbe: { httpGet: { path: '/health', port: 'metrics' } },
},

Python, Node.js and Go​

Point observability.prometheusMetrics at wherever your application actually exposes metrics, or disable scraping if it does not expose any:

spec: {
  runtime: 'nodejs',
  observability: {
    prometheusMetrics: {
      enabled: true,
      path: '/metrics',
      port: 'http',
    },
  },
},

Platform-provided mounts​

If your namespace has the relevant addons enabled, Gappynator mounts the following into every pod automatically. You do not declare them in filesFrom — see Namespace Onboarding for how the underlying ConfigMaps are provisioned.

AddonMounted atPurpose
ca-bundle/etc/ssl/certs/ca-certificates.crtGjensidige's internal CA bundle, for TLS to internal services
java-key-store/etc/ssl/certs/java/cacertsThe Java trust store, containing the same internal CAs
jwt-config/mnt/jwt-configJWT properties and signing certificate

Some Java base images keep cacerts somewhere other than the default path. Override it with an annotation rather than a filesFrom entry:

metadata: {
  annotations: {
    'gap.io/cacerts-mount-path': '/opt/java/openjdk/lib/security/cacerts',
  },
},

Applications using the Gjensidige Spring Boot Common Security library also need to point at the mounted JWT configuration:

src/main/resources/application.yaml
gje:
  security:
    tomcat:
      tai-plus:
        jwt-config-location: "file:/mnt/jwt-config/JWTConfig.properties"

Sharing a template across applications​

Once you have more than a couple of applications, the environment files start to repeat themselves. Every team has factored the shared parts into a library under templates/, leaving each app.jsonnet as a handful of overrides:

apps/my-app/test/app.jsonnet
local app = import '../../../templates/application.libsonnet';
local common = import '../common.jsonnet';
local params = import 'params.json';

app {
  name:: common.name,
  environment:: 'test',
  image:: params.container_image,
  github_labels:: params.github,
  secrets:: common.secrets,
}

The template then derives the Key Vault name, the host and the labels from environment. Look at an existing manifest repository for a working example before writing your own.

Next steps​