Automated build and deployment of Docker containerized OSGi applications on Kubernetes

Despite the numerous options available nowadays for build automation of Java applications, Maven is still by far the most popular. The side effect of this is a rich ecosystem of plugins which can be used for automating all kinds of mundane, repetitive tasks.

With simple tools like these, configured once and executed in a matter of seconds during Maven’s install and deploy phases, it is possible to fully automate build and deployment of a modern, cloud native, OSGi application, consisting of tenths of different modules, and specialized sub-systems packaged into separate Docker containers, so one or more instances of each can be deployed onto Kubernetes cluster.

In this tutorial, I will show you exactly how this can be done, sharing code and implementation tips, based on a complete application I put together, available for cloning and deploying yourself here.

The ‘k8-transcription-app‘, ‘k8-linguistics-app‘, ‘k8-web-app‘  and ‘k8-infra‘  modules, in addition to the main project POM, contain all the code referred to in this article.

Let’s get started!

1

The ‘k8-transcription-app’, ‘k8-linguistics-app’ and ‘k8-web-app’ modules are simply specialized sub-systems, which we will package as separate Docker containers, so one or more instances of each can be deployed onto Kubernetes cluster. We want to sync with our Maven build both buildings of these Docker images and pushing those to the registry, with no additional commands to be entered or switching to other tools.

2

I like tools which allow doing easy things quickly and then perhaps also have options for more advanced usage scenarios. I evaluated several Maven plugins for both Docker and Kubernetes automation and the ‘k8-maven-plugin’ is the only plugin which currently fits these criteria. Still, it was missing some of the functionality which I needed to achieve a fully automated solution, therefore I forked and added it and these changes are already merged into the master branch. By the time you’re reading this article, this new version (1.0.5) should already be released to Maven Central, but just in case there’s some delay in pushing this new release, you can clone and install it into your local Maven repository:

   mvn install:install-file -Dfile=target/k8-maven-plugin-1.0.5.jar -DgroupId=com.github.deanjameseverett -DartifactId=k8-maven-plugin -Dversion=1.0.5 -Dpackaging=jar

3

We now include the ‘k8-maven-plugin’ plugin in our Maven build. We start with the main project POM by extracting into variables some of the properties we’ll refer to later, e.g. the Docker registry being used. For local development, we can use local Docker registry –i.e.:

   <properties> 
     (…)
     <k8-maven.docker-registry>192.168.0.53:5000</k8-maven.docker-registry>
     (…)
   </properties>

4

Since we’d like to trigger building of Docker images explicitly, and not have this happen every time we build our application, we create a new Maven profile, and there we configure the plugin, i.e.:

   <profile>
     <id>k8</id>
     <build>
       <pluginManagement>
         <plugins>
           (…)
           <plugin>
             <groupId>com.github.deanjameseverett</groupId>
             <artifactId>k8-maven-plugin</artifactId>
             <version>${k8-maven.version}</version>
             <executions>
               <execution>
                 <id>container</id>
                 <phase>install</phase>
                 <goals>
                   <goal>deleteImage</goal>
                   <goal>buildImage</goal>
                 </goals>
                 <configuration>
                   <dockerRegistry>${k8-maven.docker-registry}</dockerRegistry>
                   <imageName>ala-${project.artifactId}</imageName>
                 </configuration>                   
               </execution>
             (…) 
             </executions>
           </plugin>
         </plugins>
       </pluginManagement>
     </build>
   </profile>

5

For each of the ‘k8-transcription-app’, ‘k8-linguistics-app’ and ‘k8-web-app’ modules, we now add Docker files. The convention to be used for this plugin is to have a Docker file, as well as any other files to be included in the Docker image–aside from the application itself–in the ‘src/main/docker’ directory. Since we can now use Maven variables inside Docker files, instead of hard-coding any of the info, we can simply duplicate same Docker file definition three times, i.e.:

   FROM jeanblanchard/java:8
   COPY @[email protected] /app/[email protected]@.jar
   COPY logback.xml /app
   WORKDIR /app
   ENTRYPOINT ["java","-jar","/app/[email protected]@.jar"]

6

We can now build and push to registry Docker images containing the latest versions of our OSGi applications with one simple command:

mvn -P k8 install

SEE ALSO: New for Kubernetes: Extend cloud-based applications with Kyma

7

Having automated building of Docker images containing the latest versions of our applications and pushing them to the registry, let’s move on to automating Kubernetes deployments now. Since we already have the ‘k8-maven-plugin’ plugin included in our build and referenced in POMs, in the main project POM we extract few more properties into variables to be referred to later; we also specify insecure Docker registry, which is required if using Docker local registry with ‘minikube’, i.e.:

   <properties>
     (…)
     <k8-maven.docker-registry>192.168.0.53:5000</k8-maven.docker-registry>
     <k8-maven.insecureRegistry>${k8-maven.docker-registry}</k8-maven.insecureRegistry>
     <k8-maven.memory>8192</k8-maven.memory>
     <k8-maven.cpus>4</k8-maven.cpus>
     (…)
   </properties>

8

In the same main project POM, we add the cluster execution definition, specifying the ‘startMinikube’ and ‘apply’ goals to be run as part of the deployment phase: the ‘startMinikube’ goal will start ‘minikube’ with ‘insecureRegistry’, ‘memory’ and ‘cpus’ args, as per our definition, while the ‘apply’ goal will afterward run `kubectl apply -f` on any of the Kubernetes descriptors found in the application modules, which we’ll discuss in a moment, i.e.:

   <profile>
     <id>k8</id>
     <build>
       <pluginManagement>
         <plugins>
           (…)
           <plugin>
             <groupId>com.github.deanjameseverett</groupId>
             <artifactId>k8-maven-plugin</artifactId>
             <version>${k8-maven.version}</version>
             <executions>
               (…)
               <execution>
                 <id>cluster</id>
                 <phase>deploy</phase>
                 <goals>
                   <goal>startMinikube</goal>
                   <goal>apply</goal>
                 </goals>
                 <configuration>
                   <insecureRegistry>${k8-maven.insecureRegistry}</insecureRegistry>
                   <memory>${k8-maven.memory}</memory>
                   <cpus>${k8-maven.cpus}</cpus>
                 </configuration>                   
               </execution>
             </executions>
           </plugin>
         </plugins>
       </pluginManagement>
     </build>
   </profile>

9

Since by default the ‘maven-deploy-plugin’ plugin is triggered in deploy Maven lifecycle phase, but we want to deploy to Kubernetes cluster instead, we disable it for this profile, i.e.:

   <profile>
     <id>k8</id>
     <build>
       <pluginManagement>
         <plugins>           
           <plugin>
             <artifactId>maven-deploy-plugin</artifactId>
             <configuration>
               <skip>true</skip>
             </configuration>
           </plugin>
           (…)
         </plugins>
       </pluginManagement>
     </build>
   </profile>

10

Now, for each of the ‘k8-transcription-app’, ‘k8-linguistics-app’ and ‘k8-web-app’ modules, having added Docker files to their ‘src/main/docker’ directories, we now add Kubernetes descriptors to their ‘src/main/k8’ directories. Here, we can also use Maven variables, so instead of hard-coding information we can simply extract it from Maven, thus our descriptors, just like our Docker files, can be re-used unless some special functionality is needed for that particular application, i.e.:

   (…)
   containers: 
   - image: @[email protected]/[email protected]@:@[email protected]
     name: [email protected]@
   (…)

You can view full descriptor for the ‘k8-transcription-app’ module here, for the ‘k8-linguistics-app’ module here, and for the ‘k8-web-app’ module here.

SEE ALSO: Running Kubernetes in production: A million ways to crash your cluster

11

We obviously need infrastructure, in our case a database and a message broker, to be deployed before our applications are deployed. Having Kubernetes descriptor files, we can also automate applying them via the same ‘k8-maven-plugin’ plugin. For this particular set up, I use 2 instances of RabbitMQ and 2 instances of CockroachDB, both as StatefulSets; those I adopted from RabbitMQ’s and CockroachDB’s public repositories. We then apply them during deploy phase having defined them in ‘k8-infra’ module’s POM file, i.e.:

   <profiles>
     <profile>
       <id>k8</id>
       <build>
         <plugins>
           <plugin>
             <groupId>com.github.deanjameseverett</groupId>
             <artifactId>k8-maven-plugin</artifactId>
             <executions>
               <execution>
                 <id>cluster</id>
                 <configuration>
                   <includeFiles>
                     <!-- Set up RabbitMQ -->
                     <param>infra-rabbitmq-statefulset.yaml</param>
                     <!-- Set up CockroachDB -->
                     <param>infra-cockroachdb-statefulset.yaml</param>
                     <param>infra-cockroachdb-cluster-init.yaml</param>
                   </includeFiles>
                 </configuration>                   
               </execution>
             </executions>
           </plugin>
         </plugins>
       </build>
     </profile>
   </profiles>

12

We can now deploy both infrastructure and applications onto the Kubernetes cluster with one simple command:

mvn -P k8 deploy

This will automatically generate the following: