Our very own Head of Technology, Kevin Bowman, on why we're looking to new programming technologies to drive a faster and safer user experience.
Kotlin, Kubernetes, and Kafka; the newest kids on the block are all "K" themed technologies. This trio aren’t exactly not "new", but are definitely on the more recent end of the spectrum. At Sky Betting and Gaming (SBG) we use a variety of technologies - we're far from being a monoculture, our approach is to allow each "tribe" to develop its own technical strategy within sensible principles - and we use them predominantly in our "Nexus" tribe, which is building some of our fundamental future strategic platforms, and uniquely has the luxury of being able to make greenfield choices. With this freedom, we put some of our best technical minds together to architect the most maintainable, sustainable, and operatable setup possible.
The overall design is nothing radical - small decoupled services, communicating via a messaging broker and deployed onto an emerging industry standard hosting platform. However, we've reached a point of maturity in the industry where none of these concepts are nearly as cumbersome as they once were. Go back 5 or 10 years, and there wasn't really a good set of open source technologies which could work together to enable rapid and safe developments in a way which was also a joy to work with. Sure, there were options, but somehow they all seemed to come with their particular costs, whether in time or complexity or just plain headaches to operate in a production environment. More recently, though, open source technologies have reached a joyous conjunction of ease-of-development, ease-of-deployment, and ease-of-operation.
It's notable that the three technologies we're using have all come from companies with impressive technical credentials. Kotlin, Kafka, and Kubernetes, initially from Jetbrains, LinkedIn, and Google respectively, have all been enthusiastically embraced by the open source community such that they will stand on their own feet even if the originating companies lose interest. Earlier this year (https://www.theregister.co.uk/2018/08/30/google_kubernetes_project/), for example, Google began the final step towards a completely independent Kubernetes. Kotlin is now a blessed language for use in developing native Android applications, and Kafka became a full Apache Software Foundation project soon after it was open sourced by LinkedIn.
We've been using Kubernetes in production, in some form or other, for around a year. We chose it as a way to enable rapid workflows by our product engineering teams, who had previously been responsible for maintaining their own virtual servers and had to separately configure the surrounding datacentre components (load balancers, firewalls etc) through various processes (we now have a central team who maintain Kubernetes and its various integrations, whether in AWS or in our private datacentres). Kubernetes was the step-up which we needed to codify all of these into one combined version-controlled definition - something a lot of us had been discussing ways to do in various forms previously, but always with different solutions in different areas. With Kubernetes, however, we are able to use an industry-standard which may be familiar to anyone joining our teams, and can be deployed via the same tooling which we use to deploy our regular software.
Kafka has found a variety of uses around SBG, and again is growing in popularity and finds more use-cases every month as people discover how much easier it is to work with small, decoupled applications. We use it for anything from traditional message queueing (where we would once have used RabbitMQ), to log shipping, to processing and archival of "source-of-truth" events. Again, we have a central team who look after our Kafka platforms on behalf of the product engineering teams, allowing us to develop a centre-of-excellence approach. Using Kafka gives us a great point of visibility into what our applications are doing, and how they are interacting with each other.
Kotlin is a much newer venture for SBG. Our use-case demanded a language with strong type-safety, great performance (which we can achieve with a well-tuned JVM), and a developer experience which would allow our developers to crossover from their Java or NodeJS backgrounds. Whilst tempting to go with traditional Java, we chose Kotlin due to its comparative brevity and embracement of more modern concepts.
Combined, these three technologies are allowing us to develop well-architected applications quickly and safely, and deploy them into an environment which manages the workloads in a highly automated fashion. At SBG, we're passionate about using the right technology for the right job, and with Kotlin, Kubernetes, and Kafka we really feel like we have a great platform for the forseeable future.
Rapid product innovation. Rock-solid stability. Fast platform scaling. Sound like fun?