Why do you need hybrid multi-cluster?
There are several scenarios where hybrid multi-cluster setup could helpful
If your organization stores and processes everything in its private DC, but you would like to setup multiple ingresses for your app. The way to accomplish this would be to deploy k8s clusters across the globe in available regions, and connect these clusters to your on-prem installation. Service mesh will handle service discovery, traffic routing, encryption, etc.
If you are running on-prem infrastructure and don't want to deprecate it just yet, you can save some money on your cloud computing costs.. Simply run your app in your DC, and delegate some ingress handling to the cloud
If you would like to process the data as close as possible to the client, but still have failover. The service mesh can send traffic local-first, and if local endpoints are unavailable it can send traffic to backup endpoints
Hybrid multi-clouds allows you to scale fast, and provides freedom of choice in terms of cloud solutions to run your workloads
Leverage hybrid multi-cluster to migrate to the cloud more smoothly. Migrate apps one by one to save the connectivity between cloud and on-prem installation.If you are in need of various backup, resiliency and other disaster recovery setups
Multiple primary clusters on separate networks
Pros & Cons
Everything listed in ‘Why do you need hybrid multi-cluster’ part
Limited on the blast radius: if something goes wrong with one of the clusters, others could step in to handle the same workloads
Possibility of the same configuration. With tools like GitOps, it's much easier to handle multiple cluster App configurations
Aligned configurations require additional software. To configure multiple hosts, you will need to keep your clusters configuration aligned with tools like Configuration Managers or Ansible. GitOps could help here
You will need software to overview and manage the individual clusters
It becomes more complex to monitor, log and alert solutions when using a multi-cluster system. It would be handy if those were centralized. The procedure to upgrade becomes more complex
What are the options to build a multi-cluster system?
You could build a multi-cluster system in many different ways, but the main idea is to automate the Service Discovery and Traffic Routing.
You could do this yourself using all sorts of network protocols, load-balancers and other tools, or you could leverage Service Mesh.
All the services within Service Mesh will communicate with each other without knowing where their destination is, as Service Mesh will handle the traffic routing.
These are some variants:
- Single Service Mesh control plane and multiple data plances
- Multiple Service Mesh Control Planes for each cluster
- Service Mesh can be installed on top of an existing installation and handle only part of the cluster workloads, leaving the other part as it is
- Service Mesh can be installed as a part of cluster CNI
- Service Mesh can work on top of same L2 subnet (if you already have connectivity in-place)
Organizing clusters in google anthos
Why do you need to choose GKE Anthos?
Anthos covered all the bits and pieces of the multi-cluster setups
You could deploy Anthos cluster on-prem and in cloud (not only GCP)
Clusters could be connected using Anthos Service Mesh
Clusters Apps could be managed with Anthos Configuration Management
Anthos collects metrics and logs using Stackdriver and sends them to the cloud
Anthos cloud console is a single pane of glass for these services
Alternatives & Step-by-step Guide
There are limitations and important configurations you need to take into account when planning your hybrid multi-cluster setup.
Download the blueprint prepared by Akvelon to learn more about:
- important considerations for a hybrid cloud setup
- limitations of Anthos
- the how-to of Anthos GKE setup
- open source alternatives for each of the Anthos features
We’ve also included a step-by-step guide that describes our experience setting up 2 GKE clusters (bare-metal and GCP) with Anthos, so that you could repeat the excercise yourself.