The Rise Of Platform Engineering, Kubernetes Data Plane Services

In the cloud, infrastructure is everything. If there’s one theme coming out of the cloud computing industry right now, it’s the need to provide controls that manage more complex infrastructure base layer functions. As we build the new era of cloud computing, we are now moving towards technologies that enable us to change the way the very core of the cloud backbone actually works.

Development of technologies such as eBPF and Cilium give rise to a new era of so-called ‘platform teams’ who do not just program software development tasks or take care of operations, they perform platform engineering functions to change the very instrumentation of cloud-native systems.

Self-service planning

Parallel to this renewed focus on the base layer (spoiler alert, the cloud is not above us, it is below us in the data center and in the inner core we find storage, network and infrastructure services) are attempts to provide self-service technologies, i.e. features and services that engineers need, but which is now delivered -as-a-Service in itself within the cloud layer.

So what is platform engineering, who does it and what does it entail?

This sector of information technology is focused on the ability to adjust, develop and change the way the data core works in a given system – we talk about serverless computing and automated provisioning – we talk about programmable infrastructures and Infrastructure-as-Code (IaC) and we talk on creating development environments for software applications that are container-ready and adapted to the benefits of Kubernetes orchestration.

Mountain View-headquartered Pure Storage is working on this lower substrate layer with cloud enablement and enlightenment.

Kubernetes-ready compute plane

The company has now come up with a new fully managed service for Portworx Enterprise (a multi-cloud-ready Kubernetes data services and storage platform with elastic scalability) to bring a Kubernetes-ready data plane (the part of any network where information is located, transferred from and thus becomes “traffic”, part of the management plane and control plane triumvirate) to each developer working on containerized applications.

With Portworx Enterprise 3.0 as the underlying platform for this managed service (note that Portworx was acquired by Pure Storage in September 2020 and now operates as the cloud-based business unit within Pure Storage), DevOps teams are said to be able to run mission-critical Kubernetes apps in production with good scalability and “unsurpassed” availability ratings.

Now the entire suite of Portworx offerings can be consumed as a fully managed service by users of Amazon EKS (Elastic Kubernetes Service), RedHat OpenShift, and all other Kubernetes services (fully managed or upstream deployments).

… and again, in plain English

To try to put all of this in simple language (just in case the CEO or CMO asks), this is a technology to provide software developers with an orchestrated cloud container data service, as-a-Service. So then, an arguably a key platform technology accelerator to get high performance cloud app infrastructures up and running more easily.

“The mission of Portworx has always been to help platform engineering teams provide their developers with an enterprise-class Kubernetes-ready computing platform with speed, simplicity and scale. By delivering the fully managed service for the Portworx Kubernetes computing platform, we bring the cloud experience, on any storage infrastructure, at the fingertips of any developer looking to work with Kubernetes apps in production,” said Murli Thirumale, VP & GM of the cloud business unit, Pure Storage.

If we remove this development again, we can say that this is a managed service for deploying Kubernetes data on any cloud.

By any cloud we mean (obviously) local private cloud, public cloud services and hybrid bridge services that cross both deployment models. This is meant to allow developers, but perhaps more specifically DevOps teams who want to span both software engineering and sysadmin/database etc. operational responsibilities, to run and scale containerized cloud apps into production – and it all happens in seconds, says Pure.

“Red Hat OpenShift is the industry’s leading enterprise Kubernetes platform, providing a consistent foundation for developers to build, deploy, manage and scale containerized applications faster across hybrid and multi-cloud environments all the way to the edge. Portworx data management and data protection offerings help to unlock key use cases for Red Hat OpenShift workloads in production,” said Chris Gray, vice president, North America Partner Ecosystem, Red Hat.

Gray suggests this latest managed cloud service offering from Portworx by Pure Storage addresses increased customer interest in building a single platform to offer storage, migration and data management-as-a-service to developers. In this case (or the ones he’s most interested in), on top of Red Hat OpenShift.

Love low latency

With the 3.0 release of Portworx Enterprise, Pure Storage says Portworx has a new PX-Fast feature that provides performance for low-latency data services such as Kafka, Elastic and MongoDB that need fast ingestion. It is also said to suit high throughput for online transaction processing (OLTP), online analytical processing (OLAP) and machine learning (ML) workloads.

PX-Fast makes Kubernetes data storage look like local storage for containerized apps and data, Pure says. Customers are able to harness the power of petabytes of data generated from apps like video-on-demand, VoIP, etc. with a near-zero error rate.

What we have here then is platform engineering, but with a definite and defined bias towards Kubernetes and its data plan requirements.

Data storage technologies sometimes get a bad press among technologists who relegate this level of the modern IT stack to the domain of infrastructure provisioning grease monkeys that fill the Ops operations quotient of the DevOps mix.

But those times may be about to change. Largely because this is the era of infrastructure, this is the era of cloud backbones, and this is the era of user interface requirements for upper-level applications that require an always-on data pipeline to satisfy modern usage demands.

Oh no, someone said… anyone at all, WhatsApp is down for six minutes, Gmail went down for a whole hour yesterday and Facebook looks flaky right now, what’s going on? Perhaps we should all care about the platform engineering intelligence, automation and acceleration that takes place under just that little bit more.

You can even “like” storage, after all it’s complicated.