In the world of information-centric programming and systems administration management, one question remains currently unanswered. As software developers work to move database workloads to increasingly cloud-native environments, what do we call these professionals that straddle the central pivot point of the DevOps fulcrum?
Are they data developers, or are they developer data engineers?
Enterprise open source NoSQL database management specialist MongoDB probably uses both terms; the company’s technology works to provide database engineering functionality that developers need to do their job more effectively. Logically then, it employs a lot of developer data engineers, whose job it is to serve data developers with database services.
It’s a tautological twist that MongoDB doesn’t lose sleep over; the company has now combined an appreciation for backend services with frontend programmer needs by unveiling its Developer Data Platform vision.
This expanded approach was presented this month at its annual conference, MongoDB World, at the Javits Center in New York City.
With these updates, MongoDB insists it is empowering software development teams to innovate faster by addressing a wider set of use cases, servicing more of the data lifecycle, optimising for modern architectures and implementing the most sophisticated levels of data encryption, all within a single integrated developer data platform.
Elegant Developer Experiences
“Hundreds of millions of new applications will be developed over the coming years that increase operational efficiency via more sophisticated automation – and these applications all require a highly scalable, cloud-native, globally distributed data platform,” said Dev Ittycheria, President and CEO of MongoDB.
It’s all part of a developer data platform vision that Ittycheria says is designed to provide ‘a modern and elegant’ developer experience. If that sounds like a strange expression, it really shouldn’t as this is technology built to serve developers with functions to reduce the complexity of the data infrastructure required to support modern applications.
“[That modern and elegant developer experience] will enable developers with broad support for a wide variety of use cases and delivers the performance and scalability needed to address the most demanding requirements,” added Ittycheria.
Beyond the Bread and Butter
MongoDB explains its broader approach to working with data in direct terms. It is now going beyond giving developers data infrastructure services to build operational and transactional use cases, i.e., some of the more fundamental basic functions.
The beyond element here comes down to how MongoDB has now extended its platform to serve search and analytics use cases, all within a unified platform. Further and specifically here, MongoDB has also announced a number of capabilities that make it easier for developers to build in-app analytics and power richer application experiences.
“Column store indexing, available later this year, will enable users to create and maintain a purpose-built index that dramatically speeds up many common analytical queries without requiring any changes to the document structure or having to move data to another system.
“Furthermore, analytics nodes can now be scaled separately, allowing teams to independently tune the performance of their operational and analytical queries without over- or under-provisioning,” explained the company in a technical product statement.
We can also point to core elements of the MongoDB stack here such as MongoDB Atlas. This is the company’s cloud-native document DataBase as a Service (DBaaS) that works with any cloud provider service to take data infrastructure chores away from developers.
MongoDB Atlas automates database resources infrastructure provisioning, setup and deployment so that – in working practice inside live production environments – patches and minor version upgrades are applied automatically.
This means that when data developer teams need to modify a cluster (perhaps when they want to scale out an application service or perform an upgrade) MongoDB Atlas lets them do so, in a few clicks with no downtime required.
Newly updated in this area is Cluster-to-Cluster (C2C) Synchronization functionality. This provides the continuous data synchronization of MongoDB clusters across environments whether in Atlas, in private cloud, on-premises, or on the edge.
In terms of actual use, C2C Synchronization allows users to migrate data to the cloud, create test environments, create dedicated analytics environments and support data residency requirements.
“The ability to leverage C2C Synchronization for our many existing MongoDB-based travel applications is something we’ve been wanting for a long time and will be a huge benefit to us.
“It will greatly improve many facets of our software lifecycle, such as supporting ‘blue/green’ deployments, data distribution, cloud migration and further increasing our high levels of geographic availability for our airline customers,” said Sylvain Roy, Senior Vice-President, technology platforms & engineering, Amadeus.
Data for Domain Experts
Where we go next will most likely involve not just data developers and developer data engineers, but also citizen data scientists. Where organisations have domain experts that need increasing degrees of data science functionality that is custom-aligned to specific needs and delivered more quickly than the data developer function can build it, we see citizen data scientists coming into view.
Dominic Wellington, MongoDB’s Director of Market Intelligence, has illustrated the intersection point that arises here and clarified why this reality is inevitable, but needs to be approached with caution.
If we consider a typical company data stack, data relating to different product lines might be variously distributed across different IT systems, explains Wellington. Historically, we would need a software engineer with experience across each system to bring everything together. In the past, this might often lead to logjams and dead-ends.
Today’s software is more flexible says Wellington as it allows users to work with data without the need for deep technical knowledge. So as MongoDB services start to dovetail with some of the newer low-code/no-code tools now surfacing, we still need to keep control over the health and wealth of our data.
“Business experts might not know the best practices that are second nature to software programmers and data professionals. They may not think about versioning and access control until it is too late. As the power of software becomes more accessible, we must connect with these new participants and give them the benefit of our encounters, but we must also put the support in place to manage what is created,” said Wellington in a company blog.
Data Workloads Move to Cloud
The justification for many of these technology developments from MongoDB hinges around the fact that databases are moving to the cloud. Analysts suggest that 2020 was the watershed moment when we passed the 50% mark measure for global corporate data in the cloud. It’s a figure that rose to 60% in 2022.
“Roughly 90% of enterprises are either actively moving production data to the cloud and adopting cloud databases for new workloads or have plans to do so within the next three years,” said Carl Olofson, Research Vice-President, IDC Research, in a report entitled Market Analysis Perspective: Worldwide Database Management Systems Software Market, September 2021.
MongoDB has built out a roadmap designed to provide its breed of data infrastructure tools for cloud environments with arguably rather prescient vision to anticipate the rise in DBaaS.
The company has built towards this point and the groundswell of additional cloud-backbones engaged throughout the pandemic won’t have hurt either.
Overall, MongoDB is one of those companies that pitches itself as geek-chic developer first and data-centric from the heart.
Originally known as 10gen, MongoDB took its name from the word humongous, somebody must have been thinking about growth and scale when they chose that. The next big thing might one day be the next humongous thing.