Data Structures vs. Functions in the Age of Microservices

The old wisdom of “data structures over functions” has stood the test of time for probably 50 years now. It states that long-term, a system is better built on sound data structures than functions. While functions may hide clumsy data structures for a while, when faced with evolution and new user needs, poor data structures will come back to haunt you and will eventually bring down the system.

One of the more interesting aspects of microservices (by most definitions) is that they make no attempt at data integration on a database-level. Rather, each microservice manages its own data. Data integration then requires the microservices to sync up the distributed data. Still, part of the old wisdom of data structures over functions is that data integration, if possible, is best done on a database level and not by custom code.

Thus, microservices are clearly about functions (at large). If the old truth holds, the question becomes: Will they survive for long?

The answer I usually get to the question about data integration is that (a) your libraries should not require that all microservices be (re)deployed at once and hence that (b) your continuous deployment pipeline should support independent deployment of microservices. Thus, you won’t end up with one large monolith (the combined microservices system) but rather many smaller ones (the individual microservices).

I don’t think that independent deployment, though important, will save microservices (as we know them today) as a long-term approach. My money is on sound data structures and data integration using a database. That database may not look like the databases we know today, but it will be there, replacing the data layers of the microservices being built today and in the near future. The value of microservices will have been to have the integration problem explored at scale, before databases came back and took it from them.

Leave a Reply