The now and near future
of data
A short and sweet trends forecast on a software engineer's building blocks
author
Fernando Escobar
Loka, Inc Data Lead
Hi, I'm Fer. For the better part of the last decade I’ve specialized in data analysis and data engineering for Fortune 500s and Series-A Startups. Here’s what I see coming our way in 2021.
When I think about how I’d define data, many things come to mind. It’s facts, it’s observations, it’s behaviors, all put together to be later referenced and analyzed. When data is given a reason, a purpose, and starts answering questions, then it becomes information.
Information can be descriptive (what happened?), and it can be predictive (what will happen?). As a professional in the data engineering and data analysis space, it's effectively the building block for everything I do. Based on what I’ve seen and experienced working with different startups, what their needs are, and what the cloud industry is bringing to the table, I've listed some trends and themes you can expect to see when it comes to these essential electric signals.
Machine Learning
Machine learning, what market analysts have been buzzing about when forecasting data trends for the last five years or so. Even though this is a term that’s been thrown around abundantly over the last couple of years from many startups as a sales pitch, we’re arriving at a point where the benefits are not just theoretical. There are a lot of practical applications. “Practical” is the word I’d like to center on here. From vaccine development (discovering patterns humans would take much longer to find), to farming (finding the right balance of nutrients depending on the different soil and weather conditions), to health (discovering patterns in life behaviors versus longevity), to transportation (hi autonomous driving!), machine learning is being applied in some way to accelerate a product or service. In these examples, machine learning plays a meaningful role, it’s not just a buzzword for empty promises or pipe dreams, but a feature baked in to the core functionality.
Serverless
For data engineering, the main focus for 2021 will be the push towards a serverless approach. By severless, I mean no servers to maintain yourself, but rather servers maintained and scaled by the solution’s provider. This means less time focusing on the underlying machine that hosts your solution, and more time actually developing it. From the realm of AWS, two main serverless services—Aurora Serverless and Lambda—have received major backend performance upgrades. For instance, Aurora Serverless no longer has the “cold boot” issue, and Lambda has a higher timeout than before with a faster scaling for heavy workloads. Just to name a few.
Serverless is a big focus for this year because it’s making it easier for data professionals to start implementing pipelines without a heavy knowledge on devops, which was the main pain point: having servers to maintain, infrastructure to keep updated, healthy and fast at all times.
Serverless makes this automatically. And make no mistake, there’s still servers running your code, your database, your REST API, your message brokers and real-time data streams, but it’s not something you have to worry about, it’s being managed by your cloud service provider, which takes that hassle out of your hands for a small premium. A small premium that allows you to build more efficiently, iterate faster, and deliver a better product overall, a product that can also adapt to more demanding scenarios.
In short, there are two main features of serverless that make it such an enticing (and logical) next step in cloud computing execution models: low to no maintenance and the ability to scale according to your needs.
Infrastructure as code
If you’ve heard about infrastructure as code and haven’t given it enough attention or resources, this year is the time to invest in it! This practice keeps growing and getting more adoption because of how much it simplifies the task of maintaining a whole tech stack. Imagine having your entire stack written out in simple steps as code and being able to replicate it by simply placing that code elsewhere. No need to single provision and/or connect every item in your stack. This can all now be done from lines of code that you can easily carry from one environment to another.
Fernando Escobar
Fernando Escobar is the Data Lead at Loka, Inc, “Fer” as he’s known around here, has extensive expertise in project governance, business development, and data analysis. Our Fortune 500 and Series-A Startup clients best know him for providing excellent data management from collection to analysis, along with insightful recommendations for executives. Our teammates best know Fer for key gaming and headphone reviews.
- More from Loka
Carbon
Climate Change
Technologists want to lower
carbon emissions With even
more code
Open-source tools designed
to spotlight the carbon footprints
of cloud storage
Giants
Machine Learning
Why tech giants should
post their trained
models online
Companies with deep pockets
training deep learning models
should share the wealth
Loka Dystopia
Artificial Intelligence
Loka wants to be the antidote
to dystopia fatigue
Founder Bobby Mukherjee on
the pursuit of actionable,
hopeful near futures
Swipe
Loka's syndication policy
Free and easy
Put simply, we encourage free syndication. If you’re interested in sharing, posting or Tweeting our full articles, or even just a snippet, just reach out to medium@loka.com. We also ask that you attribute Loka, Inc. as the original source. And if you post on the web, please link back to the original content on Loka.com. Pretty straight forward stuff. And a good deal, right? Free content for a link back.
If you want to collaborate on something or have another idea for content, just email me.
We’d love to join forces! 🙌

Silicon Valley Office

350 2nd Street, Suite 8 Los Altos, CA 94022

San Francisco Office

535 Mission St, 14th floor San Francisco, CA 94105