If you have a comment or are interested in sponsoring, hit reply.
When I started my first software job, I remember thinking two things:
-
I am definitely not the smartest person in the room
-
The engineers who seem smarter are people who create, explain, and follow process that makes their work predictable and understandable
I seek out teams where I am not the smartest person in the room, and I’m still trying to apply software engineering principles to the work that I do so that at some point I can look back and say “I’m a professional engineer.”
Today I’m a data plumber. Diagnosing an issue, adjusting an existing automation, or creating a new workflow are all activities that benefit from software engineering practices.
Let’s not get carried away: we’re talking about the 20% of things you could do to improve your work that will provide most of the benefit.
Here are five ways to apply software engineering to your no-code and low-code workflows so that other people will see your work as predictable and understandable. These methods also make it more likely that your automation will work. 🫡
1. Write out what you want to do before you do it
Open a document in your editor of choice (I use Google Docs) and write the process from end to end. What’s the end goal, and what needs to happen to take the initial state of the process from start to finish?
Stating clearly what you’re trying to do is an underappreciated step. If it doesn’t make sense now, it will not make sense in code or in a no-code process.
2. Each piece of code has one job
It’s tempting to build everything in a monolithic function, said mostly no one ever. But that’s what you’re doing when you use a lot of no-code tools and try to do everything all at once. This practice makes it very hard to debug when something goes wrong.
What’s easier? Building a more modular process – made easier by code orchestration tools like Pipedream – where you define the event trigger that starts the work and follow it through each individual step.
Search_for_records, determine_the_update, and update_records are examples of easy-to-read pseudocode step names to transfer from your initial design to the abstraction where you describe what actually happens.
3. Build testing into the process
“What could possibly go wrong” is a good place to start your testing. When you anticipate the most common things that will break in each step, it’s easier to catch them and refactor your process.
What are the most common user errors when you’re building this sort of automation?
For me, they are:
-
poorly scoped variables – when you think you’re getting a JSON key and you’ve targeted the wrong part of the structure
-
control logic that misses a case – when you are looping a collection and haven’t determined the possible outcomes
-
calls to external APIs that don’t resolve in expected ways – when you call an API and have to wait for it to have completed an action, you might need an additional call to complete the task
4. Self-document the process
“What’s actually happening?” If your automation reports the current progress, it’s a lot easier to see what happened and where it went right or wrong.
Use console.log or another method to document important milestones so that you can play data detective after the fact when things don’t go as expected.
5. If possible, use source control
One of the things I really like about Pipedream is that it integrates directly into GitHub. This makes it possible to treat your low-code process much more like a software project:
-
You can work on a branch while the existing automation continues
-
It’s possible to ask other people to help in reviewing PRs
-
When you need to revert your change, it’s an easy process
Source control can seem like extra work, but future you will thank you. (Especially when you leave decent comments.)
What’s the takeaway? There are a few process steps you can follow to make automation easier to understand and diagnose. When you document how the process should work, log how things are working, and create a way to roll back changes, you’ve improved the odds of building successful data plumbing.
