In software development, we talk a lot about being "agile." Few organizations fully embody that ethos.
Why? Because it's hard to coordinate a large group of engineers without some type of plan.
Unfortunately, a lot of organizations take this coordination problem too far. They try to plan out a roadmap a year in advance or maintain a months-long backlog.
The problem is that by the time you finish the plan, you might realize that you built the wrong thing! Or, your customers don't want it! (The classic problem with waterfall development.)
Agile response to change
Agile requires iteration and feedback. Preferrably as tight as possible.
Make a change. Measure how it is used. Adjust the implementation. Rinse & repeat.
This collaborative, iterative approach to building software makes you more likely to build something useful. After all, you're getting immediate feedback on small changes.
You can quickly identify opportunities to create value this way. But it becomes difficult to operate an entire engineering organization this way.
The need to plan
Software development is all about tradeoffs, and responding to change is no different. You'll never be able to fully get rid of the impulse to plan.
Planning can be valuable, too! Getting teams in sync & all pointed toward the same goals.
Executives & PMs will always want estimates of time & effort. They'll need to plan for headcount, budgets, timelines, and what you can promise to customers.
What to do?
You can't possibly know what will be most important a few months from now.
So, decide what's important now & try to focus your efforts on the things you can control. This often means deferring "good ideas" because there's something more important to do right now.
If the good idea is still relevant in a few months, you can do it then.
Often, your ever-growing backlog is your enemy. I even know of a few organizations where they believe in a zero-backlog system. Only the work in the current sprint is defined. Everything else is just an idea that will rise in priority if it's worth doing next.
For now, find an important goal. Figure out how to measure it. Then, quickly iterate to see if you can make progress.
When someone asks for a plan, roadmap, or estimate, give them the list of possible next ideas. Ask them to put them in order of importance & say "we'll do them in that order, unless something else comes up."
Pie in the sky
Maybe this agile mindset is pie in the sky. Can you really do this in practice?
I'm fully aware that this isn't always possible in many organizations. Even my own job requires more of a "plan" than this.
My point is that responding to change is more valuable than following the plan. A plan isn't worthless, but being able to change the plan is critical.
If you hold the plan less firmly & open yourself to changes, you'll create better software.