Agile Requirements Gathering

This post is part of the agile dimensional modeling series, that promotes the use of agile to speed up business value driven data warehouse development. In this post, we’ll look into agile requirements gathering.

The importance of the why

The first and most important thing to understand is always the why. The what, when, how etc. does not make much sense, unless we know why we are doing whatever it is that we are doing. In business intelligence and data warehousing, the why should always be because it adds value to the business.

Everything we do is done because it adds business value and when we need to prioritize what to do when, it is prioritized based on what adds the most value to the business.

Requirements gathering

Once we understand that what we will be doing should always be business value driven, it should become obvious that we need to talk to the business, to learn what actually adds the most business value.

The primary purpose of business requirements gathering is to understand what the business is trying to accomplish. It is the most critical step in data warehousing, as this is when we learn through conversations with the business what we need to build for them to achieve the business goals.

With the agile approach I am promoting, requirements gathering is an ongoing process unlike most conventional project management methodologies. In a conventional approach requirements gathering is often seen as a (lengthy) one time step, that will determine what will be done in the next many months.

With todays volatile business environment, we simply cannot afford to effectively tell the business that “you are only allowed to come up with requirements every six months, because we have already decided what we are doing for the rest of this project”. The answer is to work in short iterations and build our data warehouses incrementally.

Conventional requirements gathering

I have never worked with any organization, that did not already have a handful of very urgent business requirements the moment I stepped through the door.

With a conventional approach to data warehousing, these low hanging fruits or easy wins as I call them, will not be implemented until they have been very structurally considered along with all other less and very less important requirements.

Sadly you will spend weeks or more likely months analyzing requirements across the entire enterprise, before it becomes clear that you should have done these easy wins before doing anything else – including the last few months of analyzing requirements.

I love the Kimball dimensional modeling approach and his way of incrementally building the data warehouse. It is the most robust and scalable model for data warehousing. In my opinion the requirements gathering phase of the Kimball lifecycle is the biggest weakness, as it results in a very slow implementation of at least the first increment.

Incremental requirements gathering

What we should be doing, and what is fortunately being done more and more, is using some of the key elements from agile methodologies like SCRUM and XP.

As you will learn from subsequent posts, I highly recommend using user stories to capture business requirements. User stories are written by the business itself, thus you as an organization can easily capture new requirements concurrently with building a data warehouse increment.

User stories will be prioritized by business value and the data warehouse built in time-boxed increments of 1-2 weeks duration called sprints. As the business requirements that goes into a sprint is decided before every sprint, it is a very good way to ensure that you will constantly consider what makes the most value to the business right now.

User stories are placeholders for business requirements and contain just enough text to serve as planning and to remember what the story is about. With this approach you do not have to spend a lot of time with detailed analysis of the requirements, until the business decides that is now one of the current priorities for an upcoming sprint.

Next step

The next post will introduce user stories as a way to capture business requirements.