The Agile Planner

Why Big Data Fails for Agile Planning

 Email Article  

Tweet  

  

This massive, not very helpful sign does not reflect Big Data’s ability to make more sense from historical data sets; it reflects what happens if you try to apply Big Data to planning for the future.

When I turned 40 last month, I started reflecting on my past. To oversimplify, let’s suppose there are two parts of me: 1) Ben, the Business Intelligence guy and 2) Ben, the Agile Planning guy.

As Ben, the Business Intelligence guy, I am looking back and creating reports and charts as requested by virtually everyone in the business team. As Ben, the Agile Planning guy, I am looking ahead and helping management (in theory) make better decisions that drive the business forward.

A little more on my life as the BI Guy:

While I’ve used spreadsheets for planning like everyone in FP&A, I was lucky enough to first experience the joy of financial modeling software back when I was managing the Business Intelligence group at PlanetRx.com (a dotcom failure subject for a future blog). Using what was then known as Cognos Transformer/Powerplay/Impromptu, I created reports and graphs out of massive data cubes as requested by seemingly everyone in the company. We spent lots of money for the Cognos software and consulting services, and I got to go to a couple weeks of off-site training. It was fun. I didn’t know it, but I was living in the world of what is now called “Big Data.”

As the BI guy, I was in a perpetual state of “information overload” with lots of people making lots of requests for custom reports. Clearly, if I could have given everyone an iPad and set them up with self-service reporting tools, that would have eased my pain as BI manager. Enter Agile BI which enables everyone to more quickly access and understand historical data.

Even though I was “the BI guy” and could create beautiful reports based on historical data, I also needed to develop what-if models in spreadsheets to inform strategic decision making. I got to build 1-off spreadsheet models that would analyze decisions like:

• Should we partner with a major retailer?
• How should we set up the pricing model when we partner with prescription pad providers?
• What reimbursement rate parameters are profitable for us when we negotiate with insurers?

A little more on my life as the Agile Planning Guy:

Back at PlanetRx.com, I did not use the word “Agile Planning” since I didn’t even know what that meant; however, that’s precisely what I was doing when setting up planning models in spreadsheets. In these models, we did not have to worry about tons and tons of data; though we did develop tons and tons of spreadsheet files that existed in isolation. We also tried to summarize results in Powerpoint – something we now call the “Excel-Powerpoint cycle.”

Let’s look at the role of data in the context of Agile Planning which focuses on the future and is designed to quickly provide management the insight it needs to take actions that drive the business forward. While Big Data is great for analyzing historical data, it may in fact be the exact opposite of what we need to drive the business forward, to plan. To see this, let’s compare and contrast Big Data with “Small Data.”

Here’s the Wikipedia definition of “Big Data”:

“Big data is a term applied to data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time. Big data sizes are a constantly moving target currently ranging from a few dozen terabytes to many petabytes* of data in a single data set.”

Given I attend FP&A conferences regularly and the take-home point is always: get out of the weeds and reduce the data in your planning model so you can shift toward materiality rather than precision for precision sake, I was getting a headache from the contrary messaging inherent in Big Data messaging. I also realize the “Big Data” vs “Small Data” debate (if there even is one) could boil down to the CIO leading the charge for Big Data and the CFO being left swimming in a pool of data wondering how to dump it into spreadsheet models in order to support financially-sound strategic decision making.

The examples of the benefits of Big Data are big. Here’s one example from a leading group of pundits: “If US healthcare were to use big data creatively and effectively to drive efficiency and quality, the sector could create more than $300 billion in value every year” It is not my intention to bore you with the many, many other examples of Big Data. Just google the term and you’ll have more reading material than you could possibly digest.

The Big Data – Small Data Analogy

Whether we’re dealing with “Big Data” or “Small Data,” any data analysis at its core is designed to support the business. However, if we break data into two size categories and create an oversimplified analogy, we arrive at the premise of the whole blog:

Big Data : Analyzing Historical Data with Business Intelligence as Small Data : Agile Planning.

Big Data’s message is all about MORE: if we get as many people as possible using as many devices to view the most-current data possible, and could better analyze all the data that we gather at the lowest possible level of detail with the most possible meta-data dimension member tags creating massive data cubes with petabytes of raw data, we can then somehow improve our business.

Small Data is characterized by LESS: Less data at the right level of detail to support decision making with the minimal possible dimension members that are material and emphasizes collaborative thinking and analysis with only the right people (not necessarily more people). Small Data focuses on materiality and meaning; Big Data focuses on precision and details. According to the Planning Maturity Curve, the key to getting to Agile Planning is:

1. Reduce The Level of Detail
2. Implement Driver-Based Planning
3. Integrate (don’t just import) Actuals
4. Analyze Scenarios (i.e. perform what-if analysis)

So, Big Data and Agile Planning just don’t fit. When it comes to making planning an activity that informs decision making, step 1 is about cutting out the detail and getting to less data, step 2 is about focusing only on a limited set of key drivers and setting up relationships to eliminate the dependence on massive data sets, step 3 is about figuring out which historical data to incorporate into the financial plan, not simply about moving ALL the data from one system to another system.

We all know there’s a place for Big Data, but in today’s emphasis on best-of-breed solutions, let’s not forget the little guy. Small Data is what enables Agile Planning. Big Data fails.

I’d like to thank Rob Kugel of Ventana Research, Craig Schiff of BPM Partners, Nick Castellina of Aberdeen Group, and Barry Wilderman of Constellation Research who provided “analyst” support to inspire this blog. If you’d like to co-author a white paper with me on this topic, please let me know.

I also want to note that others such as Capgemini are writing about Small Data.

Lastly, I want to acknowledge Sid Ghatak who showed me an analysis that actually concluded it takes longer to update a rolling forecast when more people are involved. This supports the notion that with go-forward planning, less people can lead to better results.

*I must admit, I did not even know what a petabyte is, but it sure sounds like a lot!