Data is an Ecosystem
Firms often believe they can get more organized by purchasing all-in-one enterprise database software. But while centralized databases are a valuable component of any data strategy, most long-term solutions operate as an ecosystem rather than a one-stop shop, and need a multitude of tools.
The Myth of the all-in-one database.
When firms decide ‘to get a better handle on things’ for managing workload, projects and staffing, they usually look for a software-based solution and often adopt some form of Enterprise Resource Planning (ERP) system, an all-in-one package for projects, timesheets or financials. Frequently the ERP vendor’s accompanying brochures and slick website graphics make it look like the chosen system is all that is needed to spit out clear and useful data on projects, staffing levels and profitability.
Example of promised benefit of an ERP system. Source: NetSuite
But real-world experience with ERPs frequently do not live up to that promise. Most ERP systems handle a limited subset of an organization’s needs, and the software is often organized into separate modules that are not as seamless as they first appear. In addition, getting data points into the system can be laborious and clunky. Typically, firms get some improvement from the ERP they’ve adopted, but it is often cumbersome to operate and not as transformational as advertised.
In this article I want to shed some light on the limitations of ERP-only strategies and to help organizations realize how they can begin to broaden their toolset to create the data insights they really need.
What is an ERP?
Before we look at some of the limitations of ERPs, let’s look at what they’re supposed to do. ERPs claim to help organizations manage core business processes by providing a single source of truth and streamlining operations across the enterprise. Common ERP systems include NetSuite (a general ERP financial/invoicing system) Deltek VantagePoint (used by AE firms), and SalesForce, (used by sales-based organizations). The systems are usually capable of linking a company’s financials, operations, manufacturing or human resources activities on one reporting platform. (see image below).
The promise of an all-in-one business leads system. Source: Salesforce
By collecting an organization’s shared transactional data from multiple sources into one location, ERP systems eliminate data duplication in separate files and provide data integrity with a single source of truth.
Most ERPs are reasonably good at basic invoicing, financials, and accounting tasks, and to some extent a CRM module for contact management. But often these systems are cumbersome especially in cross-reporting data, and generating meaningful report visualizations. Their often rigid, segmented workflows can’t respond to everyday business questions or help with future business growth. As companies grow and their data needs change, the system should keep up with their needs.
Expanding the Data Toolset
Let’s focus on improvements then. I find the single biggest misconception I have encountered about data systems is when users believe an ERP is the beginning and end of their data needs. In my experience most successful solutions recognize data management has multiple, distinct sets of activities and thus different tools are required at each stage. This is probably the most important lesson: to recognize that you might need a different data tool in the first place. Most people feel it is a failure of their strategy to use something other than their ERP for their needs, but it’s actually a more informed strategy.
“This is probably the most important lesson: to recognize that you might need a different tool in the first place. Most people feel it is a failure of their investment to use something other than their ERP, but it’s actually a more informed strategy.”
In a full-fledged data solution, I would break an enterprise’s data flow into at least three steps – Input, Storage and reporting. I will post more about these stages in future posts, but for today I want to zero in on the topic of reporting.
I choose reporting because it often produces the most disappointment for data users. Firms spend a lot of time initially generating the data, and then run the reports only to find that the output is a series of tabular reports with lots of headings where it can be difficult to discern patterns. Most organizations understandably use their ERP to generate reports of business activity, but there are far better tools for reporting and aggregation, and the real trick is to figure out how to exchange the data from one system to another.
Typical reports comprise lots of columns of data with multiple headings but few recognizable insights.
This was certainly my experience. In the AEC world, most of the firms I have worked at have used Deltek Vantagepoint as their ERP, but few have really been able to leverage all the promise of the software. This could be for several reasons:
they’re not that data-aware, and don’t understand a lot of what’s going on under the hood
the system overpromises what it can deliver (mainly because of the effort required to input data)
it can take a lot of effort to generate reports that really describe what’s going on.
This is where the concept of data as an ecosystem really helps, when people realize that their ERP is not the correct tool for all data tasks, especially to generate reports. For data reporting and aggregation, there are few tools better than Power BI because it is a generalized data-shaping environment with powerful tools that allow raw data to be totally reshaped, corrected and cleaned. Most ERP systems have a capacity to export reports to .csv format, or .xlsx. Using this export methodology allows the user to run analytics from the exported ERP data within a totally different environment. And when the data is updated in the ERP, it is simply re-exported, refreshed in Power BI and the new metrics populate.
In this way the ERP and Power BI are working together with a stream of data flowing between them for optimal results. Again, I will post more about this in the future. but it is a powerful example of how there are often better tools for specific aspects of the data continuum, but are not part of the ERP you are using. The image below introduces many of the common data software formats in the data ecosystem. Not all of them are used in every solution, but being aware of them, and when to use each one, is a useful sensibility.
The ecosystem of data includes lots of programs from multiple vendors.
The concept of data as an ecosystem is a fundamental building block of this business analytics blog. Several future posts will likely build on this premise, and will highlight many of the tools, and sometimes psychology, that are required to build a successful data analytics ecosystem.
