Organisations no longer operate as a standalone entity, and instead are part of networks comprising suppliers, resellers, channel partners, regulators, and other stakeholders. Analysing external data can point out the risks, opportunities and trends that firms would miss if they relied on data generated from internal operations, customers, and first-tier suppliers.

Relying solely on internally generated information can leave gaps, and as organisations realise this, they are increasingly moving to incorporate new, non-traditional sources of data that sits outside their systems. The challenge however is analysing the tonnes of data being gathered and stored at an exponential pace. According to a study, the data stored in data centres will grow almost five-fold to reach 1.3 zettabytes globally by 2021.

An MIT Sloan Management Review report found that the firms making the most innovative use of data and analytics were more likely than others to leverage more external data sources, including social, mobile, and publicly available data.

Why external data must be a part of your data strategy

External data gives a bigger picture. Collecting, evaluating, and analysing external data – such as user generated data, public data, competitor data etc – gives business leaders the full view.

It is not expensive. Today several tools, both paid and open source are available, to make sure it does not cost much to source external data. Data from government organisations, the news, social media and other online and broadcast media is even available for free.

Real insights with external data analytics. External data analytics can have a major impact when it comes to making decisions about the future of a business. Organisations can personalise marketing offers, improve HR decisions, build new revenue streams by launching new products or services, improve risk visibility and mitigation, and better anticipate shifts in demand. For instance, investment firms can use third-party data to build models that could predict the best types of customers to target in marketing campaigns. External data can help train the models to identify potential targets that fit profiles similar to the most engaged customers, thus optimise marketing spend. Several start-ups monitor social networking data to predict customer patterns and employee sentiment.

External data helps a business stay competitive. With competition for the customer’s wallet share being at an all-time high, the ability to quickly and regularly keep track of what the competition is doing is invaluable. Organisations can also predict trends and spot patterns that will make them more relevant to customers.

Add real-world context to decision-making. Organisations must gauge and predict the impact of external events – such as shifts in global purchasing trends, pandemics, marketing campaigns and so on – and guide product/ service decisions.

Tap into a data ecosystem

Unfortunately, as studies indicate, most organisations have not yet built in-house capabilities to put third-party data to good use. This would involve identifying, evaluating, procuring, and preparing external data consistently, and designing a continuous process to identify, engage with, and evaluate new external data sources. They would also need to regularly engage with partners and fuse these data sources with analytics processes or product offerings, as well as internal data.
It may help to be a part of a larger data ecosystem which involved multiple entities that directly or indirectly consume, produce, or provide data and other related resources. Organisations can create a cross-functional group as an interface to the wider data ecosystem in order to draw on competencies from multiple areas, such as product management, business analysis, data science, legal, and procurement, to address organizational and technical challenges related to third-party data. They can also create specific roles – termed ‘data curator’ by Gartner – focused on handling third-party data and related requests. Curators can keep data requests and sources up to date, while also ensuring quality and accuracy of data.

Connecting to a data ecosystem

We can categorize data services according to the level of insight they provide, as detailed here:

Simple data services. Data brokers gather data from a variety of sources. The conditioned data they provide serves as an additional input to the decision process, be it for a human user or device.

Smart data services. Analytical rules and calculations are used to enhance the data and present it as scores or tagging of objects.

Adaptive data services. Specific analytical requests from customers are catered to by combining third-party data with data from other sources.

Other ways to segment data services include specialists in domains, such as hedge funds or health care providers.; consulting and systems integration services providers who cater to demands for new insights from publicly available data and other external data, in addition to custom analysis.

The challenges of using external data

Access to external data is getting easier in some ways, but it can still be daunting. Organizations report a wide variety of business and technical challenges in deriving insights from external data. Among the business challenges are the size and complexity of the data-provider market, which can make it hard to identify the right data sources and partners. Negotiating acquisition of data can be arduous, depending on factors such as:

• Ongoing access to data for refreshing machine learning models
• Usage restrictions
• Revenue share demand from the data vendor
• Liability if the data proves to be inaccurate or tainted

This process can involve lengthy risk and legal reviews of vendor contracts and licensing agreements. The ongoing management of a growing roster of data-sharing relationships and partnerships can be taxing as well.

Third-party data can bring lots of opportunity, but applying it for real results can be challenging.

Even before we consider the technical challenges that hamper the deriving of insights from external data, it can be challenging to identify the right data sources and partners. There are several factors involved such as updating data, usage restrictions, revenue share demands from the vendor, liability of inaccurate or tainted data and so on.

Technical challenges include essentials such as measuring data quality and filtering out inaccuracies. Data pre-processing, such as cleansing and formatting it for analysis, takes a lot of time. Once you have sorted out good quality data, cataloguing it and keeping it secure is the next hard task especially if you have systems originally designed to manage only internal data.

Few organizations have standardized procedures to deal with external data and even fewer utilize external data to its full potential. As internal data analysis teams are less familiar with external data, it might take them a bit longer to understand external data, simply because it’s more complex and quite different from the internal data they are used to compiling and evaluating. Which also means that the data team will have to learn how to package and interpret external data and apply it so that they extract relevant answers to business questions.
Several studies indicate that third-party data is riddled with inaccuracies as well as inconsistencies between external and internal data to resolve before performing an analysis. Cleansing and formatting data before analysis takes a lot of time, with 80% of the analyst’s time being taken by data pre-processing as reports suggest. Organisations may even need to update information management processes and capabilities to securely store and catalogue external data, because these systems have until now only handles internal data.
Nevertheless, the virtues of external data easily outweigh its faults. For instance, in a pandemic relevant data can be drilled down for use in local areas. The real estate industry makes extensive use of external data to draw insights, such as identifying what suburbs and cities to target based on people’s income levels. Meanwhile, logistics companies use geolocation, weather and traffic data, and data about exceptional events – such as natural disasters – to manage their deliveries and avoid disruptions in the supply chain. Generating such data inhouse is a time-consuming and arduous process that can be exhausting unless you are, or own, a data analysis firm. So external data takes away the pressure of producing relevant data themselves from organisations.

Summing up

As organisations increasingly source data from external sources, they need to take consistent steps to extract the most from this data by enhancing their ability to identify, evaluate, and contract for new data through a data ecosystem. With unrelenting pressure on them to improve the efficiency of their operations, organisations must intensify their pursuit for insights that will help them improve business.

By expanding the universe of data outside of traditional organisational boundaries and adding the dimensions of external data, businesses increase the effectiveness of decision making. Moreover, this shift towards external data driven decision making is putting to use the collective wisdom of crowds that can provide faster, better and cost-efficient predictions that reflect the experience and activity of the many, not the few. By applying external data analytics at the right place, businesses can convert standard decisions to strategic decisions. They can combine this contextualised data with internal data to unlock powerful insights for innovation, growth and profitability.

At Tibil, we believe that while dashboards and visualizations are integral parts of data storytelling, there is much more to it. We work closely with our clients to ask the “why” behind the “what”, and turn your statistics into a powerful communication tool. Tibil has the ability to zero in on hidden layers within your business data, and then materialize this information into clear and simple actions for business strategy.