Zero-Latency Data and Business Intelligence

Business intelligence enables decision-makers and stakeholders to make strategic decisions based on the information available to them. Just as the quality of the data is critical, the timeliness of the data is equally so. Laursen & Thorlund (2010) identify three types of data:

  1. Lag information. This covers what happened previously, and may be used to feed predictive models attempting to create lead information. Although the data is recorded in real-time (i.e., a flight data recorder), reading and reporting from the data is done ex post facto.
  2. Real-time data. This data shows what has happening at present. Continuing the aviation example, the ADS-B pings from aircraft are real-time data points collected by receivers across the globe and fed to flight tracking sites such as FlightAware.com for real-time reporting.
  3. Lead information. This data is often yielded from predictive models created by real-time or lag information. Airlines use a combination of flight, weather, and air traffic data to project an estimated arrival time for any given commercial aircraft at a particular destination.

There are appropriate instances for all three types. Real-time tends to be the most desired, but of course with decreased lag and immediate demand comes a trade-off of processing power, vulnerability to errors, and cost. Somewhere between “very old” and “absolutely immediate” is the sweet spot of timeliness and cost-efficiency. In other words, the push for zero-latency data may be more costly than profitable. Businesses must develop their own cost/benefit models to determine how real-time their BI data should be.

One area of real-time necessity is item affinity analysis. Every day on Amazon, customers order items and are presented with other items that may be relevant to their purchase, based on purchasing patterns from other customers who have ordered the same thing as well as their own purchasing history (Pophal, 2014). This data must be zero-latency, as a recommendation must be posted almost immediately after the customer makes their initial order. A lag time of minutes, hours, or days would lose the potential sale.

References

Laursen, G. H. N., and Thorlund, J. (2010) Business Analytics for Managers: Taking Business Intelligence Beyond Reporting. Wiley & SAS Business Institute.
Pophal, L. (2014). The technology of contextualized content: What’s next on the horizon? Retrieved from http://www.econtentmag.com/Articles/Editorial/Feature/The-Technology-of-Contextualized-Content-Whats-Next-on-the-Horizon-99029.htm

Jonathan Fowler
About the author

Data-Centric Culture evangelist and BI leader with a demonstrated history of working in a variety of environments. Skilled in Big Data, statistical methods, machine learning, database design, and research methods. Clemson University alumnus ('04, '07) and doctoral student at Colorado Technical University ('21).

Related Posts