Are You Using Modern Data Analytics to Optimize Your Business?


AI-Powered Chatbots in Customer Service and Engagement

Using AI for customer service in your company is a definite method to save time and money. If you’re like most business owners, you’re constantly searching for fresh, creative ways to improve your enterprise. We’re here to inform you that improving AI customer service is a simple and rapid win.

Read More »

“In the end, the secret to successful analytics is not in choosing and implementing the perfect technology, but in cultivating a broad understanding that pervasive analytics yields better decisions and superior outcomes.”
— Eric Knorr, Editor in Chief, CIO

Business Intelligence Managers are among those seeking to harness and synchronize all of their “organization’s data.” In some sense, we are all involved with aspects of business intelligence. We increasingly depend upon modern data analytics to achieve optimal productivity, quality, efficiency, and security while also identifying risks and opportunities. We’re realizing that our organization’s data extends to all of our real and potential relationships (partners, vendors, customers), too.

For a company like Amazon, data management includes everything from optimizing the products recommended to each website (or storefront) visitor to how many books workers can pick, pack, and ship each minute. And everything in between. It extends to the data it collects from all of its suppliers, freight forwarders, third-party sellers, affiliates, and the spending patterns of all consumers.

Modern data management is all-encompassing, at least to the extent that business intelligence managers and analysts can make it so. So, we all have some part to play in data management. No one said it is easy, but it is becoming easier. We have better tools to collect and analyze data. We have more tools and even digital assistants capable of automating more and more processes.

Why Manual Data Analytics is Labor-Intensive

For the sake of being able to make a comparison, it’s worth looking at manual data analytics. It’s nothing if laborious. Historical “data libraries” have included everything from books to periodicals and digital files in a wide range of formats (.txt, .doc, .pdf, .xslx, .xml, .jpg, and countless others). Organizations would often spend millions of dollars and years of effort building their data libraries.

If a data scientist or analyst wanted to create a new report, they’d have to search through their library, pull the relevant data, and verify its authenticity. They’d have to clean it by finding and fixing any errors in it due to corrupt files or missing data. This could take weeks, even months. Only when this process was complete, could the data begin to be used for analytical purposes.

There’s at least one really, really big problem with this manual effort. Big Data. One GB of data is about 300 copies of Tolstoy’s “War and Peace.” It’d take a decade to read, say nothing of analyzing it. Today, businesses contend with Gigabytes and Terabytes of data, daily.

The Value of Real-Time Data

The older information is, generally the less valuable it is, unless it involves some kind of archaeological discovery. Most businesses aren’t digging for fossils. Our efforts are directed to things like improving productivity, efficiency, sales, customer satisfaction, market share, etc.

These things involve decisions. But what informs our decisions? Our gut? A Coinflip? Tea Leaves? The boss likes the way it looks? This is all hit or miss guesswork. It’s gambling.

So, what about data-driven decisions? At least you can use trends to inform your decisions. But, as they say, “past performance doesn’t guarantee future results.” It’s like using last year’s “playbook” to play the upcoming season. Trends can change, sometimes overnight. With some On-Demand companies, like Uber, they can change by the minute with highway construction or accidents. And of course, if your lead developer departs for a new job, your plans are likely to change, too.

The value of data is roughly proportionate to how fast we can a) see events, b) properly analyze them, c) reach decisions – and d) take action.

What Is and Why Invest in Modern Data Analytics?

Modern data analytics involves two major areas of effort. First, it strives to automate all of the efforts that have gone into collecting, storing, integrating, organizing, cleaning, validating, and analyzing data. Second, it mandates cultivating the skills, habits, processes, and culture so people can easily use this data to find actionable insights, and implement them.

Oh, and ideally, we want to do all of this in real-time. When an interesting event happens, we want to know everything we can about it, what it means, and what to do about it – to either trigger an automated response or flag it for human intervention.

Actually, it may be better to show a state-of-the-art example of real-time data analysis rather than try to explain it. In this video, Dilip Kamar, VP for Amazon Go, details the numerous technologies that went into creating cashier-less Amazon Go convenience stores. The same technology was very recently rolled out into a pilot for two Whole Foods supermarket stores.

TLDR; Every time a customer picks up a product is an event that gets analyzed in real-time and sets off a chain of reactions. Lots of sensors, facial recognition, an absolute need for accuracy of product and price, all feeding supply chain decisions.

For the full schematic – jump to 17:00.

Invisible Iceberg: Retail Application

What Are the Five Types of Data Analytics?

We have four, arguably five, different types of analytics to use for analyzing data. Of course, each industry has different metrics to track, but the way we analyze those metrics is mostly the same. What happened (or will happen) when, where, how, and why? BI managers also like looking at “What might happen if….?” scenarios, too.

Let’s keep it to past, present, and future and avoid past perfect continuous tenses, et al, as I just did that and saw that I’d be a millionaire if I hadn’t sold my Amazon stock options in 2004.

We can’t change history, but we can avoid making the same mistakes and improve the outcomes of what we decide to do with these five types of analytics.

Type of AnalyticsWhat it Typically Answers 
DescriptiveWhat happened and when? Past cases.
DiagnosticWhy or how did it happen? Past and Present cases.
PrescriptiveWhat can we do about our situation? Present cases.
PredictiveWhat’s likely to happen if we do x/y/z? Future cases.
CognitiveDigital assistants that let you ask “Google-like” questions for real-time, automated, data-driven insights.

1. Descriptive Analytics - Past Case Scenarios

“What happened and when?” are the focal points of descriptive analytics for defining events. You’ve seen descriptive analytics in line graphs and scattergrams to show that, “Sales of Beastie Beer have increased/decreased 40% over the past 90 days!” Descriptive analytics doesn’t define “Why” something happens – that’s left to diagnostic analytics.

Descriptive analytics show us trends and when you dig deeper into them can also show patterns. One pattern is how consumer spending spikes for Black Friday and Cyber Monday. More granularity would show that six-packs of Beastie Beer increased 80% and the two-liter bottles actually dropped by 40%. For engineering managers, it may be that team productivity increased, even as some developers’ productivity declined.

2. Diagnostic Analytics - Past and Present Case Scenarios

“Why or how did that happen?” Diagnostics analyze events to find contributing factors and root causes. Diagnostic software and techniques include data mining and examining causal relationships and sequences to measure the relative impact of different variables.

Well-structured computer hardware and software have a fairly finite range of factors to analyze to determine why something’s not WAI. Software development is complex, but we’ve been using diagnostics for decades to consistently find the causes of defects, and more recently, ways to improve developer performance.

Large consumer packaged goods (CPG) companies have exponentially more complex scenarios with scores of products facing off against dozens of competitors in hundreds of stores and regions. The weather, gas prices, public events, social media, marketing campaigns, new products, new competitors, and other factors, also weigh in. With enough data, it’s possible to determine the relative impact of each variable.

3. Prescriptive Analytics - Present Case Scenarios

“What can we do about it?” Prescriptive analytics work to help analysts determine the best answers or courses of action to specific questions and situations. In the simplest sense, we’ve seen it when doctors write us prescriptions – if we take our medicine, the pain goes away. Sorta. In an automated context, it applies simple rules to complex models (“If x, do y.”).

Business-wise, prescriptive analytics is underutilized but holds considerable potential. Few problems are new. Others have dealt with them, before. We can use descriptive and diagnostic analytics to compare which response worked best under specific conditions. This often takes shape as “best practices” but when that “data” isn’t shared, it leads to information siloing.

Siloing is a concern because of turnover. When an employee leaves, what they know goes with them. Prescriptive analytics can offset siloing while sharing “best practices” with all team members, ultimately improving cross-functionality.

4. Predictive Analytics

What would happen if we did this or that? Predictive analytics attempt to forecast the future using existing statistics, modeling, data mining, and machine learning to hone in on optimal outcomes. One thing that distinguishes it from fortune telling is that predictive analytics should be accompanied by each outcome’s margin of probability.

It almost makes me want to get my money back from the Tarot reader outside Mama Laveau’s shop in New Orleans. She’d probably chastise me with, “If you were using predictive analytics before selling your Amazon stock options, you would be rich.”

Leastwise, predictive analytics makes use of each of the three previous types of analytics to assess and recommend the best of your available options. It can also be applied to generate “Stop Action” alerts to warn of undesirable outcomes.

This is a huge deal in retail, as running a promotion offering say 20% off might generate more unit sales, but at a revenue loss and without necessarily attracting new users. That’d indicate existing loyal users buying more. A discount could also interfere with the sales of other products. But, if run in conjunction with a major advertising campaign, the promotion would have a higher probability of attracting new users.

5. Cognitive Analytics

Ask “Google-like” questions for real-time, automated, data-driven insights. Cognitive analytics are typically wrapped in the form of a Digital Assistant using technologies like AI and ML algorithms that already provide “better than human” capabilities for finding patterns and insights in massive data sets, on-demand. It wraps all four types of analytics together while adding self-learning capabilities to become smarter over time.

Another hallmark of a digital assistant is to render information and insights in a way that’s easy for people to understand. Smart chatbots can already communicate nearly as well as humans. One of today’s most prominent futurists, Ray Kurzweil, predicts computers will be able to ace the Turing Test, the standard for defining human-level intelligence, by 2029.

Yesterday, we all spent a lot of time scrounging for data and trying to understand what it meant to give us actionable insights. Cognitive analytics and digital assistants with AI-driven analytics can automate a lot of that effort for us. No one’s going to lament not spending weeks to clean their data sets. The time saved on the tedious aspects of data can be put to use with more important, harder, and creative tasks.

The business-world thrives on questions just like journalists do. It’s not a matter of knowing the answer, but being able to ask the right questions – to lead us to better outcomes. There’s an exponential component to this when data scientists, business intelligence managers, software engineering managers, and other specialists help their teams to also ask the right questions.

The faster technology evolves, the faster it will continue to evolve. 

Did you like our content?

Spread the word

Subscribe to Our Newsletter

Don't miss our latest updates.
All About Software Engineering Best Practices, Productivity Measurement, Performance Analytics, Software Team Management and more.

Did you like our content?

Spread the word

Subscribe to Our Newsletter

Don't miss our latest updates. All About Software Engineering Best Practices, Productivity Measurement, Performance Analytics, Software Team Management and more.

For up to 5 repos, 10 devs and 3 months of data FREE FOR LIFE