Home > Learn > Guides > Data science in business strategy and decision making

Data science in business strategy and decision making

Bridging the gap between data science and business

Anyone who has watched Moneyball will know that it can be one of those “feel-good” movies, even if sport-themed movies may not be everyone’s go-to flick. What’s not to like? There’s drama, humor and a story about the little guys winning and achieving success despite having the odds stacked against them. For data enthusiasts that have not watched it, feel free to read on as there won’t be any spoilers, but really consider watching it through the data lens. In a nutshell, it’s a true story about the time Oakland Athletics’ General Manager successfully attempted to assemble a baseball team on a lean budget by employing computer-generated analysis to acquire new players (IMDB). In data-y words, a team leverages data science and analytics, enabling them to compete against teams with considerably larger budgets.

From managing a professional baseball team to running a company in a traditional industry, they all share a common theme of trying to succeed by advancing their analytics and data science practices with limited resources. Companies around the world, both larger and smaller, are now racing to enhance their analytics capabilities, and yet up to 85% of data projects fail despite significant time and capital investment. So, what’s behind this slightly worrisome statistic?

It is well known that data is kinda nerdy so it can get shoved into IT (simply look at Hollywood’s portrayal of data analysts in… let’s say Moneyball). Then all data problems are accordingly treated as IT issues, or treated in a silo within a business unit and the general view remains that business should primarily be concerned with the commercial aspect of running their specific part. This mindset continues to drive a wedge between the business and technology teams, where it ultimately creates a situation one may consider as “the left hand does not know what the right hand is doing”. If both hands (technology and business) are part of the same body (the company), is it too much to ask that they both work harmoniously in achieving the best outcome possible?

If one hand is too far away from the center then it is physically impossible for the other to reach it and vice versa. This is similar in the context of data science and business. If one is too focused on their own matters, it is very difficult for data science projects to achieve their full potential and be a significant driver in business success. Working together with an open mind to identify new ways of working and actively providing inputs into the final solution is a responsibility that each side must take on. One of the critical paths to the success of advanced analytics projects is through business buy-in and trusting the investments into data and AI teams. However, the teamwork between the data scientists with their technical capability and the business professionals with their market and domain expertise is also of significant importance in delivering value to the business buy-in. Bridging the gap between business and technology is absolutely essential in achieving success in enabling data science for business and decision making.

There are multiple actions and areas of focus to consider in bridging the gap between data science and business strategy and operations. These can include defining a data strategy, uplifting data foundations, and implementing data governance, but the next best actions will depend on the business’ current data, its data science maturity and the competitive landscape. However, irrespective of where the organisation is at in its AI journey, there are common factors to help unite data science and business decision making that underpin all stages of development. Let’s explore a few of these common factors and dive deeper into what could be done to address them – turning data science and analytics into a true business enabler.

1. Time dedication and capabilities in data science translation

While resourcing and capability management is a continuous battle to manage across all levels of the business, it still warrants a special mention within this problem. As the value of data science and machine learning has become prominent, businesses have rushed to hire and set-up specialised teams of data scientists, data engineers and MLops specialists. However, the same focus has not been made on the effective integration of these teams into the BAU of core business units and activities. Allocating sufficient resources and capabilities are essential for effective integration. Neglecting this will only run the risk of data science teams and other business departments working in silos, limiting the capacity to affect customers and business goals.

When considering resourcing, it is important that there is a mandate from senior leadership to allow the business SMEs and/or specialists to have sufficient time to work with the AI teams on the advanced analytics problem. This is an area that is often left unrecognised when managing the day to day. While data science projects appear to work like ‘magic’, it is underpinned by the sound and detailed understanding of the business data, operational processes and domain expertise that inform the modelling approach and final solution. This understanding and expertise is, for the most part, best provided by the business SMEs and specialists.

Despite the good intentions of adequately resourcing your AI teams and including business SMEs into the projects, this will not guarantee success in advanced analytics. From a conceptual level, data specialists have high levels of expertise in data manipulation, analysis and programming sophisticated models to compute vast amounts of data. On the other hand, specialists in various business areas have high levels of expertise in their domain, whether it be in sales, marketing, logistics, etc. Consequently a void emerges as either side does not have a sound understanding of what is possible or necessary from each other. Sufficient understanding of data science and engineering combined with sound business and commercial acumen is an emerging specialist role. McKinsey defined this role as the Analytics Translator (note: industry yet to align on a standard naming convention for these roles), and projected that by 2026 there will be a market need of around 2 to 4 million of these specialists in the US alone. If we consider the ANZ market, that would equate to approximately 200 – 400 thousand roles! Whether the business decides to go down the path of new hires, train internally or utilise consultants, having dedicated resources whose responsibility is to navigate data science opportunities of the business is an effective tactical response to bridge the gap between data science and business strategy and operations.

2. Well-defined and vetted use cases

Use cases with well defined problems and investigated solution viability are critical for bridging the gap between data science and business strategy and decision making. It acts in a similar way that architectural blueprints do for a building project. Having well defined use cases also helps to effectively evaluate and prioritise between different use cases, as well as informing broader business strategy and strategic resource allocation. While there are multiple aspects and considerations to a well defined use case, it is worth highlighting three parts that directly help in bridging the gap between data science and business.

Firstly, identifying the right problem to solve is critical. While the statement is obvious, it is seldom done really well. Design concepts like Human Centred Design can help to put end users or customers at the heart of any solution and help to critically evaluate the identified problems and needs. However, without proper planning activities, it can be difficult in practice to challenge the identified problem or need in regular BAU settings when there is a large divide between areas of expertise. The difficulty is further exacerbated with trying to maintain good rapport, especially when it is the start of a new work relationship.

Let’s consider an over-simplified example of data science use case in retail, of a business development manager wanting some help from the AI team.

During the discussion, AI problem solution areas are highlighted:

Wrangling multiple data sources
Consistent and automated performance calculations and metrics
Prediction of customer performance
Understanding the impact of different marketing actions
Accuracy of the change in future performance of customers, based on action taken

From the example, there are a number of different advanced analytics use cases that can be identified, each solving a different part of the business activity with different levels of difficulty. What initially started as a reporting problem has grown to complex modelling of customer actions, thus creating additional strategic considerations of what is the right problem to solve.

Secondly, after the business problem has been identified it is easy to assume that it will be solved, without taking proper consideration of inherent barriers and risks that the solution will not be able to reach the desired outcome.

For any business problem that is identified as the strategic priority, technical feasibility must be assessed. While factors to investigate will depend on the nature of the problem, the following questions can help lead the initial investigation:

Is all the data available right now?
Is the quality, breadth and tenure of the data sufficient?
Is the analytical approach already known to the data scientists?
Will the solution need to run in a production environment, and how frequently will results need to be updated?
Will solution output need to integrate into other systems / solutions?

Thirdly, even if the right problem to solve has been identified and the solution can be achieved, a review of the business process is necessary to ensure that the solution outputs will be actionable and can be systemically adopted. Often the excitement of a better future state can assume away the necessary steps required for its implementation. Role playing through the questions of ‘how will I, or the team, action this’ and ‘how will I, or the team, use this everyday’ can help the business to foresee barriers or dependencies within the primary recipients of the solution and/or other business units. This will aid in recognising the full extent of the required effort for solution implementation and will provide a more complete assessment against the expected benefits. In addition, this will help to define any changes to the solution or outputs that are necessary to overcome barriers to implementation at an earlier stage of development. This is important as changes that may seem minor from the business perspective, can require a significant amount of re-work from the data engineering and data science perspective, which could be avoided if communicated earlier.

3. Continuous validation

Due to the general lack of context and problem knowledge overlap between data science teams and business units, there is a heightened need to communicate complex issues effectively and in a timely manner. Accuracy and completeness of each component of the AI solution needs to be aligned to the specifics of the problem and business expectations. Good practice calls for ongoing collaborative efforts between the AI team and business to validate the progress of completed work and clearly identify what is needed from each other.

Having action led conversations between AI team and business is a great way to help filter and separate information that is critical vs contextual. It is very easy to fall into that trap of providing as much information and detail as possible, without realising that it is not helping the other side to identify the most important parts of what you need. This creates the potential for delivery inefficiency, additional re-work or non-optimal solutions to the underlying business problem. A good method to help prevent that is to focus on being clear up front with what actions are required from the other person and detail further only as required.

It is important to note that even if communication is led by actions and free from contextual information overload, the end solution still may not achieve the desired outcomes. This is where continuous validation focuses to ensure that the progress and associated completed works are aligned with the requirements for business to take the desired action. This needs to be conducted throughout the entire solution development and implementation life-cycle, including data engineering, solution design, output review, solution production and automation and business implementation. A 2020 global survey on AI conducted by McKinsey, highlighted model explainability as a key factor in business end state adoption. Continuous validation is a significant tool in helping AI teams to explain their models and how the solution works, giving the necessary confidence needed for broader business adoption. AI and ML solutions are, ultimately, journeys that both the builder and recipient of the final solution have to go on together.