AI Success Depends on the CFO, Not IT | Gartner Finance Keynote

hi everyone and welcome we are so glad to have you here at the Gartner CFO and finance executive conference now it's no secret that you have been hearing a lot about AI over the past couple of years we are at the beginning of a new and exciting era an era where AI will shape everything that we do but while the world gets excited about the possibilities of AI CL and I are here to talk to you about the realities of AI and the reality is more complicated than it seems we're in a transitional period in

our Enterprises AI is starting to take over more and more tasks and I think the question on all of our minds is what does the future look like for us in 2030 so Nisha and I wanted to start this morning with a little story about how to thrive in a post aai era see this era that we're in is a lot like what happened to the world of Chess in the '90s now in the ' 80s and 90s IBM developed its famous chess playing supercomputer deep blue in 1996 deep blue lost a series of games

to arguably the greatest human chess player in history Gary Kasparov now after a little further work the next year deep blue beat Gary and it was from that moment on computers were better at chess than humans now I think we're all starting to look at this and saying well that looks a lot like our Enterprises AI is taking over more tasks it's better at some things than we are but not everything but we're starting to see the writing on the wall and here's what we can take away from the world of Chess and it's good

news first professional chess is still a game of human versus human the other thing we can learn is that there's a new greatest in history chess champion Magnus Carlson whose key to success is AI but not quite in the way you might think and that's the point of the story we want you to focus on see it's no surprise that all chess players today use AI to train and practice it is common place and you might assume that magnus's competitive Advantage is his ability to memorize more outputs from the AI model or maybe he's got

some custom AI program that's better than his competitors but you know that's just an arms race approach with quickly diminishing returns instead here is how magnus's coach describes it modern chess is like this if we all follow what the computer tells us we both have the same Source the same ideas and guess what the game will be a draw so we try looking for on thee Edge Concepts that's chess that's Sport and what's happening in the game of chess today is no different than what's happening across our Enterprise across Industries investments in AI are at

an alltime high and will continue to Skyrocket Gartner software market analysis shows by 2027 spending on AI software would roughly double to more than $300 billion just like in the world of Chess AI is going to be common place in our organizations where every function across every organization will be working with AI to drive productivity enhance decision making automate your processes and even create new business models but it's not spending more on AI or replicating what others are doing in AI that will help us succeed it's how we use AI that will drive our success

just how Magnus uses AI to discover new winning strategies in chess the same way we use AI to discover new winning strategies for our organization that will be the key differentiator that determines our success over the next decade now here at Gartner we've been working with over 80,000 Executives around the globe on their AI initiatives helping clients like yourselves answer what are The Right Use cases how do we improve our data how do we prepare our teams for the new era of AI and so far Nish and I are excited not only by the progress

your teams are making but by some of the benefits you're reporting to us collections teams are improving settlement Times by up to 40% with AI recommendations fpna teams are delivering forecasts with 98.5% accuracy or higher in hours instead of weeks employee productivity is up 90% in highly transactional processes and some teams are saving millions of dollars by automating fraud and risk monitoring and that's probably why these Investments are at an all-time high like Nisha said within the finance function none of you are projecting any cuts to spending and most of you are planning increases to

spending in AI but as Enterprises continue to invest in AI outside of the finance function we see real cracks emerging in the promises of AI it's not the technical side that we're worried about here at Gartner it's the organizational side we see real Enterprise level organizational challenges emerging with early adopters and we want to focus today on four challenges in particular that we call the AI stalls these stalls will be common to all organizations regardless of geography industry size revenue or whether you're a public or a private Enterprise and these stalls they will derail your

teams slow your progress and could even upend your organizations and those four stalls are cost overruns misuse in decision-making loss of external trust and rigid mindsets but here's the good news these stalls they are solvable but solving them requires an active engagement from us from CFOs while all other Executives in your or organization are focused around the rewards and Technical challenges of AI it will fall on you the CFO to navigate the Enterprise through these four stalls solving these stalls will be the winning strategy that differentiates organizations in the AI era so this morning we

not only want to prepare you on what these stalls are but also show you what role you will play in navigating your Enterprise through these stalls so that we can capture the real rewards of AI and this has to be a priority for us because our Enterprise leaders across all functions they are so excited about the possibilities of ai ai has created so much Buzz across all Industries in fact boards are asking about AI 3.4 times more than cloud and 2.5 times more than digital transformation both of which were top conversations at the board not

to too long ago and AI continues to be the top technology for innovation 61% of our Enterprise leaders are relying on AI to drive Innovation within their function our colleagues across the functions they're chasing AI benefits in areas like product development marketing legal customer service and more but these stalls that we talked about that will prevent our Enterprise from reaping the real rewards of AI to show you where your Enterprise will hit these stalls let's first discuss how your Enterprise colleagues are investing now there are two types of AI Investments your executive peers are making

everyday Ai and game changing AI everyday AI is focused on productivity it helps us do what we're already doing capture the same value but faster and more effectively and that's where 77% Of Your Business Leaders are focused right now and the other type is gamechanging AI which enables us to create new value through new products and services new ways of working and new business models and of course we can apply AI internally to our people in processes or we can use it in external facing cases now what you're seeing here are four zones or opportunities

where your Enterprise leaders are either piloting or have already operationalized AI across your Enterprise and what you're seeing here these tiny yellow circles they are some use case examples in each of the zone so if you can follow me to the bottom left here is where your Enterprise leaders are using AI to improve productivity within your organization PA's back office so for example your it team is using gen to code faster now if we move to the upper left here we're entering AI use cases for the Enterprise front office where say your marketing team is

using AI to generate content in matter of minutes instead of days now if you move to the bottom right here your operations teams are transforming how we deliver core products and services for example your service team is using AI models to predict the next asset maintenance and finally in the top right here your Enterprise leaders they are generating new AI enabled revenue streams and products such as Dynamic pricing or offering robotic assisted teaching in classrooms and regardless of which zone your Enterprise is investing in you will run into these four possible AI stalls so Nisha

and I want to use this morning here to talk about each of these stalls with you and the role that we at Gartner believe that CFOs and their teams can play to help navigate the Enterprise through these stalls we wanted to start this morning with the one that you're probably most comfortable and familiar navigating cost overruns now cost overruns refers to the growing and unknown cost of AI across the Enterprise that has the potential to derail your AI Ambitions now cost management is not new of course for CFOs right so Nisha where could you and

I possibly teach something to CFOs about cost management what's different here it's a great question we do want to bring to attention how unique AI costs are see given how new AI is we don't really know how much it costs and our learning as we go and because of this newness our Enterprise collect ly have not been doing a great job at estimating AI costs in discussions with our clients we have found that AI cost estimates were often off by 500 to 1,000% and in case you're wondering no that's not a typo so to understand

what's driving this high cost variance let's look at what the typical categories of AI costs are at a high level there are three buckets of costs associated with any AI implementation your initial roll out costs ongoing costs and experimentation cost now let's start with your initial rollout costs this comes with any technology Investments we have been doing and includes things like infrastructure software new talent in this case hiring data scientists and implementation costs now the next two buckets these are new and we are uncovering these costs with every new AI implementation so the first ongoing

cost this is where we go wrong this is not your typical goal post go life support cost but this is the cost of maintaining the models ensuring it's compliant continuous data maintainance and some surprising costlike environmental cost of running these large language models and usage cost where you pay per employee per query so consider the cost you incur when your employees start using chat GPT where a simple Google search would work you will be paying per query times employee that adds up and finally the last bucket of cost which all our Enterprise leaders tend to

ignore cost of experimentation or sunk cost and this is very unique to AI because unlike other Technologies AI follows an experimentation process now with experimentations there are bound to be failures right it could be you low adoption or simply because we chose the wrong use case now let's see how these three categories of costs add up within Gartner research we have a detailed AI cost estimator in fact there's a session of this in the conference and what you're seeing here are cost estimates for a very basic AI deployment for just about couple of hundred users

where we have already committed over a million dollar now I don't know about you Clen but for a basic deployment this number seems a little bit steep right it really is and if that number doesn't quite scare you yet let me show you how we project AI costs to profile out over the next few years the initial roll out costs may be moderate to start with but as you increase you'll get into more complex AI models and then eventually it'll start to taper off again the cost of experimentation works the same we expect it to

be high today as your teams are learning how to best use Ai and then it'll taper off as you get more proficient in driving Roi with AI use cases now these two buckets of cost they should remain your immediate Focus given how high these costs will remain over the next two to three years but the more surprising thing are the ongoing costs unlike traditional Technologies the cost of running AI will actually increase as you use it more usage costs will increase as every single employee in your organization starts using Ai and you process more compute

jobs you process more tokens it's this spike in future ongoing costs that your executive colleagues are not planning for in their estimates and has the potential of negating all of the ROI of AI as usage continues to grow and this cost projection it's not limited to Investments only within finance and AI unlike other big Tech initiatives is universal and is applicable across all functions in our Enterprise all your Executives they're choosing their own vendors each with their own pricing model and they're piloting their own discrete AI use cases within their teams and remember what we

said these AI invest Ms are only going to grow over 73% of your peers say they plan to increase AI Investments through 2024 so what do you think will happen if we don't get a good handle around these discrete functional Investments and the cost projections that CLM just shared our organization will suffer from mounting cost overrun that will ultimately bring our AI ambition to a grind halt now we are Finance leaders managing costs maximizing Roi that's your bread and butter and starting with a cost breakdown on a Monday morning well that only flies in a

room full of Finance professionals right and probably only after we've let you hit the coffee bar so let me radically simplify Our advice for you here the best thing that you can do is to unmask the hidden costs of AI for your executive colleagues to avoid the risk of massive cost overruns and to be really specific in Our advice let's go back to those three categories real quick the first category initial rollout costs Our advice here is really just to keep calm carry on with your current cost management techniques meaning centralize implementation and vendor management

activities to reduce duplication of costs and scale existing vendors to have a better control over software costs this will help us manage those scary near-term costs now as our investments progress we then need to manage ongoing costs and here we recommend applying rigor around investment evaluations and how much they will cost include those in your Roi calculations and make sure you're really pressuring your colleagues on those ongoing cost projections for any new AI Investments and for any AI models that are already live monitor the ongoing costs of the running the model very closely and frequently

as you plan to increase usage review vendor contracts to ensure that you're not exposing your Enterprise to unlimited liability as you add more and more users to the software and finally the last bucket of cost which is the cost of experimentation here we need to do two things AI models that are already live should be monitored for and analyze for low adoption rates and drive specific actions to address those adoption issues but more importantly we got to refocus how we prioritize AI Investments going forward today your Enterprise is focused around running individual AI Pilots to

enable productivity within their teams instead start to refocus new AI Investments to align with the Strategic priorities of your organization whether it's profitable growth or customer satisfaction this will enable you to manage AI not as individual Investments but as portfolio of projects Gartner research shows that organizations where AI Investments are managed as portfolios of projects are 2.4 times more likely to reach mature levels of AI see just with any Tech Investments we can't run away from the inevitable costs associated with AI but if we are diligent in how we Monitor and manage these new and

unknown costs we can navigate our Enterprise through cost overruns and after you've figured out the cost issue the second stall you're going to run into is the misuse of AI in various decision-making processes and the problem is really a matter of pacing the organization let me explain over the next few years you will see AI play a bigger and bigger role as a decision maker in your organization the issue isn't so much where to use AI as much as how much you use AI in your decision-making processes how much power you give to Ai and

to prove it to you I think it helps to illuminate what AI really is to show you the danger that can come from giving it too much power too soon now for those of you who are just getting started on your AI Journey consider this your cheat sheet on AI and for those of you who are maybe a little further along this might help you you explain AI to others in your organization in really simple terms and let me start by saying AI is not intelligent it's humanlike not human one of our AI analysts Mark

McDonald who's probably hiding in the back somewhere there has a really humorous way of proving this he asked Chad GPT how long it would take to mow his lawn given a certain size lawn mower size Etc the response over 5 years well so what is AI really then well it's an umbrella term for a variety of statistical techniques or the simpler version AI is pattern recognition it includes all the terms you've probably heard things like natural language processing large language models and so on but for most Enterprises the two techniques we care most about are

machine learning and generative AI now machine learning is the pattern recognition engine for most AI applications take a massive data set ask the machine to find patterns in the data and there you have it you're doing machine learning and patterns are useful because we can do two things with patterns first I can predict the next entry in a pattern at some point in your early school years you probably encounter the Fibonacci sequence that you see here in the upper right if you can f figure out the pattern and identify the next correct entry well you've

just done a very simple AI forecast now the other thing you can do with it is identify things that don't fit the pattern so what you're see here in the upper left an approach that data scientists like to call anomaly and error detection but in finance we don't see anomaly we use words like suspicious invoice DNA fraud duplicate payments Etc and finally generative AI the current star of the AI show that's the process of asking a computer to produce something a piece of text video image music based on the characteristics of the pattern that machine

learning identifies now here's the catch AI any AI whether it's gen AI or machine learning only knows what it's been trained on basic logic processes simple ethics your company policy unless the AI has been explicitly trained on it it won't know it this is important and keep this in mind when we discuss the third stall in a moment but remember AI is humanik it's not human so think of AI as a fancy Matrix more than an intelligent operator at least for now so then an interesting Dynamic you will need to manage is how much power

you give these fancy math tricks and there's a spectrum of how AI can be used in decision making but since it's Monday morning we'll keep it simple and just show you three basic ways that you can use AI now for simple decisions you can completely automate them with AI the AI identifies the need for a decision determines what to do and actions it for slightly more complex decisions you might say well we'll augment the decision maker the AI Flags a decision proposes a solution but the ultimate Final Call is still in the hands of the

human and finally there's decision support for your most complex decisions where we plug AI into our current analytics excuse me our current decision-making processes as an Analytics tool the need for a decision the data sources and Analysis parameters and the final decision all made by the human AI provides the analysis engine now the simplest way to think about where we as an Enterprise will go wrong is moving too fast you won't often hear me say this and this is probably the only time I will but in some cases it's our job as Finance to slow

down the Enterprise but in others we got to speeden them up think about the CFO as the Enterprise Cox the person at the front of the boat po pacing or Strokes for the rowers we haven't set the direction we are just spacing the rowers to get there and there's a natural maturity progression from support to augmentation to automation with nearly any AI use case most of our Enterprise colleagues what they are excited about are all the benefits of Automation and they will likely overestimate ai's intelligence they will want to go to the automation solution right

away instead of a trial period using more of a decision support or augmentation approach so our role here is to be the pace Setter for the organization's investments in AI because automating decision making too fast can lead to poor decision making I mean just ask McDonald out there who's probably still there Mo as long now for those of you that are just getting started push your executive peers to structure their business cases around decision support and augmentation AI is complex it takes time for folks to learn how the models work how to use them and

how to use them excuse me use them in decisions and for those of you that are a little further on your journey develop a process to periodically review the performance of your decision automation tools AI has to be tinkered with and retrained on a regular basis and that will affect the performance of these decision automation tools but also like Nisha said feel free to surprise your colleagues by being a CFO that pushes your colleagues to invest faster where you think it makes sense some of you will have overly conservative leaders who need a little push

to get more out of your AI Investments but again it's really about pacing the organization and how quickly we give AI power in our organizations slower sometimes faster at other times and remember to avoid AI being misused in decisionmaking your role is to be the Enterprise coxen so remember a few moments ago where I mentioned AI only knows what it's been trained on well that is one of the reason why we may hit our third stall which is loss of external trust this is trust with our customers invest regulator shareholders resulting from an AI interaction

as CFOs we have an important responsibility to our external Partners we are the primary point of contact for our shareholders we communicate the company's strategic objectives and keep our board honest on any risks that will impact our strategy and so it's important we ensure that the Investments our companies are are making does not break the Decades of trust that we have built with our external partners and yes while AI provides us with this amazing New Opportunities it's also quickly transforming our business models we are moving from an era on what machines can do for us

to What machines can be for us in the future AI is not going to be just a coworker for for our internal operations but AI will be the face of your organization they are going to interact with our customers suppliers financial institutions are shareholders our business models are rapidly changing from B2B and B to C we will see business models transformed to m2b M Toc and even M2M where machines start interacting with other machines but there are some key failure points with this Evolution you should know about where the data that AI is using to

interact with external parties is biased or insecure where our model is not updated to reflect current regulations or where our people are not skilled to explain AI results to our customers for example if you're a financial institution and your AI has denied a loan to a potential applicant can your employee explain why that loan was denied to ensure it was free of bias and discrimination and to make this point I want to share a fun example from February of this year a passenger used an airline's chatbot to research the airlines policy on bereavement fairs well

the chatbot responded that the passenger could apply for bereavement fairs retroactively here's the problem that wasn't the company's policy the chatbot did what it was trained to do it created a reasonable text response but it hadn't been trained on the airline policy so it didn't know what it was now of course the customer went and pursued that uh that refund retroactively and the airline refused to honor it stating it was inconsistent with airline policy the passenger took this issue to court and actually won the case the court deciding the case stated that it didn't matter

whether the information came from the company website or a chat bot the company was liable for negligent misrepresentation so I think it sets two interesting precedents for us first that we are liable for the actions of our AI employees and second our customers might actually like using chat Bots now see we too can be in situations where our AI is interacting directly with our customers where the information it's sharing it directly to our investors and we cannot control that this can lead to AI actions that are contrary to our company policy providing more and more

opportunities for external risk and as your other executive peers are ramping up AI Investments hiring new digital leaders it falls on us CFOs to collaborate with other Executives to drive governance around our AI Investments to ensure sure they're acting correctly ethically and compliantly and we do that by championing an Enterprise AI Assurance framework a framework that routinely scans our AI Investments for any data model human failure modes that Clem just talked about before your AI interacts with our stakeholders and using this framework all leaders across your Enterprise can evaluate both new and ongoing AI Investments

not just on a cost to Roi basis but also to assess any risk of data model or human failure modes that will erode external trust and it's not a matter of whether you will find failure points which you will what matters is our ability to spot them and take appropriate actions to mitigate them before they interact with your external stakeholders that's what's important here so for any AI Investments that have a high risk of failures investment should be paused till mitigation is complete or proceed but with caution for Investments where there's a lower risk or

risk is contained and where AI Investments can proceed as the risk for losing external Trust is minimal see as mentioned earlier AI is moving and changing real fast it's impossible to get to a zero risk environment due to the vast scope of AI across our Enterprise but having this assurance framework that your Business Leaders can leverage will help us Monitor and mitigate any risk of external trust as we continue to SC AI across our Enterprise so to this point then we've got a better handle on how to manage AI costs how much to use Ai

and how to drisk AI but now we've got to contend with a new foe people you're probably already thinking about the skill sets you need for AI but we want to press upon you the criticality of the mindset you need for AI and before talking about AI specifically here I want to ask the room a question how many of you would say that your teams are excited to change again and before you raise your hand just know that if you do I'm going to introduce you to the Erp vendors outside because it's clearly time for

an upgrade but I bet for a lot of you the work environment has felt more chaotic and you wouldn't be wrong for thinking so the number of change events has skyrocketed our finance staff are exhausted and then when you add AI in the mix our employees across the Enterprise they are now afraid we have all been through a pandemic and what feels like a new Global crisis every quarter three years straight and every news and media outlet from Wall Street Journal to your local channel and even John Stewart on The Daily Show they're all running

stories about how everyone is going to lose their jobs to AI and that's a problem because fear and exhaustion creates change resistance we have or shortly will have new AI co-workers and fearful people are rarely welcoming of new co-workers and yes we said this earlier that AI yes it does has its limitations but it's also better at certain tasks than we are and don't underestimate the psychological impact of that that's what makes AI so frightening not that it can do some of the things that we do but it can do it better than us and

some of these things are things that our team like to do or maybe they're comfortable doing or at least it's familiar to them and ignoring this type of issue is a mistake that some teams have made in the past a few years ago I was living and working in London and at that time a number of our European clients were going through a process of converting business controllers into true business partners the CFOs had found ways to automate and centralize the parts of the role that employees considered boring or low value creating reports managing accounts

Etc so the employees were then sent off to influence business decisions instead of doing these things but 12 months later the CFOs found that the finance business partners were still creating reports or helping out on managing accounts even when they knew the work was quite literally redundant and it baffled a lot of the CFOs they' given their employees what they asked for more interesting rewarding work why were they still doing the old boring stuff but from the business partner perspective it actually made sense see at the end of the year their manager would ask some

form of the question what was your contribution so I can assess your performance and the finance business partners knew that it was hard to Tang ibly show and measure influence or impact on decision making but they could always point to a report they created or a set of accounts they managed see what the M this mistake the CFOs made was that while they told their employees what they wanted them to stop doing they didn't tell them what they wanted them to start doing or provide any support structures for new ways of working AI will cause

this exact same problem but on a massive scale across our Enterprise and it can stall our investments if we don't invest in the mindset of our organization resistance from employees will likely grind our progress to a halt consider your marketing manager that's distinguished their career on being able to create great ad copy when we evaluate investments in AI we need to consider how prepared our marketing leader is for the human change impact within their teams and there are two easy ways we can go about it first we revisit the kind of questions we ask as

we're evaluating AI Investments start by making a simple switch in the criteria you would use by including human change impact and let me give you a few examples here of what Nisha and I really mean Beyond asking some version of what are we trying to achieve with this technology also ask what will happen as we implement this AI both to the specific process process impacted and any related processes and rather than just asking is the tool easy to use also ask your colleagues how will your staff react to the use of AI and how are

you preparing for their response and finally remember when we said we need an assurance framework well an easy way to start down that path is instead of asking you know what are the main risks how do we prevent those risks recognize that there is no zero risk so also ask how do we class classify the types of risk that this AI might produce and what is our plan to recover and learn from risk events the key here is that in addition to getting a better sense of where mindset might limit the success of our investments

we're also signaling to The Wider Enterprise that the impact on humans from AI adoption has not been overlooked yes there are economic realities guiding these tough decisions but that doesn't mean that we should ignore the impact on humans now the second thing we can do differently is ask new questions of ourselves as Leaders don't forget this mindset shift starts with everyone in this room here think of it this way if you want the quickest simplest way to differentiate yourself as a popular leader in your organization start learning Ai and model new ways of working by

2026 we estimate that 50% of regretted staff attrition will be caused by a digital literacy gap between leaders and their staff so start asking questions if your marketing leader can explain the frustrations of prompt engineering with chat GPT whether your fbna leader can empathize with the frustrations of testing a machine learning forecasting model and also ask that of yourself because your staff will expect you to know that see we all know how difficult it is to be managed by someone who can't empathize with the challenges of our role hey I know I'm not qualified to

give Tick Tock dance lessons to my kids and neither would they want that from me but our research has shown that early adopters of AI the one of the key differentiators was tasking and analytically Savvy leader to take charge someone who understood the complexity and challenges of AI related tasks and when we have Enterprise leaders who can understand empathize and drive AI investment decisions considering the human impact we'll have found that our staff are more open to working alongside AI to get the best best outcomes for our organization we'd have turned mindset from a liability

to an asset so as you reflect on these four stalls this morning remember that these stalls are not technical problems for it to work out they're organizational challenges that you all can guide your Enterprise through these stalls are solved by planning for hidden costs pacing Investments to manage risk championing AI assurance and asking new questions of yourselves and others things that you're already in your wheelhouse things you're ready to help your Enterprise with while our colleagues are focused on all the technical challenges of AI CFOs and their teams can ensure the organization is ready to

use AI in the right ways as we said at the beginning we're at the dawn of a new era an era where AI will shape not just how we operate as Finance but how we operate as an Enterprise an era of tremendous opportunity if we can navigate these stalls in our path in the coming months you'll be in board meetings communicating your Enterprise AI Investments you might be contending with an Erp upgrade and call Nisha for that one not me you'll be chasing a million different use cases you'll be worrying over technical challenges and who

knows what the next global economic challenge is there's going to be a lot of distractions but if you can make sure that your Enterprise doesn't lose focus on these four AI stalls then you will be a key reason why your Enterprise is thriving tomorrow and in 2030 thank you everyone and have a wonderful conference [Music]