Real-WorldT Tips For Ensuring Digital ROI

Hindsight may be 20/20, but so is knowing when to cut your losses before investing too much money, time and effort on a project that won’t achieve the desired ROI.

That’s what Toronto-based real estate services and investment management firm Collier International has learned. Rather than committing to new projects based on traditional requirements gathering and early estimations, the global IT group starts with short sprints designed to validate a business idea, the technical solution and the potential business benefits, says Mihai Strusievici, vice president of global IT.

“The incremental sprints approach provides cost certainty and a go/no-go decision point between sprints,’’ he says. “This way, the development team learns as it goes at a known burn rate and has the opportunity to stop before too much effort is spent on something potentially not useful.”

With the pressure on organizations to roll out digital initiatives quickly, it’s not surprising that sometimes progress gets stalled and outcomes don’t meet expectations. Even when IT leaders have estimated and measured what ROI should come from a project, often they have to pivot and make changes to maximize their investment.

The average digital transformation stands a 45 percent chance of delivering less profit than expected, while the likelihood of surpassing profit expectations, on average, is just one in 10, according to McKinsey.

This is often the case simply because you don’t know what you don’t know, industry leaders say. The key is to set tangible goals and establish practices that ensure clarity and communication every step along the way. Whether your organization seeks to grow by 30 percent, expand its market to another continent, or increase its number of customers, you need to have — and communicate — key indicators for each goal, says M. Nadia Vincent, a digital transformation and AI executive advisor.

“Then you have to have milestones by which you want to achieve those metrics,’’ she adds. “Then monitor what’s happening with any project management tool. It’s the strategy that matters — not the tool.”

Only by measuring progress can you determine whether you are meeting your target, need to raise your level of activity, or change your strategy altogether, Vincent says.

“Seeing changes in revenue or margins is one litmus test for success, but companies should also regularly measure the customer and employee experience to ensure the technologies are having the right impact,’’ agrees Tom Puthiyamadam, global digital and BXT leader for PwC.

Following are lessons organizations gleaned as they measured the progress of their digital initiatives. The common denominator? Speed and an agile approach to development.

Learning to pivot

McKinsey research shows that 66 percent of software projects go over the budget, largely due to unclear objectives and shifting requirements, Colliers’ Strusievici notes.

“In other words, in the context of moving targets, the initial estimate is more of a guess than science, a fact described by the Cone of Uncertainty Theory developed by Barry Boehm in the early 1980s,’’ he says. After all, if your end product is not well defined, “an initial estimate is impossible.”

To avoid this pitfall, Colliers global IT takes a measured approach that involves clarifying the goal as the project evolves, along with the opportunity to stop before officials commit too much money and time, Strusievici says.

“We start by investing in a functional prototype that allows us to get feedback from real users,” he says. “As we learn, we either pivot … or stop, saving the effort that would have been wasted in something potentially useless.”

Changing direction, or pivoting, is a regular occurrence in all projects, and is a result of learning and adapting to new information, he says. But in the traditional approach, pivot equals “change request,” which usually means increased cost, Strusievici says.

“In an agile, incremental process, the pivot is expected, even celebrated, and the team is ready to make decisions without the pressure of staying on a predetermined course,” he says.

The waterfall methodology has its merits when you know what you’re going to do, he adds, because it is predictable and reduces ambiguity. But a lot of digital transformations are experiments, and organizations don’t know if they will resonate with users.

“What we learned over the years is that we need to figure out very early the context on which we operate,” Strusievici says. “If it is experimental, unclear [or] aspirational, we need to adopt an incremental, agile approach and be ready to pivot as we learn. Not every digital initiative has to [be completed] and stopping early is as valuable as completing on time.”

The real magic of measurement

When Toyota Financial Services decided to get into the private car financing business, IT was tasked with developing a new technology stack comprised of more than 15 systems in less than six months to service Mazda, its first client. Officials opted for a “speed of decision-making” approach using an agile-inspired methodology, says Amit Shroff, vice president, office of the CIO, at Toyota Financial Services.

“With any system there’s a process, the business objective and how do I digitize that process, which requires people from a multidisciplinary effort,’’ he says. “Our idea was really to focus on breaking that up into areas of expertise and focusing more on what decisions need to be made.”

The business and IT “squads” met daily or multiple times a week to deal with any project impediments. Senior management met with the squads at least weekly to inject a sense of urgency around transparency on the issues, Shroff says.

“What we realized is there is really a need for … a level of transparency to information. Nothing else matters.” That way, decisions get made on how to proceed more quickly, he says.

There also has to be a “hyper focus” on what is a hurdle that can’t be solved by the team so it can be elevated to the next level. “That’s huge. Very few companies do that,’’ he says. “Once you have that transparency — and it requires some level of confidence and risk taking — then you have people who can … reallocate resources or substitute if someone doesn’t have right expertise.”

People tend to have a microfocus on achieving ROI, Shroff says. But the key is to have “a confidence level on the return and purpose of the return. That’s the magic. What is the purpose and why are you doing the digital transformation. If it’s just purely financial, why do it?”

Most projects focus on time and money, he adds. “We’re saying if you focus on moving these things forward, the time and money will be a good outcome,’’ Shroff says. “We can manage time itself, and the money you can manage by moving quicker.”

Measuring customer feedback and social media ratings

Customer-facing businesses care deeply about user ratings. NJ Transit deployed an app a couple of years ago that was “not customer friendly or digital friendly,’’ says Lookman Fazal, chief information and digital officer for the transit system, the nation’s third largest agency, which transports one million people a day.

IT added what Fazal says was missing functionality: integration with Apple Pay so users could more seamlessly purchase tickets, and a simpler user interface that wasn’t “small and ugly,” he says. Several other features were also added, including notifying riders about the location of trains and buses and letting them save their favorite routes.

Instead of pushing the app out to its 800,000 customers right away, Fazal says they deployed it early to internal testers, then gave a “sneak peek” to the digital community for testing and feedback purposes.

IT used the agile methodology and conducted short sprints, he says. “Now, we could address people concerns and issues in a week to two weeks rather than months.”

The resulting feedback was a “whole pipeline of requests” once IT has rolled out the native apps for Android and iPhone as opposed to having a single app, Fazal says.

IT measured the apps’ ratings along with feedback from riders and how fast people were interested in downloading either version. Roughly 70 percent have so far downloaded the iPhone app while 30 percent have downloaded the Android one, he says.

The ROI ultimately met what IT measured, Fazal says.

“The ROI was not from a dollar perspective, but we did recover development costs … it was more on the ratings,’’ he explains. “We didn’t want to roll out any app that would give us less than a 4 rating [out of 5].”

The proof was in the pudding. “We exceeded it: It’s at 4.7” in the Apple store, compared to the previous 2.3 rating for the old app, he says.

Measuring manual labor and user satisfaction

When you’re in the business of helping children, the last thing you want is for employees to have to spend time away from their core mission. New York Foundling, one of New York City’s oldest and largest child welfare agencies, found that employees were spending too much time on repetitive, data entry because state and local systems were not integrated. That led to a 42 percent turnover rate in 2017, says Arik Hill, CIO of New York Foundling.

The agency collects about five million attributes on clients each year and staff were spending too much time inputting data into five disparate systems, which also created inconsistencies, and led to significant delays in communication, Hill says.

A survey revealed that social services staff were spending up to four hours a week copying and pasting data into these systems. “This directly impacted the job retention and satisfaction rate,’’ Hill says.

The decision was made to invest in a robotic process automation (RPA) system, utilizing an agile approach, forming teams comprised of clinical users, supervisors and technical staff, he says.

The teams went through a series of agile development cycles and came up with “oversight activities” to ensure the quality of data going through the systems was being automated  correctly.

IT knew staff were spending four hours a week doing data entry so the goal was to eliminate that completely, Hill says. After an action was completed by a bot during a supervised robot development cycle, the team received a response back.

One of the issues that cropped up during development was lag time in the city’s system for inputting a new client, which impeded the bot’s ability to record transactions. That meant staff still had to manually input information.

IT monitored how long it took the bot to complete a transaction, as well as how many transactions the bot couldn’t complete. Hill says officials have been tuning the timeframes so that 100 percent of the transactions flow through without human intervention. The goal is for the system to run in near real-time.

Staff productivity was also measured. Today, staff turnover has decreased to 18 percent, Hill says.

With the RPA system, “you have to have a good grasp of your baseline metrics in knowing where to target the tool,’’ he says. “You can do a lot of things, but there are certain areas that are overly complex that don’t lend themselves to automation. You want things with a predictable outcome that are constantly happening.”

Measuring agent backlogs, customer satisfaction

The focal point of digital transformation at Liberty Mutual Insurance, the country’s fifth largest personal insurance carrier, has been moving from traditional IT metrics and a waterfall approach to more business-focused metrics, says Andrew Palmer, CIO of U.S. retail markets.

“We used to size 1,000 projects and hope we got business results,” he says. “Now, it’s a more iterative process” that involves teams comprised of business users and IT.

One such digital project involved creating a capability for customers to text photos to agents to start the claims process instead of them calling into the call center.

The agile approach included incremental development that considered feedback from agents on what would make their lives easier, Palmer says. “Along the way, we made dozens of pivots in terms of what we thought would be effective versus what resonated” with the agents, rather than hearing from business analysts speaking on their behalf, he says.

“It helped us get much closer to customers and prioritize backlog,’’ he says. By doing “dozens of releases a day” now, the software being built is simpler and more usable, he says.

“If we had done that project two years ago we might have done it over seven months as a big-bang release among agents and it may not have met the need,’’ Palmer says.

It’s one thing to have a hypothesis that a customer would want to take a picture of a car after an accident and text it to their agent, he notes. Like the others, Palmer says the way to deal with an unknown is to conduct a pilot test and pick some metrics to measure before building out all the infrastructure.

“In the past, we’d say we’d have to ingest all these images and spend time and money on putting that project into the market without knowing,” he says.

Now, with a modern architecture, Palmer says instead of doing maybe 12 software releases a year, IT does several a month using “micro tests” to hone in on the customer experience. All customer feedback is fed into a Slack channel so project teams can react accordingly, he says.

“How we accept risk and have teams win or lose together is part of building that strong business-IT relationship,’’ Palmer says. If a team works on something for a week and finds it has to regroup, “it’s not a big deal.” That is also the case if something doesn’t perform when it is put in production, he adds.  

“We’re much more comfortable stopping projects if something’s not working big and small — it’s no longer the failure of the team, it’s just that we need to reallocate resources in a different direction,’’ he says.

Whereas people used to get attached to projects and outcomes done over a long period, now it’s easier to redirect teams to different business objectives, he says.

“It’s about trying to align teams to business outcomes and creating a risk-tolerant environment,” and adopting the agile principles while adapting the practices. Palmer says the biggest switch has been moving from a “one-size-fits-all mindset” to one of a “test and learn and pivot mindset.”

Liberty Mutual’s U.S. Personal Insurance Technology team has seen employee engagement rise over 50 points as measured by Net Promoter Score, he says. Delivery productivity of new capabilities has improved by 30 percent, Palmer adds.