To Upgrade Apps, Listen to Users

When adding features to software applications, developers should balance user feedback with their own instincts

Based on the research of Anitesh Barua and Ashish Agarwal

How do apps improve? For some of today’s most popular applications, it’s by listening to their customers.

• Instagram responded to requests for in-app editing tools by offering filters, brightness, and contrast adjustments.
• Offline maps, by Google Maps, answered users who wanted to use the tool when they couldn’t get online.

But listening to user feedback isn’t an easy task. The Apple App Store alone offers 3.8 million apps with as many as 1.8 million reviews apiece.

New research from Texas McCombs offers insight into whom and what to listen to. It uses artificial intelligence to analyze user reviews and their influence on app updates. The results suggest when developers should rely on users for adding new features and when it’s better to follow their own ideas.

“Firms can obtain product feature ideas from customers, who may have a better understanding of their own needs,” says Anitesh Barua, a professor and chair of the Department of Information, Risk, and Operations management (IROM). “However, the effectiveness of user-suggested features has not been studied in the literature.”

Such features generally fall into two categories, he says: imitations of features from competing apps and innovations of features competitors don’t offer.

With Ashish Agarwal, associate professor of IROM, and  former doctoral student Aditya Karanam of the National University of Singapore, Barua looked at innovations and imitations that were initiated by developers and those proposed by users.

Using a sample of 853 top-rated Apple apps from 2012 to 2016, they used AI to sift out suggestions from 7 million reviews. They adapted a language model called BERT to extract features, based on sequences of words used to describe features in version release notes and user reviews.

In trial runs, their model outperformed the better-known GPT-4 in identifying user-suggested and developer-initiated features, especially with limited data.

The researchers then analyzed each app and its updates to sort out which feature ideas came from users and which from developers, and whether these features were imitations of other apps. Finally, they estimated which new features had the biggest effects on demand for apps, as measured by consumer rankings and demand for apps.

They found that the value of user suggestions depended on the kind of feature:

• For innovations — features that don’t imitate other apps — those that came from developers boosted demand. Those that came from customers reduced it.

• For imitations, on the other hand, the effects were reversed. User-suggested features increased demand, while developer-initiated features did not.

Why the difference? The researchers suggest that customers may be better at describing features they’ve seen elsewhere than ones they’ve never seen.

App developers, Barua says, can use the results to help determine when to heed their customers — and when not to.

“For adding innovative features, developers should follow their own instincts,” he says, “unless they are able to fully comprehend user suggestions, which are often ambiguous, and which can do more harm than good.”

When it comes to imitating other apps, however, the customer is more often right, he adds. “In such cases, they should listen to users instead of their own inclinations.”

“Follow Your Heart or Listen to Users? The Case of Mobile App Design” is forthcoming in Information Systems Research.

Story by Hope Reese