The Challenge of Mobile App Creative Testing

In the world of performance marketing, ad creatives are the cornerstone of effective advertising. They are the primary vehicle for conveying an advertiser’s message and persuading potential players or customers to take action. While technical elements such as targeting, campaign structure, and budgeting play vital roles, it is the creative aspect that has the most significant influence on audience perception and motivation.This article explores the foundational role of ad creatives, the common challenges faced in their optimization, and effective solutions to overcome these hurdles and achieve sustained marketing success.

Creative, Testing & Iteration: Ad Creatives are foundational to marketing – they are the medium through which an advertiser’s message/objective is conveyed. Ad creatives carry the weight of impressing, enticing and/or persuading potential players/customers to take action. Other more technical aspects of performance marketing – targeting (now mostly AI & ML), campaign structure, event optimization, budgeting – all still influence campaign performance in very important, but less clear ways than creative, which remains the most crucial element in influencing the perception/motivation of prospective players. Given the major impact that creative can have on performance throughout the acquisition/retention funnels, it is no surprise that many of the most successful game developers have a rigorous creative production & testing strategy that yields hundreds or even thousands of creatives in a year per game.

Creative Ambiguity & Fatigue: Despite the significance of ad creative, many advertisers find themselves uncertain about what value creatives add to their campaigns and/or they find themselves unable to improve marketing performance via creatives. Many advertisers have experienced difficulties mitigating persistent declines of formerly top performing creatives, and have found that not addressing creative fatigue takes a toll on KPI performance. After many experiences with inconclusive test results, ads in decline and/or new creative batches that simply don’t get ‘traction’, it is understandable that an advertiser might want to deprioritize new creative orders and tests.

The Challenge: Machine learning algorithms power most mobile app advertising channels these days and while they have their benefits, they also have their drawbacks, especially when it comes to unambiguous creative testing. These ML-based algorithms perform better as the quantity & quality of their ‘learning’ dataset scales. Since most ML advertising channels accumulate ‘learning’ across a couple or multiple entity levels (Account, App, OS, Audience, Placement, Campaign, Ad group and Creative), ML-based algorithms will inevitably favor those entity levels that they have more data associated with (this is a general explanation for why it can be difficult to get sufficient spend/learnings on newly introduced creatives). While at least one component of an advertising algorithm (which varies greatly by channel) does ‘test’ creatives against each other, often looking at predicted (or real) conversion rates or upper-funnel indicators (or combinations of these), additional factors in the algorithm more often than not outweigh the creative portions – usually in pursuit of a performance KPI, impression fill rate or daily spend / budget pace. As a result, many advertisers experience difficulties getting sufficient traction on new creatives, and typically interpret this as a failure of the new creatives themselves.

Solutions: Separating your creative tests & new creative launches from your primary UA campaigns can be an effective way of sidestepping the difficulties of getting a sufficient amount of traffic to new creatives. This is usually achieved in a separate campaign and/or account. While this often goes against the “best practices” of the advertising platforms, if done tactically it can yield actionable learnings and inspire confidence in your creative testing methods, with a minimal or even negligible effect on your UA campaigns’ performance. Utilize dedicated creative testing tools like Meta’s A/B Tool in Ads Manager, and commit to investing the necessary minimum spend and time to get statistically significant results. Additionally, leveraging a third party like Kaizen Ad to create, run & test ad units distinct from your own is another effective way of circumventing the ML creative testing issue. 

Last but not least, understand that (creative) testing is an ongoing process, and requires many rounds of experimentation and evaluation to yield results.

-------------------------------------

[1] MobileAction observed that top games circulated thousands to tens of thousands of unique creatives in 2023 – Mobile Ad Trends Report 2023

[2] 4 Data-Driven Strategies to Optimize Your Paid UA Campaigns - Find and Combat Creative Fatigue – Andre Kempe, CEO, Admiral Media

[3] “Black box”. There’s no way to determine how the algorithm came to your decision. – Oscar Garcia, Towards Data Science