TBThumbBlitzOpen the tool
GuideUpdated October 28, 2026·9 min read

Roblox Thumbnail A/B Testing— what the official test does, what it will not tell you, and how to feed it five variants that mean something

Roblox shipped thumbnail personalization in November 2024. It is not experiment software with a start and a stop button: once you mark two or more Home page thumbnails as active, the system starts serving different images to different players and reallocating impressions every hour.

Roblox reported an average lift of +8.5% in qualified play through rate during testing, with some experiences seeing around +50%. In a later staff update the platform-wide figure was+12% relative lift on average, up to +56%, across more than 8,000 experiences that had adopted the feature. Those numbers come from Roblox, not from us, and they describe the population, not your game.

Most personalization setups still produce nothing useful. The reasons are consistent: the five images are too similar to separate, the set gets replaced before the model settles, or the developer reads a number that moves for reasons unrelated to the thumbnail.

Where the feature lives

In the Creator Dashboard, pick the experience, then Configure ⟩ Places, choose a place, open Thumbnails, and switch to the Home Page tab. That tab holds the images the system personalizes. The Experience Detail Page tab holds the carousel players see on your game's page, and those images are not part of the personalization run.

Personalization only starts at two active thumbnails and accepts up tofive. Uploads need to stay under 3 MB and at 1920×1080; the site displays thumbnails at 16:9 and will stretch an upload that arrives in another aspect ratio. If only one thumbnail is active, every player sees that one and nothing is being tested.

It is personalization, not a classic A/B test

A conventional A/B test splits traffic evenly, holds everything else constant, and reports one winner at the end. Roblox's system does something different. It shows each thumbnail to random groups to collect data, then gives more homepage impressions to whichever thumbnail is winning for each user group, while leaving the others a small amount of traffic so it can keep exploring. The reallocation runs hourly, so the leader can change as player preferences shift.

The metric being optimized is qualified play through rate (qPTR) — the share of users who, after seeing your thumbnail on Home, join the experience and then stay engaged for a significant amount of time. It rewards a thumbnail that brings in players who actually want to play. A thumbnail that wins clicks and loses players is a bad thumbnail that happens to look good.

What the dashboard does show you

The performance table fills in a few hours after you start, and it has more in it than "no data at all": impressions, qualified plays, average playtime, qPTR, and a Winning Segment column — the thumbnail with the highest play through rate during the current personalization process. Two graphs sit underneath: impressions by thumbnail and qPTR by thumbnail over time.

So you can see who is ahead. What you do not get is a click-through table or a statistically declared result. The winner is reported per audience segment rather than as a single verdict for the whole game, and it is a snapshot of a running process, not a closed experiment. Treat it as a direction, not a trophy.

The 7-day rule

Roblox staff guidance is explicit: wait at least seven days, or until your next content update, before changing the set. Frequent restarts stop the model from identifying the best performing thumbnails. The recommendation attached to that is to test new thumbnails alongside a real content update, then leave them alone until the next one.

The mechanics are simple. The system learns per-segment preferences from accumulated impressions. Replace the set and that learning starts over. A set that ran four days and got swapped did not fail; it never finished. Low-traffic experiences need longer than a week to collect enough qualified plays to compare anything.

Five thumbnails, or one thumbnail five times?

This is where most solo developers stall. Staff advice is to keep as many active variants as possible, but five thumbnails that differ by a text strip are one thumbnail wearing five hats. The system cannot separate images that say nearly the same thing, and you end up paying a week for a result you could have guessed.

Roblox's own examples point at spreading the set across different gameplay elements, characters, environments, and player motivations (competition, creativity, social). A practical way to check your set before uploading: describe each image in one sentence. If two sentences come out identical, you have four variants, not five.

For an obby, five honest variants might read like this: the course from a distance, the character mid-jump over a gap, the same jump with a three-word hook, a close-up of a new obstacle, and the opening shot under night lighting. Nobody needs a design degree to tell those apart.

One more constraint worth designing around: the player count and other metadata can sit over the bottom of a Home thumbnail, so keep essential text and faces away from that edge.

Reading results without fooling yourself

Mistakes that burn a cycle

Turning one scene into a set of five

The leverage is not in the dashboard. It is in the material you upload. If producing five different thumbnails takes an evening in an image editor, the test does not happen.

The ThumbBlitz thumbnail maker is built for that step: take one screenshot, then export several versions at 1920×1080 — headline on, headline off, different backdrop, tighter crop on the action. Exports happen one at a time and stay under the 3 MB upload limit, so filling five slots is five short passes rather than a session in a graphics program. Roblox stretches anything that is not 16:9, and the editor exports at that ratio.

Three quick variants worth trying first: the same frame with a three-word headline, the same frame with no text at all, and a crop that pushes the character to one side with space for a scene behind. Same game, same quality bar, three genuinely different first impressions.

What YouTube does differently

YouTube's manual A/B testing tool lets a creator attach up to three title or thumbnail variants to one video and declares a winner by watch time after a run of up to two weeks. In September 2026 YouTube announced dynamic thumbnails, which work closer to Roblox's model: several images matched to different audience segments, with no single winner for everyone. Both approaches assume the same thing — you can only learn from variants that actually differ.

One habit does not transfer. On YouTube you can read click-through rate and watch time per variant. On Roblox the useful signal is qPTR and the segment leader, and both of them move for reasons outside the thumbnail.

The short version

Roblox gives you five slots and an automated allocator that retunes itself every hour. The work is upstream: five images people would describe differently, a full week of not touching them, and the discipline to read qualified play through rate instead of vibes.

When to use it

When this is worth your time

Personalization needs enough homepage traffic to learn from. It is a good fit once your experience already gets consistent Home impressions, and a poor fit for a game nobody sees yet — no amount of thumbnail variation fixes zero impressions.

  • ▸The game gets Home page impressions but the click-to-play step underperforms.
  • ▸You have a content update coming and can pair it with a new thumbnail set.
  • ▸Your thumbnail has not changed in months while the game itself has.
  • ▸You want to know which player segment responds to which image.
How it works

The workflow, one line each

All of it happens between the Creator Dashboard and a browser tab.

  1. 1Open Thumbnails ⟩ Home Page in the Creator Dashboard for the right place.
  2. 2Build five images that differ by composition, not by wording.
  3. 3Keep each file under 3 MB at 1920×1080.
  4. 4Activate all five, then leave them for at least seven days or until the next update.
  5. 5Read qPTR and the Winning Segment column by week, not by day.
  6. 6Log the change and the direction it moved before starting the next cycle.
Limits

What it will not do

These are the boundaries worth knowing before you plan around the feature.

  • !It does not split traffic evenly and it does not close with a significance test.
  • !It reports a segment leader, not a click-through table per thumbnail.
  • !It only covers Home page thumbnails; the detail page carousel is separate.
  • !Two active thumbnails minimum, five maximum, and one active means no test at all.
  • !Restarting the set resets what the system has learned.
FAQ
›Is Roblox thumbnail personalization free?

Yes. It is part of the Creator Dashboard for published experiences and there is no separate charge or subscription for using it. You still need an experience with thumbnails uploaded to the Home Page tab.

›How many thumbnails can I test at once?

Up to five active thumbnails on the Home page, and you need at least two for personalization to start. Roblox staff recommend keeping as many active as possible, because more variants give the model more to work with.

›Can I see which thumbnail won?

You can see who is ahead. The performance table has a Winning Segment column showing the thumbnail with the highest qualified play through rate during the current process, plus impressions, qualified plays, average playtime and qPTR per thumbnail. What you do not get is a click-through table or a single closed verdict, and the leader can change over time.

›How long should I wait before judging results?

At least seven days, or until your next content update. Roblox staff warn that frequent restarts prevent the model from accurately identifying the best performing thumbnails, and low-traffic experiences need longer than a week.

›What if all five thumbnails perform about the same?

Read that as a result rather than a failure. Either the images were too similar for the system to separate, or thumbnails are not the limiting factor for your audience right now. Next cycle, spread the concepts further apart and change one thing at a time.

›Does this apply to my game icon?

No. The icon is a separate setting with its own requirements, and it is not part of the Home page thumbnail personalization run. Treat icon work as its own project with its own images.

›Why did my qualified play through rate drop after a spike in impressions?

Roblox staff describe this as normal. A surge in homepage recommendations brings in many first-time players who are still exploring, which pulls qPTR down temporarily; it tends to stabilize once traffic returns to normal levels.