Same apps on top for years, new apps buried in days: how does Connect IQ ranking actually work?

There are three main things used for ordering.  Rating, #of download, and # of views (no download need and can just be social media). There could be other things, but Garmin keeps a tight lip on them, because in the past, some devs tried to scam the system.. If you ask me, "views" have too much influence.  It's been like this for years and has been posted about by many.

Some of this came about because new devs sometimes complained that long established and popular apps (say 1m download) were too visible.

It’s simple: they use a large number of accounts to post fake reviews. I’ve tried it myself. If I write 20–30 reviews a day, I can usually reach around 15th–20th place in the rankings. The developers at the top use automated scripts to generate roughly 80–100 or more reviews. When they first upload a watch face, they sometimes post over 200 reviews in the first few days.

I started out as a legitimate developer, but nobody can beat these guys.

However, as soon as I try to do the same thing, other developers report me and my account gets banned. Meanwhile, these developers—around four to eight of them—don’t seem to be subject to the rules. They’ve been posting fake reviews continuously for four or five years.

For comparison, you’ll notice that the number of reviews on these legitimate popular watch faces is roughly one-tenth, or even one-fiftieth, of theirs.

It’s like the elephant in the room. Garmin pretends not to see it, but cracks down on other people—possibly after reports from competitors. Their detection method is actually quite simple: if you use the same accounts to review multiple watch faces, you’ll attract attention. But these established developers have enough accounts to assign a separate set to each watch face.

How do I know so much about this? Because I’ve tested most of the methods. I’m also more popular than they are on social media, but when both my watch face and theirs have 1k+ downloads, theirs already has 500+ reviews, while a normal watch face has only 10–50.

The ranking has always depended solely on the number of reviews. Garmin is unwilling to disclose this, but if they checked their own code, they would see just how absurd their algorithm is.

It's probably not that clear.  For a time, some devs used to post in their app descriptions, things like "free unlock code for a 5 star review"  The result is they got a ton of reviews as well as downloads,  That was banned in the app store, if I recall, and I've not seen it in a couple yeras, but could still be happening from social media.

There's also now the ability for an app to ask the user if they want to leave a review.

The images you show likely don't show anything wrong.  I say that as a long time developer, who never begged for reviews, who has watch faces with up to about 750k downloads and a reasonable number of reviews. I do things a bit differently, in that I often say something like  "don't use reviews if you have a question or problem  - use contact developer so I can ask question, like which watch and FW"

What methods do you think could allow someone to receive 20 times as many reviews as other developers with the same number of downloads? I built a web scraper to monitor these developers’ work. With 750,000 downloads, do you think you could get 5,000 reviews? I find that hard to believe. You can check your own records.

I’ve also downloaded some of the top-ranked watch faces and tested whether they use the watch to call a review API. Unfortunately, I found no evidence that they do.

Before making this accusation, I did far more research than you have. I also have more than a thousand accounts that were registered in bulk. If you haven’t done these things yourself, you naturally may not be familiar with how they work.

For example, if one of your watch faces has reached 750,000 downloads, you’ll know that its popularity doesn’t necessarily carry over to your next watch face or app. Your next release might get only a few thousand downloads.

Yet almost every release from these top developers quickly reaches 10,000 or more downloads. When a watch face performs poorly and fails to attract enough downloads even while it remains near the top of the rankings, they abandon it. They artificially inflate its review count for only a few weeks. That’s why you often see watch faces where positive reviews amount to one-tenth, or even more than one-fifth, of their total downloads. For popular watch faces, however, they continue inflating the reviews for years.

Some apps currently have more than 144,000 and 50,000 reviews, respectively—and I’m talking about reviews, not downloads. Can you make sense of those numbers?

Another striking irregularity is how few of their reviews contain written comments compared with the number of reviews they gain. Even when one of these supposedly “popular” watch faces receives 50 to 200 new reviews in a day, fewer than 10 users leave a written comment.

By comparison, when I receive 20 reviews from real users, more than half typically include written comments. That makes the pattern on these watch faces particularly unusual.

I’ve been tracking the M4G from the first image in the comment above from 5965740… this app is more often than not in the overall trending and always in top-3 headlines in Navigation so gets a high number of eyeballs. And 1M+ downloads and 4.7*, boy this app must be amazing so what’s the deal…

1) It’s maps and that’s my job and I’m a cartography snob but this is really a piece of steaming something else, so why the self-perpetuating ratings keeping it at the top?

2) It’s a paid app, for some features, and according to the pretty clear T’s and C’s any app that needs payment to enable some features should have the [PAYMENT] flag set. The app gets more views because it’s not filtered out. Komoot do the same and others in the top-spots… Garmin don’t seem to care.

3) The reviews history is 95%+ made up of single-word comments, translated from many languages that all say “great” or “good”. I originally thought this was cheap labor positive review buying but, though it’s useless feedback I think it’s some phone automation when giving a good review and AI-type suggested auto response. So here’s the nub, why so many good reviews for a poor solution?

4) The answer is in the “clever” review-prompt adopted by this app which uses a background task to fire off an open-app which initiates the review process before you are in the app. The same works on first install; you can say “no” to the app terms and still get a review prompt before even seeing the solution in action. In reality this means the background review launch without any clear app-context makes people think they are reviewing the watch itself. The reviews that put some more words in support this with messages like “It’s an excellent device” — doesn’t sound like an app review to me.

So, just like the Garmin “Find My Car” app that hasn’t been updated in 3+ years and has millions of downloads and increasingly poor 2.7* rating…. But lots of reviews so stays at the high-end of the positioning, apps playing the system will be around forever self-fulfilling downloads and stagnating charts and development.

Who looses? Users who have poor insight into what might be valuable in the store and developers who can’t break into a rigged marketplace. Watch faces are pretty much already a no-go area.

As I said in a post last week on pay-per-app-submission, CIQ isn’t why people buy Garmin - but it might be a real deal in why they stay, being able to use that specific app on their next device they can’t get on a Coros etc… but if new users can’t get good apps then this brand retention upsell goes. It’s business Garmin, sort it out!

And Garmin seems to have completely overlooked the Matthew effect. Because these developers manipulate the system to gain more exposure, their watch faces also receive a large number of genuine downloads, which in turn bring in real reviews and installations—far more than new developers can get.

Meanwhile, Garmin expects new developers’ watch faces to earn more reviews than established watch faces whose download counts combine genuine installs with scripted ones—all while the newcomers are barely visible in the rankings. Only then do they have a chance of appearing near the top.

Hi!

Сould you please make your analytical tools publicly available so that others may find more similar deceivers?

With such an army of bots, you're obviously not afraid of losing them by publicly acknowledging their existence. Besides, if these bots are exposed, you'll likely be able to get new ones. With that said, have you tried fighting bots with bots? I think you have the opportunity to push them out just as an experiment. If Garmin ignores owners' bots of these listings they may also ignore yours.

You don’t need any special analytical tools to observe this. You can simply take a screenshot each day and track how the number of reviews changes. I also had AI write a tool to collect the data automatically, and I gathered approximately two weeks’ worth of data. However, the Connect IQ Store website is currently unavailable to me. Previously, the store pages could be viewed in a desktop browser.

This page used to be accessible: Connect IQ Store Trending. Based on my automated tracking, the watch face ranked first gained around 150 reviews per day. To reach the top nine and appear on the homepage, a watch face appeared to need more than 40 new reviews per day.

Based on what I observed, this ranking seems to be driven by the number of reviews: the more reviews a watch face receives, the higher it ranks. The developers I monitored appear to rotate which watch faces they artificially inflate reviews for, rather than focusing on just one.

If a watch face stays near the top of this chart for a long time, that visibility can bring in genuine downloads. Those downloads can then help it appear on other charts, where download numbers seem to carry more weight. Of course, if the watch face itself is poor, the extra exposure may not be enough to make it successful。

I think you’re right.

There was a developer with watchfaces called something like “Forester X” and “World X”. They were in the 1k+ downloads tier (so under 10k), but he still had over 1k reviews, which struck me as very odd. For comparison, a few of my free apps are in the same 1k+ tier and have only around 20-30 reviews each.

His account has apparently been removed since then, because I can’t find it anymore. I can’t know the reason, but it makes me suspect that some developers really are gaming the system. This one didn’t even try to be subtle about it, so I’m fairly certain others are doing the same, just more carefully, and staying under the radar.

This is one more reason why Garmin needs to be transparent about the ranking factors. Even in the best case, over 1k reviews with under 10k downloads means at least one review per ten downloads, while real apps typically see only a few percent. Such a mismatch is easy to detect, and as long as nobody outside Garmin knows what the ranking rewards, nobody can tell legitimate success from manipulation. The honest developers are the ones who pay for it.

By the way, here is how Google Play Store and the iOS App Store handle it:

Neither store publishes its ranking formula, but both disclose the main ranking factors.

Google Play has a help page for developers that explains the main factors behind its rankings, such as how relevant an app is to the user and to what they are searching for. Google also states that these factors are weighted differently depending on where the user is browsing, which device they use, and their personal preferences. The exact weights are not public.

Apple names the main parameters in its terms: the metadata provided by developers, how customers engage with the apps, and the popularity of the apps in the store. The algorithm itself stays secret.

That is exactly what I’m asking Garmin to do. The EU P2B Regulation (Article 5) requires platforms, app stores explicitly included, to set out the main ranking parameters and the reasons for them in their terms. Nobody is asking for the algorithm, only for the main factors and why they matter.

Both stores also treat ratings and reviews as official ranking signals, so this is not a security risk. If Google and Apple can be this open while still protecting against manipulation, Garmin can too. As long as the Connect IQ ranking is a complete black box, nobody can tell legitimate success from manipulation.