Does startUserReview() work?

That was a good challenge and as I do have my own subjectivity it’s not easy to thought-experiment myself out of so I asked an independent arbitrator who said quite a lot of interesting things including;

Bottom line

The evidence does not support the blanket claim that asking users for a review is bad for consumers. In fact, there is good evidence that a well-timed, neutral request can benefit the review ecosystem and consumers indirectly by producing a larger and more representative sample of user experiences.

However, there is an important distinction:

Asking for an honest review is generally defensible; interrupting the user, repeatedly pestering them, or steering them toward a positive review is not.

and …

Soliciting reviews increased participation and made the resulting public review distribution more representative of the underlying customer experience.

and …

Full transcript (though it goes into my-app specifics I think most of the points are generalised); https://chatgpt.com/share/6ac3db23-f78c-83ed-b620-7ecbefb2f880

crispgarmin said:

The evidence does not support the blanket claim that asking users for a review is bad for consumers.

The evidence does not support that it’s good for customers. We do know that some customers don’t like it.

crispgarmin said:

In fact, there is good evidence that a well-timed, neutral request can benefit the review ecosystem and consumers indirectly by producing a larger and more representative sample of user experiences.

There you go.

Otherwise, ChatGPT appears to be blowing smoke up something and telling you what you want to hear. I especially like the idea that you can figure out that a random user isn’t doing something important (relying on what the app is doing is not “customer focused”).

crispgarmin said:

Asking for an honest review is generally defensible;

It’s generally defensible to say that asking for reviews is not a good idea too.

You are falling over yourself justifying it.

You obviously didn’t read the independent, research-evidence-supported (not my opinion) bottom line, you are cherry picking your quotes now. Pot, kettle, black.

Some of the research isn’t relevant. Your pop-ups aren’t “WOM”.

Chatgpt is telling you what you want not hear. 

“► Significant decrease of willingness to pay for a brand that interrupts online flow.► Reduced willingness to pay attributable to negative reactance.► More control over iterrupt does not alter the effect.► Aggressive advertising can raise awareness, but may reduce its bottom line.”

“The average user reacts to pop-ups with responses ranging from mild irritation to vocal annoyance.”

“solicitations fail to elicit reviews where they matter most.”

“Review solicitation, however, generates a notable negative spillover effect on the volume of organic reviews.”

Some of the research supports my position.

(Yes, these are cherries but my point you seem to think there is no possible downside.)

(My point isn’t to convince you of anything. It’s to have other people thinking of doing this to look more closely and not ignore the downsides.)

dpawlyk said:

you seem to think there is no possible downside

It’s a good, healthy, rigorous debate that is opening far more thoughts and angles than I expected.

But in response to the line above, I do see downsides, and although I was going to say “but there is only one right solution for each app or developer” … that’s not quite right either.  If you act smart there are options. If you solicit review carefully, without interrupting flow and with the right language it can garner rewards. But blanket carte blanche approaches are more likely to have a negative impact. I will review positioning again in my apps and hope this is a useful reference for others.

crispgarmin said:

It’s a good, healthy, rigorous debate that is opening far more thoughts and angles than I expected.

Pretty much what I was aiming at.

crispgarmin said:

I do see downsides

You didn’t really mention any.

crispgarmin said:

If you solicit review carefully, without interrupting flow and with the right language it can garner rewards

Maybe. (You keep assuming this must be true.) The issue is whether the rewards (if there are really any) outweigh the costs (we know some don’t like them).

==================

The general problem with things like this is that the developer/whatever can easily convince themselves that their thing is fine. They don’t consider that there are lots of other developers who think the same thing. From the user’s point of view, it’s just a lot of selfish developers thinking that their thing is so very important and their one thing couldn’t possibly irritate anybody.

It’s the same justification that drives a lot of spam. (Not all spam: some spammers are just evil.)

For this kind of thing, I think the user’s perspective is much more important than the developers. I still haven’t seen anything that shows how this benefits the user being asked.

dpawlyk said:

I still haven’t seen anything that shows how this benefits the user being asked.

This is the holistic approach… if something is a small deficit to one user, but facilitates awareness and expanded use to a wider community who would not otherwise not experience it then there is a greater, communal good.

I don’t buy an argument that all review processes are wrong, and I don’t think the idea of an elite reviewer system (Michelin, famous “expert” named critics etc) is better than a community approach. I appreciate Amazon, Google, CIQ awareness of things I didn’t know and validation of things I did or helpful direction in a complex marketplace of similar products.

Yes, I am (again) creating support and justification, but as a means of explanation. This is not a purely consumer-oriented environment or narrow single-consumer model; the user does not exist in isolation. By being part of the “store” ecosystem the consumer/user and product have a transactional relationship that can work both ways to the benefit of all.

crispgarmin said:

This is the holistic approach… if something is a small deficit to one user, but facilitates awareness and expanded use to a wider community who would not otherwise not experience it then there is a greater, communal good.

We know it irritates some users (*).

We have no idea whether it (review requests) does any of the positive stuff (or enough of the positive stuff). Or even whether the default review process is actually (significantly) worse.

crispgarmin said:

I don’t buy an argument that all review processes are wrong

???

No one is making this claim.

crispgarmin said:

I don’t think the idea of an elite reviewer system (Michelin, famous “expert” named critics etc) is better than a community approach.

There isn’t any way of knowing a community approach is better.

There are certainly problems with a community approach. Often, community reviews are negative and give the impression that things are worse than they are.

It’s not an “either/or” situation either. You might want both.

The better expert reviews do more comparison and tend to be deeper  The expert (presumably) knows how to use the product. Consumer reviews tend to be more random.

——————————

* we might not care if this is a tiny number (you aren’t going to make everyone happy. But your research results suggests it’s not an insignificant number.