Tag: online

Sample Size of One: Towards a Possible Solution

This post explores an alternative to fix the replication crisis (particularly in the behavioural science and economics fields, and if relevant, in other fields too). This post is in continuation to an earlier post titled Sample Size of One: The Rose Negotiations. It would help to read that one first before coming to this one.
 
What can we do to solve our human desire to create or find patterns and thumb rules to how we function? Or to find keys to getting humans to behave in a particular manner, be it to drive a more healthy culture, or improve finance sense among populations. Especially when few patterns exist. And when our desire might be overly simplifying a pattern which might be far broader than what we might want it to be.
 
Here is a broad suggestion towards what a possible solution looked like, at least in my head.
Consider a “hypothetical” scenario where a group of researchers wants to find the effects of an ‘opt-in/opt-out response’ for organ donation.
Up until now, behavioural economists or scientists would identify a large, diverse study group of volunteers, and conduct the experiment. Let’s suppose at the end of the study, they found that 70% of respondents opt for the organ donation program when the form requires them to physically opt-out of organ donation.
Now, a non-profit across the world tries this tactic on a local population, but perhaps has a less than encouraging (and far less than a 70% success rate) outcome. This leads to questioning the research findings, and the broader hue and cry around the reproducibility and replicability of such studies/ experiments.
 
For a moment, consider currencies. They are always fluctuating, and there is a definite exchange rate to convert between any two currencies at at a given point in time.
Or consider diverse marketplaces across the world. Any single product would be differently priced in these different marketplaces. And within a single market, the price variance might not be too much. But it might vary if you went to a market in the next village.
 
Coming back to trying to find an alternative to traditional experiments that try to find thumb rules to then apply to social or business causes.
 
The alternative I see, is where the economist or scientist creates a simple experiment or study around the hypothesis they would like to test. They would then put it on an online platform (and share it with their colleagues and counterparts across the world, who would then deploy it among local populations).
The experiment would be introduced via a website. Deployment could be done online with voluntary participants, or random people. The experiments would run in perpetuity (hence online), and results of the same would keep evolving over time and geographies.
 
The outcome for the same opt-in/ opt-out hypothesis with this alternate deployment might look something like this:
The experiment is designed to be unbiased, simple (easy to deploy without the original team of researchers being physically present), and yet robust enough to provide meaningful data.
The results of this experiment would not be captured as a single value (like 70% in the first hypothetical scenario), but rather as a function of (age/sex/location/study response/point in time).
As a result, what the outcome might look like, is diverse data points from across the world at diverse points in time.
It is possible that patterns will emerge in localized groups, or even at a nation-level for some experiments (since respondents or the general population might share a similar national history, current political and economic environment, and similar fears and concerns – whether it is about inflation, unemployment, or a multitude of other variables that were possibly getting ignored when a research study focused on finding a thumb rule.
With a global, perpetual study, for the same opt-in/ opt-out experiment, we might perhaps get results like an average of about 65% in Mumbai, India, but a 20% on the outskirts of Mangalore, India, and maybe even an 80% in Itanagar, India.
That way, researchers and anyone trying to use these research findings would be mindful that it isn’t a one-size-fits-all finding. But rather that perhaps (cautiously), one might expect to get a similar response to an organ donation campaign in a town in Country 1 and a city in Country 2, because their outcome values over a particular period of time have been similar.
 
And these values that emerge across individuals and locations are not fixed values. They are ever-evolving, to reflect the evolution of humans in a particular society, given the context of its changing sociopolitical and socioeconomic landscape, among other variables. So perhaps the same individuals too could participate in the same experiment multiple times over the years, with different results. In that sense, it would be similar to taking an IQ test or an MBTI test.
Which means, the same non-profit that is driving an organ donation exercise in a particular country in a particular year, would refer to the current result outcomes for different parts of that country, to determine what strategies they might have to employ (government intervention, financial incentives, etc.), towards driving a more successful change effort.
 
An obvious extension of this proposed solution will be in the next post.
 
#SampleSizeOfOne #BehaviouralScience #BehaviouralEconomics

Creating and Understanding Customer Feedback

A waffles order packaging (doesn’t it look like a Viking head?)

Creating and Understanding Customer Feedback

If you’ve ever ordered waffles online, most likely they’ll come in one of two kinds of packaging. One is clean like in the pic above. The other is where all of them in thin paper holders will be stuffed into a box. Quite messy.

Anyway, say you ordered a few dishes for dinner via a food ordering app from a local restaurant. packaging by the restaurant is horrible. The food has leaked into the outer bag, and slightly onto other food containers below.

However, the food itself is delicious.

Now consider you ordered from another restaurant on another night. Exceptional, airtight and impressive looking packaging.

However, the food tastes somewhere between horrible and just-average.

Now, if both restaurants, or even the food ordering service used a simple rating mechanism, chances are, both restaurants will be oblivious to what customers love and hate about them.

The first restaurant might see a bad rating and think their food sucks. The packaging quality never crossing their mind.

The other restaurant might feel proud with a high rating, assuming it was for their food, while customers struggle to consume it. Or they might think the bad rating was because of some delivery error or delay.

If you are going to take the trouble to capture user feedback, take a little more trouble to capture more detailed feedback. Because vague feedback can sometimes be more dangerous than no feedback.

Without boring the customer, try and split up your service feedback into its components. In the case of the home order, it could be the food quality, packaging quality and service delivery. For a product, it could be the effectiveness of the product (in doing the job), ease of understanding and use (instructions, design simplicity, etc.), and effectiveness of customer service (if it comes to that).

If you own, manage or work at a company, and are grappling with a complex challenge or are in need of innovation for growth, get in touch. More here.

And you might find my book, ‘Design the Future’ interesting. It demystifies the mindset of Design Thinking. Ebook’s on Amazon, and paperbacks at leading online bookstores including Amazon & Flipkart.

%d bloggers like this: