Posts Tagged ‘research’

Update II – Some Brief Reflections on Crowdsourcing Research (9/27/10)

After hosting this crowdsourcing site for a little more than three months, Jonathan and I have decided to shut down the experiment. The paper has been accepted at M4D 2010 in Uganda, so we want to stabilize the data and take this chance to provide some brief reflections on the experience.

We began this small research project curious about what we feel is the next stage in connectivity for the developing world: prepaid mobile Internet. Business models, amongst other subtle factors, will have influential roles in how this next stage unfolds. In exploring the availability of prepaid mobile Internet, we discovered that the African landscape is mottled, with different operators in different countries at various stages of offering the mobile Internet. But given the nature of the desk survey we conducted, we were unable to conclusively determine the offerings in every country, so we decided to open up the data collection to outsiders. The experience was a mixed one, but here are some thoughts:

Make Sure the Technology Fits the Project

Although we initially planned to use a publicly editable Google Map, we quickly switched to an Ushahidi instance because it was designed for mapping crowdsourced data. We found, though, that the initial purpose of a project may make it less flexible than two researchers with limited coding capability may hope. Ushahidi is great for mapping crises information, but that means its categorization and language used may make collecting information on mobile phone plans a bit confusing. For example, as is appropriate and understandable for crisis mapping, but confusing for our needs, is the hard-coded language of “Report an Incident” on Ushahidi. This slight barrier may make people less likely to participate.

The Importance of Personal Networks

During this experiment, we uncovered some great insights from individuals – far more nuanced than the simple “yes mobile Internet is prepay here” that we initially sought. In particular, the complicated nature of Africa mobile Internet was explained by Linda Raftree from her personal experience. Katrin Verclas, of Mobile Active, provided the initial introduction, but via email, not the platform we expected. These types of personal, pre-existing contacts are likely to be very helpful for efforts like this.

Unclear Nature of Selection Bias

Researchers take great pains to make sure they study replicable and fairly sampled groups. Because we were using personal networks and open crowdsourcing, we had to consider what, if any, bias this exploratory study had. Using the convenience of these tools, are there data being systematically excluded? Given that we had a well-defined population (African mobile network operators), this was less of a problem, but for researchers navigating new methodologies, it is worth considering. More specifically, Jonathan and I operated within the English-language blog and Twitter networks; did we miss Francophone Africa through crowdsourcing?

Summary

Putting the experience together, we might propose that crowdsourcing success is a function of (a) the ease of the task, (b) distribution of knowledge, (c) accessibility of the population with knowledge, (d) the willingness and capacity of researchers to drive the message and effort. If any of these fall short, the critical mass may never be reached.

Update – We are crowdsourcing the data on a website here.

Mobile Internet is going global and the newest chapter is taking place in developing countries. In order to better understand the paths towards widespread adoption of mobile Internet, Jonathan Donner and I have written a brief note on the availability of prepaid mobile Internet in Africa. Because prepaid models are more appropriate for poorer consumers, we argue that the availability of prepaid mobile data will be a key driver of inclusive mobile Internet usage.

Starting in late 2009, we collected data on the availability of prepaid mobile data in all 53 African countries. Unfortunately, without the budget to travel the continent, we have been unable to conclusively determine the presence of this form of connectivity in every country. So, we’re asking you to help build the database of prepaid mobile data in Africa.

Available here (in PDF) is a draft version of our paper where you can get a sense of our project.

For the crowdsourcing, we’ve created an editable Google Map with entries for each country. Green indicates existing knowledge that prepaid data is offered by at least one provider. Yellow means we have been unable to determine the presence of prepaid mobile data. And Red suggests confidence that it is not available in that country (though if you know otherwise, please do correct us!).

If you know for certain that prepaid data is available for mobile phones from at least one network provider in one of the countries marked Yellow, you can either get in touch directly or login with a Google account to the African Prepaid Mobile Data map and, in the upper right of the left-hand sidebar, click “Edit” to create your citation (the more supporting evidence, including links or your name/affiliation, the better).

This is a small-scale experiment in crowdsourcing data for use in an academic paper, so we’re not sure how much detail we will be able to gather, or what end-state the map will be, but we’re grateful for your help. Thanks!

I recently finished and highly recommend Portfolios of the Poor, a book examining the financial lives of dozens of impoverished families in Bangladesh, India and South Africa. The understanding of how the world’s poor survive on less than $2 per day was captured through an intensive research process that spent a year chronicling their financial activity through “financial diaries.” Researchers visited the surveyed families every two weeks to discuss their uses of money and in doing so came to have intimate insights into how they made and spent money.

The study sought to fill in the gap in between small ethnographic studies and large-scale surveys. The former cannot capture how lives change over time and the latter fail to capture the nuances of individuals. Portfolios is filled with revelations that came about solely due to their method.

While reading Portfolios, I couldn’t help but feel that ICT4D suffers from the same gap as microfinance. On the one hand, there are plenty of small, short-term studies of people’s interaction and relationship with technology (e.g. Chipchase), and on the other are large-scale surveys of national ICT usage (e.g. RIA).

So, what would an ICT diary project look like? Luckily, the authors of Portfolios provided an appendix detailing their methodology and processes. Some key points:

• Conceptualizing as diaries is useful because tracking in (near) real-time avoided memory loss (who really remembers how they used a cell phone two months ago?). It also helps to capture the relationship between money (or ICTs) and time. Looking at the flows was important for the Portfolios project because if they had simply surveyed irregularly they would have thought that large assets, which were the majority of the wealth for the subjects, were important; instead, it was the margin – small loans, daily wages – that provided most of the activity about their welfare.
• Given the deeply personal nature of both finances and communication, a long-term relationship between researcher and subject is necessary to develop trust. The Portfolios researchers were close in socioeconomic characteristics to their subjects to minimize the barriers. Even then, subjects were not always forthcoming and the researchers needed to really probe to discover all the financial instruments used. The trust issue also meant that the bimonthly visits needed to be conversational, not interviews. However, they write that “the strength of the diaries approach is that it can, over time, break down much of this reticence and confusion.”
• The effort wasn’t meant to be statistically representative, but they did choose families in low, middle and high levels of poverty.
• The information they gathered did not match their preconceived categories, so the researchers had to iterate towards perfection.
• There is the possibility that merely observing the financial lives of the subjects changed their behavior (a Heisenberg effect).

For an ICT diary project, there would obviously be differences. I imagine that ICTs are used more frequently and with less thought than major financial transactions, so there will definitely be uses that are forgotten or not mentioned to the interviewer. This problem could be minimized by using software to track usage, but capturing this quantitative data would need to be done very carefully to avoid privacy implications. A solid framework for usage would need to be utilized to make sense of various uses of ICTs and the behavioral evolution that I would expect to see in a year’s time.

Discussing this at Mobile Tech for Social Change Cape Town on Saturday, it was pointed out that the ISPs and telcos have much of this data, but getting them to share it would be next to impossible. Certainly looking at that would be useful, but the ICT diaries would be another way, and perhaps a better one, because the discussions with the participants would provide insights into their relationship (perceived and real) with technology.

If development practitioners are going to build ICT solutions for the poor, they must understand how they use technology. That understanding is certainly aided by the existing studies, but the gap in the middle seems to be where much of the insights will be found. Following ICT usage for a year would allow researchers to explore the evolving behavior that comes from both internal and external changes.

[Image from Kiwanja's mobile gallery.]