monitoring and evaluation | TechChange | The Institute for Technology and Social Change

PreMAND field workers testing data collection tablets in Navrongo, Ghana (Photo: N. Smith)

Mira Gupta, one of the star alumna of our courses on Mapping for International Development and Technology for Monitoring & Evaluation (M&E), is a Senior Research Specialist at the University of Michigan Medical School (UMMS). Last October, USAID awarded UMMS $1.44 million to assess maternal and neonatal mortality in northern Ghana. This 36-month project, “Preventing Maternal and Neonatal Mortality in Rural Northern Ghana” (also referred to as PreMAND: Preventing Maternal and Neonatal Deaths) will help USAID, the Ghana Health Service, and the Ghana Ministry of Health design interventions to prevent maternal and neonatal mortality by investigating the social, cultural and behavioral determinants of such deaths across four districts in northern Ghana. For this project, UMMS will be partnering with the Navrongo Health Research Centre and Development Seed.

Project Regions and Districts

We sat down with Mira to learn more about this project and how her TechChange trainings in digital mapping and technology for M&E gave her the skills and background she needed to develop her team’s project in Ghana.

1. What interested you in taking the Mapping for International Development and the Tech for M&E online courses?
I was in the process of trying to learn everything I could about our GIS options when I heard about TechChange’s Mapping for International Development course. It provided a fantastic introduction to the range of approaches being used on international development projects and the variety of organizations working in that space. The course material helped me identify which types of visualizations would be most appropriate for my team’s research. I especially benefited from the many sectors represented in the TechChange sessions because while I was trying to create a project for the Health sector, I actually learn best through a Democracy and Governance framework given my previous background in this field. TechChange provided access to mapping specialists in both areas through its instructors and other class participants.

Just as I heard of the mapping course right as I needed it, the same thing happened again with the Technology for M&E course, which I took a year later. By that point, the PreMAND project had just been awarded and I learned that I would be responsible for the evaluation components. I was excited to take the TechChange course because I knew it would provide a great overview of the many different tools being used, and that I would benefit immensely from the participation of classmates working on projects in similar settings. As expected, the content presented was incredibly valuable in informing our project approach in terms of our field data collection, methods of analysis, and presentation of findings.

2. How did the mapping component of this USAID-funded project come together?

The Three Project Phases: Research will inform the Visualisations, which will inform Programming

While working on a maternal and neonatal health qualitative study a couple of years ago, I sensed that there were themes and patterns in the data that were difficult to verify since the locations of the respondents had not been geocoded. Some of the variables indicated 50/50 probabilities of any particular outcome, which seemed to suggest that there was no pattern whatsoever when viewed as a large dataset. Because my background is in Democracy and Governance, I used election maps to illustrate to my research team that once geocoded there might in fact be very distinct geographical trends in the data, drawing parallels to the locational breakdown of political party support in the United States.

I was in the process of researching mapping resources when I first heard about the TechChange’s Mapping for International Development course, and through the course I met some of the mapping experts that ultimately served as key resources in the development of our project strategy. The course gave me the necessary base knowledge to effectively liaise between our health researchers and the mapping experts to determine the best approach to meet our data visualization needs. We were extremely fortunate to have USAID-Ghana release a call for outside-the-box submissions under its Innovate for Health mechanism, right as we were developing our program concept.

3. What are the biggest challenges you anticipate in undertaking this project?
For the visualization component, generating the base layer maps will be more difficult than we originally anticipated. The various pieces of data we need are spread throughout different government sources such as the Ghana Statistical Service, the Lands Commission, and the Ministry of Roads and Highways. We will need to consult with each of these groups (and likely many others), to explore whether or not they will allow their data to be used by our project. It will require some agility on our part, as we need to stay flexible enough so that we collect any outstanding geographic data we may need through our team of field workers. While there are many moving pieces at the moment, it’s exciting for us to think that we’re building what may be the most comprehensive geographic base layer map of the region, as an initial step in developing our health indicator analysis tool.

There are also a handful of challenges related to evaluation. The primary purpose of our project is to provide new information to clarify the roles of social and cultural factors in determining maternal and neonatal deaths, and shed light on a valuable set of drivers which up until now have been unclear. We are currently in the process of finalizing our M&E framework, which has been a complex process because our project doesn’t fit the mold that most performance indicators are designed for. As a result we’ve been carefully drafting our own custom indicators through which we’ll measure our project’s progress and impact.

One of our most interesting evaluation challenges has been the development of our Environmental Mitigation and Monitoring Plan, which is traditionally intended as a tool for implementing partners to take stock of the impacts their work could have on the natural environment. In our case, we’re using it as a tool to think through our ethical approach to the potential impact of our project on the social and cultural landscape, given the challenges associated with collecting very sensitive health information and the need for data privacy. It’s pushing our team to think through every step of our project from the perspective of our various stakeholders, and has yielded many valuable insights that have strengthened our program approach.

4. What are the tools that you became familiar with in Mapping for International Development and M&E and plan on using in this project and how will you apply them to your project?
I came into Mapping for International Development knowing very little about the resources available in that space. Several of the tools that I became familiar with through the class, such as OpenStreetMap, MapBox and QGIS were highly applicable to our project in Ghana. After participating in the session led by Democracy International and Development Seed, I reached out to those instructors for their input on how I could best translate my project concept into actionable steps.

The visualizations I hoped to create were complex enough that I soon realized it would make the most sense for our research team to work directly with a mapping firm. We were so impressed by the technical feedback and past projects of Development Seed that we established a formal partnership with them and worked together to refine the vision for the project that was ultimately funded. TechChange’s training gave me the knowledge I needed to select the right partner and understand how best to combine our research goals with the available mapping resources to maximize our project’s impact.

Programs used on the PreMAND project

In Technology for M&E I learned about the capabilities of different devices, survey apps—those able to capture geodata were of particular interest to me—and even project management tools. There were many helpful conversations both in the class sessions as well as in the participant-led threads around the data collection process, data privacy, and the ways in which project findings can be best communicated to a variety of stakeholder groups. What I found to be most relevant and applicable to our Ghana project were the conversations surrounding human-centered design, and the use of rich qualitative data. I gained a lot from the session led by Marc Maxson of GlobalGiving, who discussed which forms of data are the richest and easiest to interpret. The University of Michigan and our partner the Navrongo Health Research Centre already excel in qualitative data collection techniques, but the conversations throughout the TechChange M&E course inspired some new ideas as to how we might incorporate multimedia such as video and photographs in our qualitative data collection process to make our project deliverables that much more substantive.

5. What is your advice for researchers working to integrate more data visualization and mapping in their research and project interventions?
My advice would be to focus on the end user of your data and identify their needs and interests early in the process. That clarity can then be used to inform 1) what content will be most useful, and 2) what presentation format(s) will be most effective. It’s important to do some form of a needs assessment and let stakeholder feedback guide the project’s design.

In the case of our Ghana project, we are implementing a two-prong approach to our visualizations because both the government representatives and our donor will find an interactive web application most useful, while local community members in the rural North will benefit more from group discussions centered around printed maps.

Feedback loop with two stakeholder groups: the government of Ghana and local communities

It is common to sometimes present health indicator data solely as points on a map, but we are designing our visualizations to be much more detailed with background layers including health facilities, schools, compounds and roads so that those viewing the health indicator data can orient themselves a bit better to the local context. Had our end-users only been the leaders of those individual communities such detailed maps may not have been necessary. Similarly, the visualizations for one stakeholder group might incorporate a lot of words or even narrative stories based on their level of education, while for other stakeholders, those visualizations will be more image-based and we’ll orient them to the maps through presentations in their local communities.

About Mira Gupta

Mira Gupta is a Senior Research Specialist at the University of Michigan Medical School (UMMS), where she focuses on program design, strategy and evaluation. She has developed successful international aid projects in 18 countries, including 13 in Africa. Mira began her career in the Democracy and Governance sector where she worked for organizations such as IFES, the National Democratic Institute, and the Carter Center. She also developed projects in the Economic Risk and Conflict Mitigation sectors before transitioning into Global Health. Her research on the effects of local power dynamics on health-seeking behavior in northern Ghana is published the current edition of Global Public Health.

With the rapidly growing field of monitoring and evaluation, there are many technology tools that are designed to help the many roles of M&E practitioners. The fastest growing area has been digital data collection, which currently uses mobile phones and portable GPS systems. Reporting has become easier with all the tools for data visualizations and data cleaning. There are also many research options with statistical software and programming languages for data entry, documentation, and analysis. In addition, real-time M&E tools let you do program and data management with real-time project updates. What we found in the course was that there are tech tools that integrate multiple aspects of M&E.

Here are several of the tools we discussed in the inaugural round of our Technology for M&E online class last fall, as crowdsourced by over 100 IT experts and M&E practitioners based in over 30 countries. In the next iteration of this course, we’ll be covering some of the latest tools including satellite imaging, remote sensors, and more.

What M&E technology tools do you use? How has your experience been with these tools listed? Are there great tech tools for M&E that we missed? Please share with us in the comments or tweet us @TechChange.

Interested in learning about these tools more in depth? Join us for our popular Tech for M&E online course that runs 14 September – 9 October.

GIZ Nepal participants Pushpa Pandey, Valerie Alvarez, and top TechChange student Bikesh Bajracharya with TechChange Communications Associate Samita Thapa, (and TechChange cubebots).

In our most recent mHealth online course, twelve participants from GIZ (Deutsche Gesellschaft für Internationale Zusammenarbeit) Nepal enrolled in the course to support its mHealth pilot for adolescent sexual and reproductive health. This holiday season, I was fortunate enough to return to my native home of Nepal to meet these TechChange alumni in person at the Nepali-German Health Sector Support Programme (HSSP) at their new office in Sanepa, Nepal. Since the September 2014 mHealth pilot launch, more than 150,000 adolescents have used their interactive service.

Nepal’s National Health Education, Information and Communication Center (NHEICC) developed a National Adolescent Sexual and Reproductive Health Communication Strategy (2011 – 2015) that stressed strongly the use of modern methods of communication in its implementation. GIZ, Health for Life (H4L), and the UN Population Fund partnered under NHEICC’s leadership to initiate this SMS based mHealth project – the first in Nepal. The SMS messages and interactive package focus on delaying marriage and pregnancy, healthy timing and spacing of babies, health and hygiene, and addressing gender based violence. The local mobile services provider, Nepal Telecom and NCELL, distributed the interactive SMS package that includes an encyclopedia, role model stories, quizzes, and a hotline for further questions.

Mr Khaga Raj Adhikari, Minister, Ministry of Health and Population launching ‘m4ASRH’ (Mobile for Adolescent Sexual & Reproductive Health) on 18 September 2014.

Since Pushpa had shared the status of the GIZ mHealth pilot in Nepal as her final project for the mHealth online course the day before we met, it was especially great to catch up with her in person! She expressed that this mHealth course was much more engaging and fun to complete than other online courses she has tried out. Bikesh, the top user in our course with over 400 tech points, is new to the GIZ team and very excited to apply what he has learned in the mHealth course to his work in Nepal. Valerie recently arrived in Nepal and very new to the GIZ-team, was also excited to learn how much the other participants were engaging and that she can still access all course material for four more months.

All three GIZ Nepal participants shared their astonishment on how many tech points Bikesh was able to stack up in the course and also the fantastic course facilitation by Kendra. They also admitted that hearing Pushpa present in the mHealth course gave them insights that they weren’t aware of even though they work at the same office. While taking a technological approach to development projects in a country like Nepal can be challenging, it is an even bigger challenge to get the government’s buy in. It was exciting to learn that despite some hurdles, this mHealth pilot was an initiative supported by the government of Nepal.

We are excited for the future of mHealth in Nepal and wish GIZ all the best in their continued success! We are also excited to welcome six more participants from GIZ Nepal in our upcoming Technology for Monitoring and Evaluation course in January to better measure the impact of this mHealth pilot! It is wonderful to see how GIZ is committed to mHealth and M&E through their investment in technology capacity building in Nepal.

Photo Source: EvalPartners

Today marks the first day of the International Year of Evaluation, which kicks off with an official celebration at the UN Headquarters in New York City. More than ever, evaluation is becoming increasingly important in international development. The global EvalPartners and the United Nations Evaluation Group officially declared 2015 as the International Year of Evaluation earlier this year. As the Millennium Development Goals (MDGs) will be replaced by Sustainable Development Goals (SDGs) in 2015, measuring results and impact of development activities is increasingly critical. How can the international development community align their monitoring and evaluation skills with new technology to make progress toward the SDGs?

TechChange couldn’t be more excited to kick-off 2015 with the launch of the next round of our online course on Technology for Monitoring & Evaluation. Starting January 26th, this four week course will explore the vital role of technology in enhancing monitoring and evaluations efforts which could have a critical impact on SDG fullfillment. Sign up here to join the learning community!

The global development industry is generating a lot of data on the ‘developing’ world–data that has not always been available. As technology has made data collection easier and scalable, many in the development industry have already established that monitoring (i.e., data collection) is much easier than evaluating (i.e., data insights). However, both aspects of M&E require good methodologies to ensure the data are accurately represented.

Despite making my living working with data, I am somewhat of a data skeptic. Specifically, I am skeptical of the notion that numbers and data are truth. Much like geographer Doreen Massey’s conceptualization of space as a product of social relations, data embodies social relations and biases. In other words, it is difficult to guarantee the neutrality of data and numbers in terms of how they are collected, what they show, and how they are analyzed. All of this information is subject to human bias – whether intentional or unintentional – with the way humans label data, the limitations of finite data samples, and the human-designed technology that might reinforce biases.

The way humans label data
Does the way we identify data represent cultural bias? In some ways, yes. Labels can be culturally problematic in the way we classify data and the way people interpret those classifications. For example, when collecting demographic information for a survey, limiting gender to two categories, we can reinforce our own notion of gender categories and unintentionally bias the data. India and Nepal, for example, both recognize a third gender on official documents. M&E data in these countries however, do not always reflect this change. Mortiz Hardt, a researcher at IBM, notes five ways that big data is unfair. Along with different cultural understandings and the consistent, if unintentional, representation of social categories (e.g., race and gender), Hardt notes sample size as a problem.

Limited sample sizes of data
The issue of certain groups not being represented in the data is a particular problem for global development. A recent study by the Global Web Index highlights that geolocation can lead to groups in the ‘developing’ world not being counted by web analytics. Virtual private networks (VPNs), which are a common tool for accessing blocked sites, and shared devices are some of the main culprits. Additionally, issues of privacy can change responses and skew the data and limit the sample size of quality data. For example, in some societies, even if a woman owns a cell phone, she is not always free to respond without having her calls and text messages monitored.

Are we training machines to mimic our cultural biases that are in data?
This human bias within data is of particular concern for predictive modeling and big data, both of which are starting to enter development as seen in report reports by UN Pulse and the World Economic Forum. But an algorithm for predictive modeling is just training a machine based on the data that it’s given. So if the data are biased, the prediction will be biased. According to Wired Magazine article with Danielle Citron, a University of Maryland law professor, humans can trust algorithms too much, in that “[…]we think of them as objective, whereas the reality is that humans craft those algorithms and can embed in them all sorts of biases and perspectives.”

So what does data bias mean for global development and M&E professionals?
Global development needs to continue being data-driven. This is emphasized by one of the principles for digital development being focused on data driven decision making. It is equally important we recognize and understand the biases we incorporate into datasets and the biases of the datasets of the datasets we use.

At the end of the day, Tech for M&E begins with the humans behind the data. With the vast amounts of data provided with modern digital data collection tools, M&E practitioners need to understand how they can act as gatekeepers to ensure that we note the bias we are embedding in our data.

===

Interested in this topic on data in global development and measuring results? Join our top selling online course on Technology for Monitoring & Evaluation, which begins April 20, 2015.

At TechChange, we’re always looking for ways to make online learning more interactive, engaging, and relevant for busy, global professionals interested in technology and social change. One way we do this is by bringing together our online TechChange community offline with hybrid learning. Specifically, we try to overlap the schedule of some of our online courses with industry events such as the recent M&E Tech Conference and annual mHealth Summit. We also arrange in-person meet-ups in various cities across the world including happy hours in Washington, DC and other meet-ups including most recently in Lusaka, Zambia.

Last month, we launched our very first round of TC111: Technology for Monitoring & Evaluation with a class of over 100 participants. As one of the top guest experts of the online course, Christopher Robert, who is CEO of Dobility Inc. and a Harvard adjunct lecturer, joined us in the first week of the course while he was traveling in Zambia. To take full advantage of the course, some of our participants based in Zambia asked him if he would be willing to meet with them in Lusaka. So, three of these M&E tech course participants (Ladislas, William, and Mine) met Christopher and his colleagues on the same day to continue the technology for M&E discussions from the online course in-person.

Here’s what happened at the TechChange Tech for M&E meet-up in Lusaka:

Reuniting alumni from different communities
It turned out that Ladislas, William, and Mine had already known each other as alumni of the Global Health Corps (GHC) fellowship. According to Mine Metitiri, a Senior Research Associate at the Zambia Ministry of Health, “A number of Global Health Corps fellows are taking the TechChange Tech for M&E online class and we recommended Chris to be a speaker at our annual training at Yale. Hopefully it works out because he had a lot of great things to say that are relevant to our fields of work.“

Strengthening online connections and learning offline
TechChange alumni such as William Ngosa who works at the Ministry of Health in Zambia appreciated the chance to reunite with his GHC colleagues and to meet Christopher and his team members, Faizan and Meletis. “It was a privilege to meet one of the speakers in the online course to provide a meaningful and enriching learning experience,” said William.
Christopher Robert and his team really enjoyed meeting the Zambia-based course participants as well. “It was lucky that we had the chance to meet!” said Christopher. “These Tech for M&E course participants are doing some wonderful things with ICT for social good there in Zambia. It’s always inspiring to meet people doing good work!”

Sharing good news of a job offer for M&E consulting
One of the participants, Ladislas Hibusu, received a M&E consultant job offer after interviewing with Jhpiego while taking the M&E online course.

“At this M&E meetup in Lusaka, I mentioned that during the M&E course, I interviewed for a position at Jhpiego. I am happy to announce that I have been offered an M&E Consultant role and thanks to the valuable insights to this course, as I was able to apply the knowledge I learned in the course. Although I have had limited experience in applying much of my M&E theoretical work in the field, I am happy to say this Tech for M&E online course is addressing most of challenges that I anticipate in my new role.” – Ladislas Hibusu

Everyone congratulated Ladislas and Christopher Robert joined us for another live event the following week wanting to continue the discussions with other participants in our course.

Several of TechChange’s online courses are designed to facilitate interactions like the one in Lusaka. Participants from all over the world are able to connect with like-minded professionals in the international development sector and continue discussions on specific topics. Watching live and recorded videos, completing different activities, and participating in ongoing discussions on an online forum combined with offline, in-person learning is really what enriches e-learning.

Interested in technology for M&E and want to connect with other M&E practitioners across the world? Register now to lock in early bird rates for our next round of our Technology for Monitoring & Evaluation online course which runs January 29 – February 20, 2015.

On September 25 and 26, over 200 development practitioners and technologists filled FHI 360’s conference rooms and hallways in Washington D.C. to discuss the intersection between technology and monitoring and evaluation (M&E). Some of the conference attendees included participants and guest experts in TechChange’s ongoing online course on Tech for M&E and it was great meeting so many these online course participants in person and offline.

Supported by the Rockefeller Foundation, GSMA, and FHI 360, the M&E Tech Conference was a two-day conference held in D.C. (followed by another one event in New York) to discuss the emerging role of technology in M&E and its implications to everyone involved in the international development industry. The release of the discussion paper, Emerging Opportunities: Monitoring and Evaluation in a Tech-Enabled World kicked off the first panel, followed by two days of great panel discussions and engaging break-out sessions. The event also featured a panel facilitated by TechChange online course facilitator, Kendra Keith on “What’s Next in Visualizing Data for Better Decision Making?” and a lightning talk on ‘How to Use APIs for Real-Time M&E’ that I presented.

In case you didn’t make it to the D.C. M&E Tech conference, here are a few key takeaways from the event.

1. Technology is not a substitute for good M&E methodologies

One of the panelists perfectly summed up the current state of technology in M&E from a technology perspective: it is not always clear why the M [monitoring] is associated with the E [evaluation]. While technology has made data collection and reporting easy, making sense of the data and how it affects programming still requires newer tools and M&E methodologies.

Good M&E requires capturing and analyzing both quantitative and qualitative data. While there are emerging M&E tools (e.g., SenseMaker) and techniques (e.g., natural language processing), cost and expertise continue to make capturing and analyzing qualitative data difficult. As mixed methods–using both qualitative and quantitative data — are discussed more in an M&E setting, practitioners still have a great deal of work to do.

Kenneth M. Chomitz, Kerry Bruce (a guest speaker for TechChange’s Tech for M&E course), and Maliha Khan discuss the current state of Tech for M&E

Mobile phones have been a hot topic in development, but one of the lightning talks revealed a surprising fact: Mobile surveys are not as effective as we think they are. The average response rate for surveys conducted in-person was 77% compared to 1% for mobile phone surveys. While the comparison may not be fair, it is worth noting that the introduction of any technology also requires M&E.

2. Technology introduces new data issues to M&E regarding responsibility, security, and selectivity biases

During the first panel, one of the panelists mentioned that the next “scandal” in development will be about revealing sensitive data. While not a new topic in development, data responsibility becomes increasingly important with the introduction of technology. Unfortunately, data security was primarily relegated as a panel (Whose Data? Whose Privacy?) and a shout out as one of the 9 Principles for Digital Development. This panel also marked the release of the Responsible Development Data Guide, a resource I co-authored, that focuses on protecting digital data and beneficiary privacy in international development.

Linda Raftree (a regular TechChange guest expert) and Michael Bamberger lead a break-out session discussing the emerging opportunities and challenges of using ICTs for M&E

Another common problem that arises with introducing technology to M&E is selectivity bias. Digital surveys tend to be limited to digitally literate populations with access to technology. Yet, even digital literacy and access to technology doesn’t guarantee a truthful response. For example, the panel on data privacy shared that women often didn’t respond truthfully to mobile surveys because their husbands or family members often monitored their personal phones.

Technology also biases researchers and evaluators towards quantitative methods and data. Qualitative data collection, analysis, and visualization software has not kept pace with tools for quantitative data.

3. Data matters more than ever in development

Recognizing the challenges and opportunities technology brings to M&E, the event included many conversations on data. There were break-out sessions on topics like data visualization, leveraging big data, how mobile phones can help in M&E, data security issues, and more.

Data visualization

The data visualization panel showed that there are a lot of new techniques to visualize data. Network visualization is a fantastic way to view a system (e.g., an organization or a program). It’s a simpler way to see where multiple links connect and understand where they need to be strengthened. Mapping allows for easier analysis of aid efficacy. Development Gateway presented their Aid Management Platform with its mapping feature that is aimed at governments in countries with development programs. In particular, they highlighted the success in Malawi and their public facing site with an interactive map. Excel–the tool that most people have and use to create pie charts–can be used to create some outstanding visualizations.

Neal Lesh of Dimagi presents trends in mHealth systems from over 175 CommCare projects

Open data

Most importantly, we are all looking forward to the day when open data is the standard. Many organizations spend a lot of time and money collecting the same data. If open data was the standard, available data can be used as baselines and potentially show impact after a project. Open data can also push for data structure standards (e.g., IATI) and allow data to be decentralized with application programming interfaces (APIs) connecting the data sources.

The challenges technology introduces to monitoring and evaluation, like data security and access, are topics we are also currently discussing in our Tech for M&E online course with a class of 100 students from all over the world. Due to popular demand, we are offering the next Tech for M&E course in January 2015. If you are interested in joining the discussion, apply before November 1 to get $100 off the full price of the course during this early bird discount period for the next Tech for M&E course.

If you did attend the event, what did you take away from the conference? Did you attend the conference in New York? Share some of your highlights and insights!

In his 2006 TED talk, Hans Rosling used data visualizations to deconstruct his students’ assumptions about the ‘developed’ and ‘developing’ dichotomy of countries. He looked at the patterns and demonstrated how they were easily recognizable and showed something contrary to the original belief. Pattern recognition is the core power of data visualization and more companies are embracing the notion of “putting humans back in the decision making process”.

Good data visualizations make patterns and outliers easy to recognize and aesthetically pleasing. The data are “liberated” from numbers and letters into a form that can be easily analyzed and understood by everyone.

Here are some great examples of liberating data through data visualizations

1. Microsoft’s SandDance Project

Microsoft recognized the importance of humanizing data with the SandDance project in terms of designing the data exploration experience using “natural user interaction techniques.”

2. Cooper Center’s Racial Dot Map of the US

US Census data is made freely available online for anyone to transform into a complex and understandable visualization. The data is available geocoded and as raw survey results. Last summer Dustin Cable took the 2010 census data and mapped it using a colored dot for every person based on their race: blue is White; green, African-American; red, Asian; orange, Hispanics; and brown, all other racial categories. The resulting map provides complex analysis quickly.

At a glance, it is easy to see some general settlement patterns in the US. The East Coast has a much greater population density than the rest of America. It slowly gets less dense until the middle of America where there is extremely low density until the West Coast. Cities act as a grouping point: density typically decreases in relation to the distance from a city. The population of minorities is not evenly distributed throughout the US with clearly defined regional racial groupings.

As you scan through California, an interesting exception stands out just north of San Luis Obispo. There is a dense population of minorities, primarily African-Americans and Hispanics. A quick look at a map reveals that it is a men’s prison. With more data you can see if there are recognizable patterns at the intersection of penal policy and racial politics.

3. Google Public Data Explorer

Google has created dynamic visualizations for a large number of public datasets. There are four different graph types, each with the ability to examine the dataset over a set period of time. With the additional element of time, new patterns can emerge.

Examining the World Bank’s World Development Indicators data set to compare fertility rate and life expectancy a pattern emerges: as life expectancy increases, fertility rate decreases. However, some notable exceptions occur. In 1975, Cambodia has a life expectancy slightly over 20 years, less than half of most countries with a similar life expectancy. It is also the year the Khmer Rouge took power leading to mass killings in Cambodia.

This exception to the normal pattern shows how strong of an impact a single event made. Data visualization makes recognizing this pattern and outliers as easy as watching a short time-lapsed video.

I’ve always believed that data are more than just collected information. Data have a purpose and are meant to be analyzed. New technologies have made visualizing data easier than ever and the data are more accessible to everyone.

What are some of the best data visualizations that you have seen, or maybe even created yourself? Please feel free to share in the comments or tweet @normanshamas or @TechChange.

Want to learn more about data visualization and analysis? Enroll now in TechChange’s new online course on Technology for Data Visualization and Analysis that runs June 1 – June 26, 2015.

In the last decade, new technology has made advances in data storage and analysis to leverage the greater volume of data available. The digital universe made up of all the data we create and copy will only increase in the future. The International Data Corporation and EMC’s research says that the digital universe is doubling in size every two years and by 2020, will contain nearly as many digital bits as there are stars in the universe (reaching 44 trillion gigabytes). We now live in a period of time defined by data: Data scientists are the new must hire position yet, McKinsey & Co.’s research says that by 2018, the U.S. will experience a shortage of 190,000 skilled data scientists.
What industries are the data scientists working in now

Government surveillance through internet data has been in the news since Edward Snowden’s leaks, and the popularity of sites such as FiveThirtyEight has popularized data journalism. International development donors have recognized this and are demanding more data from implementing partners and placing greater emphasis on monitoring and evaluation (M&E). While M&E is used to cover a wide variety of activities–from reporting to research–at its core, it is a way to ensure international aid programs are providing effective interventions.

Much like how the data revolution has sparked innovative software for the private sector through NoSQL data storage, software and technological innovation for M&E is beginning in international development. A reflection of conversations during Tech Salon’s M&E discussion show that M&E tend to be afterthoughts to program design because of fear of failure and the lack of funds. However, today technology and M&E are increasingly being requested in international development.

Here are a few reasons why we need to better integrate technology, M&E, and international development.

Greater Transparency

The US government has embraced the data revolution by providing open access to some of its data. This access provides not only greater transparency, but also greater scrutiny over spending and its efficacy. Anyone can easily see how much money the US government is spending on foreign assistance and engage in dialogue on whether the money is being spent properly and effectively. Initiatives such as the International Aid Transparency Initiative (IATI) work towards greater transparency across all donors and have established one standard for donors to report information on monetary flows in development.

Using technology-enabled M&E effectively will allow development implementers to prove program efficacy more quickly and easily. Programs can adapt activities on the basis of real time M&E, providing more benefit to beneficiaries. Global indicators can be used to show impact throughout multiple projects. Visualizations, such as maps, can present the wealth of data collected into an easily understood form.

World Bank Data Visualizer world map

Proving impact and greater accountability helps USAID and other clients justify spending money on development programs to their stakeholders. In turn, the clients can keep funding programs and continue helping people throughout the world.

Data responsibility

Snowden’s revelations have brought the conversation of data responsibility and privacy to the general audience. For development practitioners, as donors request more and more data, we need to think about how to collect the data while protecting the beneficiary. It is important to consider what technology is appropriate for M&E as well as the metadata that it might reveal.

Utilizing technology-enabled M&E is more than including mobile phones into the process. It requires considering what data needs to be collected and whether it can do any harm to a beneficiary if the wrong person gains access to it. Technology and their limitations need to be understood to design data collection and any limitations for data analysis.

Put simply, technology is a tool for the M&E practitioner, not a solution on its own. The concerns about data responsibility are not new to development, but understanding the technology is.

Technology makes practitioners’ lives easier

Most importantly, technology-enabled M&E eases the work of practitioners. Imagine working in the field to collect information and instead of using pen/pencil and paper, you are using a tablet with a data collection app. This app allows you to work without internet connectivity and sync data when connectivity is available. You don’t have to worry about entering any geographic information, because it is either associated with a service location (e.g. school, community center) or the tablet saves your location for each entry.

M&E software companies, such as DevResults and SurveyCTO, create these tools so practitioners can focus on helping beneficiaries instead of recording and transcribing data. Field practitioners no longer need to record the same information in multiple locations and continually check to ensure no transcription errors occurred. Headquarters staff can use different types of data visualizations to more effectively develop theories of change and write reports much quicker.

By providing greater transparency, data responsibility, and making the the practitioners’ lives easier, technology is allowing practitioners to focus less on administrative tasks and more on effective program design.

To learn more about integrating technology and M&E in international development, sign up for our upcoming Technology for Monitoring and Evaluation online course.

Norman Shamas is the course facilitator for TechChange’s Technology for Monitoring and Evaluation online course. He also currently works as a data architect and wrangler to analyze foreign aid data at Creative Associates International. Previously he worked as a graduate student instructor at the University of Minnesota, where he studied identity from theoretical, social science, and policy perspectives. He has extensive experience in Israel and the West Bank where he has worked as an archaeologist and led dialogue groups. Norman speaks Hebrew and Persian and reads numerous dead languages. Norman enjoys telling stories, whether in words, images, or numbers. He has more than five years of experience teaching online and in person and facilitation in the US and abroad.

My current focus on tourism development led me to explore how some of the tech tools discussed in the Mobiles for International Development class can be applied in the tourism industry. In particular, a large part of the tourism assessment and development process involves both evaluating the visitor experience in a destination and examining the attitudes of local residents towards tourism development.

Surveys are the most common tools for carrying out these evaluations, but most of the time they result in stacks of papers that need to be keyed into a computer, introducing errors and wasting valuable time. In the M4D class, we saw a physical example of this where a pickup truck was loaded with stacks of questionnaires.

Today, mobiles and tablets are overcoming the challenges faced by paper-based surveys and evaluations as they bring efficiency, a variety of user-friendly survey platforms, and real-time feedback.

1. Quick and easy access to better processed surveys
Compared with paper questionnaires, a more efficient data collection method would be to use the Formhub tool that we learned about during the course. The additional cost of purchasing a few basic tablets and rugged cases could be offset by savings in labor costs for data entry and the added value of the data being processed in a more timely and accurate manner.

2. Variety of user-friendly survey platforms
The advantage of a tablet over a smartphone is that the tablet more closely resembles a paper format questionnaire, making it easy to hand over to visitors or residents to complete. Formhub can also be used offline; completed questionnaires can be uploaded once a connection is reestablished, making it particularly useful in remote tourism destinations lacking wifi or cell service.

Since visitor surveys are usually carried out in places where large numbers of tourists congregate (city plazas, transportation system waiting areas, etc.), the survey-takers often hand out paper forms to many people simultaneously, presenting a potential disadvantage for Formhub if only a few tablets are available. A potential solution could be a QR code to scan that takes tourists to a web site on their own personal smartphones to complete the questionnaire. This method could be used in conjunction with the tablets (i.e. tablets could be used for those visitors without smart phones). There would have to be measures in place to ensure that the same person doesn’t submit multiple questionnaires, but I think that could be designed relatively simply.

3. Real-time feedback
Another way to make surveys valuable to both tourists and destination planners and developers, would be to couple geolocation with an SMS service. Tourists could opt in to the program upon arrival at a destination, and upon entering certain geofences they would automatically receive an informational text describing the attraction with links to more information if they’re interested. For instance, upon approaching a monument a visitor could receive historical information about the attraction, or upon entering a local market the user could receive a link to a detailed map showing where certain stalls are located. This system could be coupled with an SMS survey system like TextIt. This way, the destination could get real-time feedback from tourists about certain aspects of an attraction as the visitor is experiencing it (i.e. rating scale questions about customer service, facilities, etc.). This would help to eliminate the problem of recall bias that often exists when tourists are asked to recall certain aspects of their trip days (or weeks) after it’s over.

There’s obviously a ton of potential for mobile tech in the context of tourism, from the inspiration and planning stages, to booking and experiencing, to sharing the tourism experience with others. I’m super excited to see what kind of apps and novel technologies will be launched in the next few years to further enhance and add layers of value to the tourism experience.

About Jason Kreiselman

When he’s not backpacking through far-off corners of the planet, Jason Kreiselman works as a digital marketing specialist with Brand USA in Washington DC helping to promote international visitation to the U.S. He also works with the International Institute of Tourism Studies conducting tourism research for public and private sector clients. Jason spent four years in Ecuador as an ICT Advisor to the Peace Corps where he worked to promote small businesses and secure grants for organizations focusing on environmental conservation and sustainable development.

Jason holds a Master of Tourism Administration degree with a concentration in Sustainable Destination Management from The George Washington University. You can find him on LinkedIn here.

Interested in learning more about this topic of digital options for surveys and evaluation? Register now for our Technology for Monitoring & Evaluation online course, which runs 26 January – 20 February 2015.