monitoring and evaluation | TechChange | The Institute for Technology and Social Change

By Zacharia Diing Akol and Ella Duncan

“It is a once in a lifetime opportunity to be older than your country, to raise it and help it grow.” – Zacharia Diing Akol

The crisis that broke out in South Sudan in December 2013 has multiple root causes. A broken sense of national identity and lack of trust in the state at the national level, coupled with intercommunity grievances and mistrust, as well as high level of individual trauma and frustration have resulted in 22 months of a conflict that killed an estimated tens of thousands of people and displaced more than 2 million. South Sudan gained its independence in 2011, yet the cycle of violence stretching back to the First Sudanese Civil War in 1961 continues today. It is in this environment of violence and war that peacebuilder Zacharia Diing Akol was born.

It was a long path for a child in war-torn Jonglei State to become a researcher who shapes the growth of a new nation. At the age of 11 Zacharia left his family to cross into Ethiopia by foot, traveling with groups of other children and young men, all fleeing the Second Sudanese Civil War. His personal journey has taken him around the world and back to South Sudan, where he now works as a Senior Policy Analyst at the Sudd Institute in Juba, supporting good governance and policy as part of what Zacharia calls the “collective journey” to build South Sudan.

When Zacharia first returned to South Sudan, he intended to stay for two weeks, but quickly became convinced his skills as a policy expert and researcher were needed to support the fledging nation. As Zacharia says, “It is a once in a lifetime opportunity to be older than your country, to raise it and help it grow.”

Zacharia believes data can effect positive change by helping people, organizations, and governments make better choices. The pursuit of this belief drove him to co-found the Sudd Institute.

The Sudd Institute is an independent research organization premised on the belief that public policy, especially in this key historical moment of state building in South Sudan, must be informed by reliable data, objective analysis, and thoughtful debate. Backed by data and evidence, Zacharia and the Sudd Institute make policy recommendations to the new government.

Men gather in a South Sudanese IDP Camp, Photo Credit USAID

For Zacharia, giving solid evidence-based recommendations is the best way to effect smart and responsive policy. Furthermore, Zacharia believes evaluations are for posterity, that they capture the realities of the new country – he says, “People will look back in the future and ask, what were they thinking? What were they doing? The work of the Sudd Institute will help people understand what was going on during this time in South Sudan.”

Zacharia hopes that his work will inspire other civil society organizations so that evidence-based policy recommendations become the norm in South Sudan. The next project for Zacharia and the Sudd institute is to include trainings for CSOs so that local organizations can provide their own recommendations for action, and policy advocacy efforts across South Sudan may be strengthened by evidence. Through sharing information and skills, Sudd hopes to make their goals and impact sustainable. By empowering more groups to engage in evidence based advocacy, the institute’s work strengthens public policy lobbying power to inform government decisions.

The Sudd Institute also works though public research publications, including a weekly review, monthly brief, and quarterly special report. These publications aim to give everyone – from civil society organizations (CSOs) to government officials – the research they need to make informed decisions about the trajectory of South Sudan. Moving forward, the Institute is increasing the influence of their work by forming more direct relationships with CSOs using their publications.

These goals are especially important in South Sudan because as a new country, that was formed by and continues to be defined by conflict, “It is not just important to end conflict, it is about HOW you end it. The process is as important as the ending.” Zacharia sees that there must be a collective effort to address the issues candidly with honest data and evaluation, because he says the country has this choice: “deal with the issues, or the issues will deal with you”.

“It is not just important to end conflict, it is about HOW you end it. The process is as important as the ending.”

Visit the Sudd Institute’s website at http://www.suddinstitute.org/
Read their policy briefs and other publications at http://www.suddinstitute.org/publications/

Zacharia Diing Akol is the Director of Training at the Sudd Institute. Diing has extensive experience in community outreach, government and organizational leadership. He is currently working on M.Res./Ph.D. in political science at the London School of Economics. Diing’s research interests include the role of civil society organizations in peacebuilding, traditional leadership and democratic governance, post-conflict reconstruction, faith and public policy, and the dynamics of civil war. Diing holds a Master’s degree in Peace and Justice Studies from the University of San Diego and two Bachelor’s degrees from Michigan State University in Public Policy & Administration and Policy & Applied Economics.

Ella Duncan is the DME for Peace Project Manager, DME for Peace is a project of Search for Common Ground which connects a growing global community of over 4,000 members to over 800 resources on Design, Monitoring, and Evaluation for Peace and Peacebuilding programming. Ella received her B.S. from Cornell University.

This post originally appeared on DME for peace.

Featured image: Daniel X. O’Neil (Creative Commons License)

In August, along with announcing our Tech for M&E Diploma program, we kicked off a M&E Professionals Series, where we will be talking one-on-one with M&E professionals to give you the inside scoop on the industry.

For this second post in the series, we are featuring an interview that one of our alumni, Stephen Giddings conducted, with Janet Kerley, Senior Director, Monitoring and Evaluation Practice at Development and Training Services, Inc (dTS), a Virginia-based consulting organization that does considerable work with USAID.

Janet Kerley is a master evaluator and an accomplished trainer in evaluation and performance measurement. As Senior Evaluator in the Monitoring and Evaluation Practice at dTS/Palladium, she provides technical leadership for evaluations in the ME unit, provides technical direction on design and field methods, and supervises the preparation of the evaluation reports. As Chief of Evaluation, Research and Measurement for the Peace Corps, she established an impact evaluation system at Peace Corps.

Ms. Kerley was the Team Leader for Monitoring and Evaluation in the Office of the Director of Foreign Assistance, US Department of State, leading a 200-member inter-agency team to develop standard indicators for the 2007 Foreign Assistance Reform reporting tool. She worked at USAID in the Bureau for Policy and Program Coordination, CDIE and as the Monitoring and Evaluation Office in the Bureau for Africa and the Bureau for Europe and Eurasia. Prior to joining USAID, Ms. Kerley was a Senior Research Associate at Aguirre International. She has lived and worked in many countries in Latin America, Africa

S: How has technology changed the way M & E is conducted over the past decade in international development?
J: The change has been remarkable! A decade ago, most of the data gathering and analysis work was all paper-based, making it difficult, time-consuming, and costly. Especially in overseas environments, it took considerable time and effort to gather, transcribe (and often translate) and analyze the data. But today, the tech tools have made data collection and analysis more efficient and save time and money.
However, there is still a considerable “digital divide” between the much more tech savvy young people and the older professionals originally trained using SPSS (or even earlier)
technologies.

S: Does paper-based data collection still have a place in M&E today?
J: Yes — in certain circumstances paper-based data collection may be preferred.

In very rural areas where electricity may not be available, where batteries for electronic devices cannot be charged or where internet connections or mobile phone services is inconsistent or not available, paper-based data collection is still the best option.

Not everyone is comfortable with data collection using electronic devices, but they may be more open to paper-based questions.

S: What are some of the pitfalls of some of the popular tech-based data collection tools?
J: With so much tech available, it is easy to get carried away.

Some less experienced or less than fully trained data gatherers may lose sight of the fundamental questions the monitoring or evaluation is trying to get at. If evaluators lack sufficient training in sound principles of research, they may be tempted to substitute technology for sound reasoning and good judgment.

Some data collection tech tools may also have a tendency to collect too much data, some of which may be irrelevant to the task at hand. USAID, in particular, is burdened by data overload where data management systems fail to filter out data that is of little use and complicates the monitoring and evaluation practices.

S: What challenges have USAID Missions faced when integrating new technologies into their M& E functions?
J: By and large, USAID Missions have been quite open to technological improvements to M&E functions. That said, there is still a “digital divide” where younger employees (including local staff) who have grown up in the digital age are more comfortable with and more adept at using new technologies to enhance M & E. But more senior and older USAID staff seem generally open to embracing and appreciating the advantages that new technologies can bring to M & E while leaving the technical analysis and the new data gathering tools to younger techie staff. USAID staff have generally been very receptive to training in using new M & E technologies to their advantage.

S: Have new evidenced based technologies made decision making by senior USAID staff easier and more informed?
J: Most USAID Mission Directors recognize the value that good evidence on performance can bring to the achievement of program results, and the added clarity that good data and visually well-presented documentation can bring to decision making.

Photo Source: UNDP in Kigali, Rwanda

S: What are the advantages of mixed methods evaluations?
J: The most important starting point for an evaluation is doing the research required to understand what questions you want answered. Only then should you begin to look at evaluation methodologies to acquire necessary information.

When done at a proper scale, well executed quantitative data collection and analytical methods can bring statistical rigor and clarity. For example, the scale of some of the evaluations done for USAID’s food security (Feed the Future) programs has generally provided reliable data. Unfortunately, USAID Missions sometimes do not make available sufficient budget to assure that sample size for quantitative methods is sufficient to draw reliable conclusions. This is where qualitative methods can help to fill gaps.

Storytelling, an evaluation tool, is one of the most useful qualitative data collection methods. Sometimes quantitative data collection methods do not allow beneficiaries to open up and provide adequate and reliable information, but they react much more positively if they are allowed to tell a story. If you get enough good stories they can provide insights and nuances that purely quantitative methods cannot. Thus mixed method evaluations can provide more reliable evidence of performance than quantitative or qualitative methods.

S: Do you think there is a bias towards quantitative methods in international development because of a lack of free and easy to use qualitative tools?
J: Not at all. Many USAID evaluations make good use of qualitative methodologies. A
decade ago, there was an overuse of “the windshield wiper” approach (an evaluation that is not given time to do adequate field work and they report what they observe “through the windshield.”) to evaluations but more recently qualitative methodologies have become more sophisticated and reliable and can provide a lot of extremely useful information for decision makers.

S: What questions should we be asking to select the best technology for M & E?
J: Evaluation planning should begin with framing the research questions — what is it that we need to learn? The preferred technological solution should be one that can best answer the research questions and must also take into account cultural sensibilities. It is very crucial that technology be viewed as a tool, and not as a substitute, for knowing the basic principles of research.

Stephen Giddings, a TechChange alum, has served for 25 years as a Foreign Service Officer with the USAID, retiring in late 2005. For most of his USAID career, he specialized in managing housing and urban development programs, serving in USAID offices in Panama, Kenya, Cote d’Ivoire, Russia and Rwanda, as well as Washington, D.C. During his last four years with USAID he was the Chief of the Policy Division for USAID’s Africa Bureau.

For the past ten years Mr. Giddings has been an independent consultant providing assistance to the Development Assistance Committee (DAC) of the OECD, and consulted with USAID, the International Real Property Foundation (IRPF), among other international development organizations. He serves on the Development Issues Committee of the USAID Alumni Association and is Co-Chair of the Africa Work Group in the Society for International Development’s Washington, D.C. Chapter (SID-Washington). Prior to his USAID career, Mr. Giddings managed low-income housing development programs at the U.S. Department of Housing and Urban Development and was Director of Planning and Development at the Boston Housing Authority. Mr. Giddings received a BA in political science from Wesleyan University and an MPA degree from the Maxwell School of Citizenship and Public Affairs at Syracuse University.

——-

Hope you enjoyed our second installment of our M&E Professionals Series! Don’t forget to follow our blog for the next post in the series!

Interested in engaging in similar conversations with M&E professionals like Stephen and Janet? Join us in our upcoming course TC211: Technology for Data Collection and Survey Design that starts on October 19. If you want the whole package, you can join our second session of our Tech for M&E Diploma program.

At TechChange, we pride ourselves in teaching our participants the crucial skills needed for a career in social good. And how do we find out what those skills are? We go straight to the source! In our new M&E Professionals series, we’ll be talking one-on-one with the pros who are recruiting for these very positions.

2015 was marked as the International Year of Evaluation, so it’s no wonder that M&E is increasingly becoming a sought after skill in many organizations today. We spoke with Michael Klein to learn more about the ideal skills a M&E professional should have and how to get them:

Michael Klein is the director of International Solutions Group (ISG), a company that works with governments, U.N. agencies, international organizations, NGOs and other companies to improve the implementation of humanitarian aid and development programming projects.

Mike’s work is at the self-described intersection of ‘old-school’ M&E and ‘new-school’ ICT, working with partners to build on established M&E strategies, streamline data flows and analysis systems, facilitate access to key information, knowledge, reporting and data in a fast, reliable and secure manner.

Q: It’s an exciting time to be an M&E professional. Do you see a big need for young professionals who are trained to take on this kind of work?

M: Yes, it is definitely an exciting time to be in my field. Just looking at the types of conferences and forums being held on the subject, it’s really clear that M&E is a rapidly developing focus in our field. Specifically, I see M&E growing in two separate, but overlapping, areas.

Standard M&E careers: If you were to search job opportunities listed on Devex or Idealist, they are the typical of M&E positions you would find, ones that take a traditional approach (i.e M&E personnel are used to provide managers the with analysis and data they request.) These opportunities are certainly growing, as organizations will always need highly trained staff help to address their M&E.

Beyond the standard label of M&E: Just as in other sectors, skillsets such as analytics, knowledge management, data collection, and information sharing are highly valued in M&E, and the field is increasingly embracing individuals who have these skill sets. This is especially true for people who understand analytics, and how data can be collected, used, and analyzed.

As more and more players in the field are using new technologies and tools for data collection and analysis, a college graduate or young professional entering the field of M&E has a great opportunity to make his or her mark on the industry by leveraging his or her digital knowledge to provide guidance to some of the most prominent development organizations.

Q: What do you look for when hiring?

M: First off, I think it’s important to have a passion, an area of expertise that you enjoy. As a profession, M&E can take you in myriad directions, and it is important to identify what type of work in which you most want to engage. A strength of my team at ISG is that everyone has differing professional interests, ranging from gender equality to how ICTs can catalyze development. Having these range of specialties strengthens what we can offer clients, and when looking at new hires, I look for individuals who are already established or are on their way to becoming an expert in a field or sector.

Aside from passion and subject-matter expertise, people need to appreciate the big picture. When we look to make hires, we want an individual who understands what organizational performance means, ranging from back-office activities such as business development and marketing, all the way through to front-line programming at the field level. If you’re interested in M&E, you have to understand that organizational effectiveness is made possible by a complex interplay of many elements. Having that general appreciation for how organizations function, the types of struggles they face, and how you can improve upon their performance, are keys to success in this field.

Mike with his group at ISG

Q: What does career progression look like for someone in M&E?

M: The reason I was drawn to this field is that there is no set path. Before working in M&E, I worked in Mergers & Acquisitions (M&A), which attracted me for similar reasons. When I worked in M&A, my clients represented a wide range of sectors, and I worked with management to help these organizations restructure, fundraise, find new investors, and generally position themselves to be more successful at what they were already doing (sound familiar?). Monitoring and evaluation is no different. People come to the field from a variety of different backgrounds—such as corporate finance like me, IT, agriculture, and plenty of others, including academic programs focused on M&E—and they serve clients representing the same diversity.

Because everyone enters the field with different skills, it is hard to say exactly how one progresses in the field. Someone entering the field after studying M&E at a university will likely have a fairly technical background may take a M&E support role within a larger organization and begin to assume more responsibility over the years. Whereas, if you’re transitioning into the field with an already established skill set from a different sector, you’re likely to take a different direction and provide either consultancy services or specific project-level guidance related to your expertise.

Q: What role do you think technology plays in M&E?

M: My personal take is that technology in and of itself is not necessarily transformative. Whether M&E is done on pen and paper or done using state-of-the-art IT solutions, good M&E is good M&E. However, technology has allowed organizations to quickly come up to speed with regards to implementing more robust approaches to M&E. If an organization is just shaping their M&E approach, using a technology solution can offer ease and expediency. The way most of these tools are constructed is based on industry best practices, so their structured, hierarchical approach is what we think of as good M&E. Clients who use these tools are then trained to capture their data in a systematic way that has the end in mind.

A lot of our clients initially say, “I really like that heat map,” or cite another specific visualization they see proved by an M&E tool and ask how they can create it for their program. This then launches into a discussion about how to collect data in a way that can deliver these types of reporting and highlight what is most important. It is much harder to go the other way.

Q: Any other pieces of advice you have for people considering the TechChange diploma in Tech for M&E?

M: Know how your study applies to what you want to accomplish and set some goals for yourself before you get into the nuts and bolts of the diploma program. If you come to these courses with a general idea of what skill gaps you want to address, you’re going to be very well placed to make the best use of the TechChange diploma program.

When I was taking one of TechChange’s M&E courses, I knew that I wanted to leverage the new skills I was developing to enhance my company’s marketing efforts. Thus, when I studied data visualization, I created multiple infographics for ISG, using tools that I would not have come across on my own. This was one of the best parts of the class: discovering cutting-edge tools in the field that I could utilize immediately.

———————————

That’s all for this installation in our M&E Professionals Series! Be sure to check out our Technology for Monitoring and Evaluation diploma program – deadline to enroll is September 4!

If you have taken a TechChange course, you know that the participants are all doing amazing things wherever they are in the world. Some go on to start their own organization, some collaborate with other participants for future projects, and some take what they learned in the course and apply it in their current projects. Ameneé Siahpush took our Tech for M&E online course in January and has since been leading tech integration in Trickle Up’s M&E programs.

Tell us about yourself

A: I’m a Pacific Northwesterner who moved to New York City in 2010 after spending the prior few years in Latin America. My current role at Trickle Up is Senior Monitoring & Evaluations (M&E) Officer, where I support our economic and social empowerment programs in India and Central America. My work aims to increase our understanding of sustainable livelihood development for highly vulnerable populations, including outcomes around food security, health, coping mechanisms, and social empowerment. I’m particularly interested in expanding our use of participatory methods to improve and deepen our program learnings and developing simple mechanisms for sharing knowledge across participants, partners, staff, and offices. (If you have any ideas, please let me know!)

What does Trickle Up do?

A: Trickle Up is an international NGO that works to create a world in which it is unacceptable for anyone to live in extreme poverty. In collaboration with local partner organizations, we empower and support the poorest and most vulnerable people to develop the confidence and knowledge to build sustainable livelihoods by 1) providing training, coaching, and seed capital grants to jumpstart microenterprises; 2) forming savings and credit groups to build financial capital and literacy; and 3) improving access to information and financial, health, and social services. We also provide technical assistance to other development organizations and government agencies to help them deliver social empowerment and economic programming that reaches “last mile” populations, including women, people with disabilities, and marginalized ethnic populations living on under $1.25/day in rural areas. Trickle Up currently works in India, Central America, South America, West Africa, and the Middle East.

How did you hear about TechChange?

A: My colleague at Trickle Up learned about the Technology for M&E course through a Yahoo M&E group, and quickly forwarded me the information given my interest in the topic.

Why did you decide to enroll in the Tech for M&E course?

A: I feel very fortunate to work for an organization that has invested in a robust M&E system, including the use of mobile data collection for some of our projects. However, as we scale our programs, it’s essential that we adapt our M&E systems to become more efficient and effective across an increasingly large and diverse number of partners and program participants. Integrating new technologies and tools is key in this adaptation process – yet, I knew that I needed very practical guidance in understanding which combination of technologies and tools would be best suited for Trickle Up’s current and future programs. The Tech for M&E course felt like the perfect companion for exploring these issues. It offered practical tools and resources, connection to a wide network of experts, forums to collaborate with other NGOs, and flexible access to course materials to accommodate my travel schedule. I also really appreciated that the discussions were geared towards international organizations who often work in remote, rural places where connectivity and electricity challenges must be considered in their M&E tools.

How has the course impacted your work at Trickle Up?

A: I entered the course with a deep interest in exploring technologies to increase the efficiency and quality of our M&E data. I came out of the course with the language, framework, tools, and resources to actually take the lead in designing and implementing new technologies within Trickle Up’s M&E system. Since completing the course, I have successfully added “M&E tech upgrades” into our upcoming year’s strategic plans. This includes a detailed roadmap of how we will integrate and utilize mobile data collection and a data visualization/reporting platform across all of our projects to increase access to real-time data for project management, promote cross-regional learning, and, ultimately, improve our ability to direct resources towards combating extreme poverty. Yes, it’s a very lofty goal, but one that is greatly enabled by simple technologies that help to ensure our program data is more efficiently and effectively used.

What would be an advice to other participants taking a TechChange course? How can they get the most out of it?

A: If possible, approach the course with a specific, tangible challenge that you hope to confront in your daily work. Keep this challenge in mind as you choose which webinars to attend or resources to explore, and then organize your course notes in a way that will be easily accessible in the future.

Another obvious, but important, suggestion is to be an active participant! Connect with fellow students, ask questions, follow up with presenters, experiment with the recommended tools. Luckily, the course provides a wide variety of ways to engage with the materials and people, despite being in different time zones, and everyone felt very approachable and enthusiastic. We’re all current or future tech nerds, after all.

You can join participants like Ameneé in our next Tech for M&E course in September. If you are looking to dive deeper, check out our brand new Diploma Program in Tech for M&E!

About Ameneé

Ameneé is the Senior Monitoring & Evaluations Officer at Trickle Up, where she supports their economic and social empowerment programs in India and Central America. She holds a BA in sociology and psychology from the University of Oregon and an MPA, with a specialization in international policy and management, from the Wagner School of Public Service at New York University (NYU). As an NYU Gallatin Global Fellow in Human Rights, Ameneé partnered with Global Workers Justice Alliance to conduct research on gender and migration in Oaxaca, Mexico, and has spent multiple years in Latin America, more broadly, volunteering with small-scale farmers and studying Spanish. Prior to Trickle Up, Ameneé was a Program Evaluator at Morrison Child & Family Center in Portland, OR, and a Research Supervisor at the Oregon Social Learning Center. Outside of work, Ameneé loves to play soccer, dance, and spend time in the mountains.

Meet Jennifer, she took her first TechChange course on Technology for Conflict Management and Peacebuilding in October and is now facilitating multiple TechChange courses.

Drawn by our teaching model, after completing her course, she wanted to become involved as a facilitator for our courses. She is currently co-facilitating TC111: Technology for Monitoring and Evaluation with Norman Shamas, and facilitating TC105: Mobiles for International Development. Jennifer will also be facilitating TC109: Technology for Conflict Management and Peacebuilding in the coming months, bringing her full-circle in her participant-to-facilitator involvement with TechChange.

Prior to joining TechChange, Jennifer participated in several research symposiums and conferences like the Institute for Qualitative and Multi-Methods Research, the Association for the study of the Middle East and Africa Annual Conference and more. She has also served as a guest speaker for the American Red Cross and has mentored several high school and undergraduate students regarding school-sponsored and independent international development and peacebuilding start-ups.

Jennifer is an emerging comparative politics scholar and methodologist focused on answering questions related to individual and community involvement in conflict, post-conflict, and peace processes. She holds a Bachelor’s in Political Science from the Colorado College, a Masters in Public Health from Indiana University, and is a Doctoral Candidate in Political Science with the University of New Mexico.

By Norman Shamas and Samita Thapa

In a previous post, we wrote about why global development practitioners need to be data skeptics. One of the many reasons that we need to be skeptical about the data we are collecting is the biases that are incorporated in the data. The data bias is especially significant when it comes to gender data. Women and groups that don’t identify with binary genders are largely missing or misrepresented in global development data.

Data is a crucial component of any program or intervention. It justifies the need for a specific program, show its effectiveness, and allows us to improve it through evaluation. But this important information tells us little if more than half of the population is missing or misrepresented, so we need to start looking at data with a gender lens.

Data on women raises awareness of women related issues

With 62 million girls not attending school worldwide, the U.S. government was able to justify their “Let Girls Learn” initiative. This initiative was announced in February and is aimed at making education a reality for girls around the world. USAID is one of the agencies involved in the government-wide initiative and have presented their approach with data to support it.

But there is still a problem getting good data on women. GSMA’s 2015 Bridging the Gender Gap Report highlights two systemic barriers to mobile ownership and usage for women:

lack of disaggregated data and
lack of focus on women as a market.

However, we need better gender data for more than just the economy. Oxfam conducted a household survey on the gendered effects of the December 26, 2004 tsunami that hit several Asian countries. Women were found to be more severely affected than men. Despite the need for better gender data in the field, it is not always happening. Lack of data on women leads to lack of awareness of issues related to women and consequently, lack of programs designed to tackle those issues.

Survey design can promote non-binary gender inclusion

The problem of gender and data bias gets even more complex when we talk about non-binary genders. Twitter, for example, determines its users’ gender based on data it analyzes from tweets. There are only two gender options: male and female, and users cannot choose to opt out from automatic gender assignment or manually choose their gender. By the simple fact that Twitter is using a gender binary of male/female, individuals who do not identify with a binary (e.g., transgender individuals) or have anatomically mixed sexual characteristics (i.e., intersex individuals) are ignored in the data.

It is important to ask questions about gender on a survey to improve interventions. Instead of restricting gender to a binary, a third option to opt-out or define oneself as ‘other’ can be instituted. When appropriate, additional questions can be used to determine whether practice and self-identification fit into pre-defined categories.

Data must represent local gender categories

It is also important to localize the survey where gender categories and practices may vary. India acts as a good case study for the difficulties in language for demographic purposes. India initially provided three gender options: male, female, and eunuch on its passport forms. However, these three categories marginalized other transgender populations, so in 2014 Indian courts changed the category of ‘eunuch’ to ‘other’ on the election ballots. This simple change in language not only promotes the human rights of India’s non-binary gender individuals, but also provides better data on its non-binary gender communities.

The hijra are a transgender community that has existed in South Asia for over 4,000 years. Along with a few ‘Western’ countries, at least four South Asian countries — Nepal, India, Pakistan, and Bangladesh — recognize a third gender in some legal capacity.

Global development is moving forward with programming for non-binary gender communities. The Swedish International Development Cooperation Agency put out an action plan for working with lesbian, gay, bisexual and transgender (LGBT) issues from 2007-2009. Last year USAID announced its LGBT Vision for Action, a general roadmap of how the donor would support non-binary gender communities. As programming for non-binary gender communities continues and increases, we need to think closely about the language we use and how we collect data on gender to create effective, data-driven interventions.

With development becoming more data driven, the data we collect and the biases we include in the data are having a larger impact. Technology can make these biases more entrenched through features like data validation. Even though data validation is important for survey collection–it limits responses to particular choices or data types (e.g., phone numbers)–it also restricts options based on the choices of the survey creator and can marginalize groups whose identities are not included or allowed as a valid option. Going forward, we need to be careful we are not unintentionally marginalizing other groups or genders with the data we collect.

Interested in engaging in similar conversations around data and tech in M&E? Join us and more than 90 other international development practitioners in our upcoming course on Technology for Monitoring and Evaluation.

Image Source: Global Forest Watch

Open data has been a popular topic in international development recently, but what about the evaluation of open data programs? We are excited to welcome Dow Maneerattana from World Resources Institute (WRI) to our upcoming Tech for M&E online course to talk about evaluations of open data movements. Before her guest expert session next week in our course, we asked her for a little sneak peek:

What does World Resources Institute do?

World Resources Institute is a global research organization that turns big ideas into action! We focus on six critical issues at the nexus of environment and development: food, forest, water, climate, energy, and cities and transport.

What is your role at WRI?

I work on one of the critical issues of our time: deforestation. We have very big ideas about how to reduce deforestation, and I have a unique opportunity to build monitoring and evaluation (M&E) processes and systems from the ground up to ensure that we are capturing results and impact, and adapt our strategy and approach over time. I am involved with WRI’s M&E advocacy, trainings on basic functions of M&E at all stages of a project, and finding strategic opportunities to learn from our work through evaluations.

What will you be speaking about in the course?

I’m excited to share my passion for evaluations (and geek out) with the course participants. I will talk about emerging trends and impacts of open data movements in developing countries and discuss our initiative evaluation design, which includes Global Forest Watch, an interactive forest monitoring and alert system and open data portal. Global Forest Watch leverages remote sensing, big data and algorithms to produce the timeliest and most precise information about the status of forest, including near real time suspected locations of recent tree cover loss.

What excites you the most about Tech for M&E?

I am most excited about the opportunity to shift the dialogue on monitoring and evaluation, which often gets a bad reputation for how it is full of confusing terminology or synonymous with donor requirements. Once you move past the superficial fear, monitoring and evaluation tools and processes are common sense solutions; and when they are coupled with appropriate technology can create one powerful, time-saving machine. It’s been fascinating to see many ways smart techies create and apply tech for M&E solutions for international development. For me, I enjoy connecting the dots and putting powerful tools in the hands of people with the least power.

We are looking forward to Dow’s session in our Tech for M&E online course that begins on Monday, April 20. Want to engage in conversations about the use of tech in M&E with guest experts like Dow? Join us in our upcoming online course on Technology for Monitoring and Evaluation.

About Dow

Dow is a monitoring and evaluation manager at the World Resources Institute, a research organization that works on issues at the intersection of environment and development. She loves to empower teams to systematically capture success stories as well as learn from evaluations and often draws from her international experience as a community organizer and human and labor rights activist. Dow has worked on impact and performance evaluations of governance projects at the National Democratic Institute and on rule of law, human and labor rights advocacy, and livelihoods projects with Pact and Human Rights and Development Foundation. In her free time, you can find her playing tennis, glamping/camping, or mentoring an 8th grader through Asian American Lead.

Wondered who designs and teaches our online courses? You met one of our course facilitators Kendra Keith in a previous post, and today we introduce you to Norman Shamas.

As TechChange’s Director of Curriculum and Pedagogy, Norman brings his passion for education to designing online courses. Norman facilitates our popular online course on Technology for Monitoring and Evaluation and will be co-facilitating our newest course on Technology for Data Visualization this summer. He also facilitated our previous Mapping for International Development course and our customized course on Technology for International Development for IREX.

Norman has quickly become a thought leader in the field of technology for social good. He was recently invited to be part of the review committee for the Development Impact Lab’s Spring Innovate Grant. Additionally, he is also helping lead important conversations on gender and digital security and bringing important techniques and technologies from data science into conversations around monitoring and evaluation.

Before joining TechChange last year, Norman was at Creative Associates International where he developed an internship program. He completed his graduate studies at the University of Minnesota (where he was a graduate student instructor) and undergraduate studies at Arizona State University.

Join Norman in his upcoming Tech for M&E online course that begins Monday, April 20!

Simply including technology in your M&E plan does not lead to better M&E. This was the mantra as I started TechChange’s Tech for M&E online course in January. I heard about this online course from a colleague, and since Commonwealth of Learning is constructing a new six year strategic plan with a crucial M&E component, enrolling in the course was a no brainer. I enrolled and entered a community of more than 180 international development professionals from over 50 countries. All of us in the course agreed that technology is not the main focus for M&E but rather an important enabler to collecting, analyzing and presenting data.

At the end of the course, I reflected on things you must do to successfully integrate technology in M&E:

1. Establish trust between different stakeholders

When it comes to using technology for data collection, data security and data privacy are very important concerns. As stakeholders are requiring stronger evidence of impact and the use of data to make informed decisions, building a partnership based on trust among all stakeholders is one way to ensure regular flow of data. This includes having all confidentiality and transparency issues addressed with the stakeholders. All stakeholders need to have confidence in each other and in the process and know that their data will be protected, so there must be enough time and resources for engagement with the stakeholders.

2. Spend adequate time in preparation

Another key element is the level of preparation that is required for any technology use for M&E. The use of checklists, ICT in the different steps of a program/project cycle (diagnosis, planning, implement/monitor, evaluate, report/share lessons) and using an M&E expert are all important. All ethical and cultural issues need to be raised with the stakeholders and to the extent possible, addressed upfront.

It is important to ask these questions as you prepare your M&E plan:

How does M&E fit into the organization’s strategy and contribute to achieving the goal?
What training and support are required so the field workers are trained, assessing if they have good relationships with the community and what technology will enable best data collection?
What role will the field workers play to increase responses to qualitative and quantitative data collection?
How does one build confidence with all partners so that the quality of data collected enables better analysis and results?

The course checklist of having a quality M&E plan, a valid design to achieve the results and determining the appropriate technology is very helpful. It also emphasizes the importance of testing this with the stakeholders and identifies how to collect data at the different levels of the project and ensure there is a strong motivation to participate.

3. Consider a mixed method approach to data collection

The key focus of my work is in education, both formal and non-formal. So far, using quantitative data has been the norm for any analysis of education provision and its value to society. But mixed method data collection (quantitative and qualitative) is being increasingly used in education, especially to inform the teaching and learning processes. It allows for an understanding of classroom practices, the learner’s context and its impact on education, offering a more holistic view of whether a project has improved learner performance and learning.

Using ICT for mixed methods of data collection has made the processes easier and ensured easier data storage, tagging, and analysis. Mobile phones can be used to record, transmit and tag the data, and data storage platforms can provide easier access to data, aggregate the responses, and analyze.

The course points to four key questions when designing a mixed method evaluation:

At what stages/s does one use the mixed method?
Are qualitative and quantitative methods used sequentially or concurrently?
Will each method have equal weighting or will one method be more dominant?
Will the design be at a single or multi-level?

Just plugging technology in your M&E plan doesn’t do much, but there are steps you can take to integrate technology into M&E so that you can collect and analyze data better. These were the takeaways from the course for me, but the course offered many useful insights into the use of technology for M&E like valuable checklists, platforms that can be used, issues to address for the successful use of technology and a focus on mobile phones. I would recommend the course to anyone looking to learn more about the use of technology in monitoring and evaluation.

Author bio

Vis Naidoo is the Vice-President at Commonwealth of Learning in Vancouver, Canada. He has spent much of the past 20 years involved in the development of open and distance learning systems, educational technology policy and the applications of technology to education – both in South Africa and internationally.

As an online learning community, we are always looking for ways our students can learn more effectively. Many of our amazing alumni, who take our online courses on top of their full-time jobs, have used what they learned in our courses to accelerate their careers, or successfully launch or complete various projects. But we seldom see how our students are learning. We came across the work of Katherine Haugh in our twitter feed when she posted a picture of her graphic notes for our on – demand course on Mobile Data Solutions.

Since these notes have been helpful for Katherine, we wanted to make them available for the TechChange community, we are excited to have Katherine help us as our graphic facilitator for some of our live events in our upcoming courses. She already took notes for one of our live sessions with guest expert, Vanessa Corlazzoli of Search for Common Ground in our Tech for M&E online course.

A snapshot of Katherine's notes for our guest expert session with Vanessa Corlazzoli in our Tech for M&E online course

Some of Katherine’s notes for our guest expert session with Vanessa Corlazzoli in our Tech for M&E online course

Since taking notes by hand is found to be more beneficial than taking notes on the computer, we sat down with Katherine to learn more about her note taking strategies and any tips she has for those of us who would like to get started.

How did you hear about TechChange?
I heard about TechChange through Twitter. I am new to the M&E field, so I am constantly looking for online courses that relate to evaluation, especially ones that focus on combining technology and international development initiatives.

How long have you been taking notes by hand?
I have been taking notes by hand since I was in the fifth grade. In middle school, I took all of my notes by hand and always carried a bag of multi-colored pens around with me. What I like most about note-taking is being able to use my notes as a learning or discussion tool (for myself and others). When I was in college, I worked as a note-taker through the McBurney Disability Resource Center at the University of Wisconsin-Madison. I worked as a notetaker for a variety of courses–from Microeconomics to Global Health and Poverty–and I loved it. (Not to mention, I was being paid to attend my own classes. It’s hard to beat that!). Drawing out historical events or evaluation plans helps me to visualize and better understand what I am thinking and also allows me to share my ideas and thoughts with others in a concise and creative way. I enjoy taking extremely complicated issues or concepts and making them simple and easily digestible.

Why handwritten note taking? What are the pros and cons?
Many of my friends and professors asked me this same question. There are countless pros to taking notes by hand:

You understand better. It requires that you understand and think through what you are writing because everything you do is intentional when you takes notes by hand.
You have more control over the style and layout of your notes. There is more room for creative expression.
It forces you to learn as you write — which is the main purpose of taking notes.

The major downsides to taking notes are hand cramps (yes, this is a thing) and not being able to catch every word, detail or concept. You have to work speedily and you run the risk of missing some points.
However, if you are able to capture the major points (for yourself to remember later or to relay to others) you’ve done your job.

What are some of your strategies to take great notes?
The three important strategies I suggest are:

Find the right mix for you (and your audience): For me, writing on a white paper with enough text and symbols works the best. Remember to keep the notes short, and have triggers in key words, images, or symbols to help jog your memory when looking them over.
Get the right tools: Try out different options to know what works best, I like using darker base colors, and bright colors to highlight main points.
Make it personal: Write notes in your own words, I also include dorky jokes in my notes to help me recall what I wrote.

What is your advice for people who want to get started with note taking?
Just give it a go! You’ll be surprised by how easy it is and remember, you don’t have to be an artist to be a brilliant note-taker. Take a concept that you know very well and try to draw it out. It could be in list format or with pictures–whatever makes the most sense to you!

If you want to start taking beautiful notes but can’t stay away from tech, you can also use MindManager as one of our other alumni, Daniel Acosta did in our previous Tech for M&E course.

You can see more of Katherine’s beautiful notes on her blog. Stay tuned to see more of Katherine’s notes for TechChange’s upcoming courses as well!

About Katherine

Katherine Haugh graduated Phi Beta Kappa from the University of Wisconsin-Madison with degrees in Political Science, International Studies, and Professional Chinese Communication in May of 2014. As an undergraduate, Katherine developed an interest in a wide range of security issues—from nuclear non-proliferation to counterterrorism—as well as a regional interest in South Asia. She is currently a merit scholar at the International Student House and works as a Research Assistant at Innovation Network, a nonprofit evaluation firm in Washington, D.C. She is a long distance runner, board game enthusiast, hiker and lover of gummy vitamins.