In August, along with announcing our Tech for M&E Diploma program, we kicked off a M&E Professionals Series, where we will be talking one-on-one with M&E professionals to give you the inside scoop on the industry.

For this second post in the series, we are featuring an interview that one of our alumni, Stephen Giddings conducted, with Janet Kerley, Senior Director, Monitoring and Evaluation Practice at Development and Training Services, Inc (dTS), a Virginia-based consulting organization that does considerable work with USAID.

Janet Kerley
Janet Kerley is a master evaluator and an accomplished trainer in evaluation and performance measurement. As Senior Evaluator in the Monitoring and Evaluation Practice at dTS/Palladium, she provides technical leadership for evaluations in the ME unit, provides technical direction on design and field methods, and supervises the preparation of the evaluation reports. As Chief of Evaluation, Research and Measurement for the Peace Corps, she established an impact evaluation system at Peace Corps.

Ms. Kerley was the Team Leader for Monitoring and Evaluation in the Office of the Director of Foreign Assistance, US Department of State, leading a 200-member inter-agency team to develop standard indicators for the 2007 Foreign Assistance Reform reporting tool. She worked at USAID in the Bureau for Policy and Program Coordination, CDIE and as the Monitoring and Evaluation Office in the Bureau for Africa and the Bureau for Europe and Eurasia. Prior to joining USAID, Ms. Kerley was a Senior Research Associate at Aguirre International. She has lived and worked in many countries in Latin America, Africa

S: How has technology changed the way M & E is conducted over the past decade in international development?
J: The change has been remarkable! A decade ago, most of the data gathering and analysis work was all paper-based, making it difficult, time-consuming, and costly. Especially in overseas environments, it took considerable time and effort to gather, transcribe (and often translate) and analyze the data. But today, the tech tools have made data collection and analysis more efficient and save time and money.
However, there is still a considerable “digital divide” between the much more tech savvy young people and the older professionals originally trained using SPSS (or even earlier)
technologies.

S: Does paper-based data collection still have a place in M&E today?
J: Yes — in certain circumstances paper-based data collection may be preferred.

In very rural areas where electricity may not be available, where batteries for electronic devices cannot be charged or where internet connections or mobile phone services is inconsistent or not available, paper-based data collection is still the best option.

Not everyone is comfortable with data collection using electronic devices, but they may be more open to paper-based questions.

S: What are some of the pitfalls of some of the popular tech-based data collection tools?
J: With so much tech available, it is easy to get carried away.

Some less experienced or less than fully trained data gatherers may lose sight of the fundamental questions the monitoring or evaluation is trying to get at. If evaluators lack sufficient training in sound principles of research, they may be tempted to substitute technology for sound reasoning and good judgment.

Some data collection tech tools may also have a tendency to collect too much data, some of which may be irrelevant to the task at hand. USAID, in particular, is burdened by data overload where data management systems fail to filter out data that is of little use and complicates the monitoring and evaluation practices.

S: What challenges have USAID Missions faced when integrating new technologies into their M& E functions?
J: By and large, USAID Missions have been quite open to technological improvements to M&E functions. That said, there is still a “digital divide” where younger employees (including local staff) who have grown up in the digital age are more comfortable with and more adept at using new technologies to enhance M & E. But more senior and older USAID staff seem generally open to embracing and appreciating the advantages that new technologies can bring to M & E while leaving the technical analysis and the new data gathering tools to younger techie staff. USAID staff have generally been very receptive to training in using new M & E technologies to their advantage.

S: Have new evidenced based technologies made decision making by senior USAID staff easier and more informed?
J: Most USAID Mission Directors recognize the value that good evidence on performance can bring to the achievement of program results, and the added clarity that good data and visually well-presented documentation can bring to decision making.

UNDP in Kigali, Rwanda (Creative Commons image)Photo Source: UNDP in Kigali, Rwanda

S: What are the advantages of mixed methods evaluations?
J: The most important starting point for an evaluation is doing the research required to understand what questions you want answered. Only then should you begin to look at evaluation methodologies to acquire necessary information.

When done at a proper scale, well executed quantitative data collection and analytical methods can bring statistical rigor and clarity. For example, the scale of some of the evaluations done for USAID’s food security (Feed the Future) programs has generally provided reliable data. Unfortunately, USAID Missions sometimes do not make available sufficient budget to assure that sample size for quantitative methods is sufficient to draw reliable conclusions. This is where qualitative methods can help to fill gaps.

Storytelling, an evaluation tool, is one of the most useful qualitative data collection methods. Sometimes quantitative data collection methods do not allow beneficiaries to open up and provide adequate and reliable information, but they react much more positively if they are allowed to tell a story. If you get enough good stories they can provide insights and nuances that purely quantitative methods cannot. Thus mixed method evaluations can provide more reliable evidence of performance than quantitative or qualitative methods.

S: Do you think there is a bias towards quantitative methods in international development because of a lack of free and easy to use qualitative tools?
J: Not at all. Many USAID evaluations make good use of qualitative methodologies. A
decade ago, there was an overuse of “the windshield wiper” approach (an evaluation that is not given time to do adequate field work and they report what they observe “through the windshield.”) to evaluations but more recently qualitative methodologies have become more sophisticated and reliable and can provide a lot of extremely useful information for decision makers.

S: What questions should we be asking to select the best technology for M & E?
J: Evaluation planning should begin with framing the research questions — what is it that we need to learn? The preferred technological solution should be one that can best answer the research questions and must also take into account cultural sensibilities. It is very crucial that technology be viewed as a tool, and not as a substitute, for knowing the basic principles of research.

StephenIDphoto
Stephen Giddings, a TechChange alum, has served for 25 years as a Foreign Service Officer with the USAID, retiring in late 2005. For most of his USAID career, he specialized in managing housing and urban development programs, serving in USAID offices in Panama, Kenya, Cote d’Ivoire, Russia and Rwanda, as well as Washington, D.C. During his last four years with USAID he was the Chief of the Policy Division for USAID’s Africa Bureau.

For the past ten years Mr. Giddings has been an independent consultant providing assistance to the Development Assistance Committee (DAC) of the OECD, and consulted with USAID, the International Real Property Foundation (IRPF), among other international development organizations. He serves on the Development Issues Committee of the USAID Alumni Association and is Co-Chair of the Africa Work Group in the Society for International Development’s Washington, D.C. Chapter (SID-Washington). Prior to his USAID career, Mr. Giddings managed low-income housing development programs at the U.S. Department of Housing and Urban Development and was Director of Planning and Development at the Boston Housing Authority. Mr. Giddings received a BA in political science from Wesleyan University and an MPA degree from the Maxwell School of Citizenship and Public Affairs at Syracuse University.

——-

Hope you enjoyed our second installment of our M&E Professionals Series! Don’t forget to follow our blog for the next post in the series!

Interested in engaging in similar conversations with M&E professionals like Stephen and Janet? Join us in our upcoming course TC211: Technology for Data Collection and Survey Design that starts on October 19. If you want the whole package, you can join our second session of our Tech for M&E Diploma program

At TechChange, we pride ourselves in teaching our participants the crucial skills needed for a career in social good. And how do we find out what those skills are? We go straight to the source! In our new M&E Professionals series, we’ll be talking one-on-one with the pros who are recruiting for these very positions.

2015 was marked as the International Year of Evaluation, so it’s no wonder that M&E is increasingly becoming a sought after skill in many organizations today. We spoke with Michael Klein to learn more about the ideal skills a M&E professional should have and how to get them:

Mike Klein Headshot

Michael Klein is the director of International Solutions Group (ISG), a company that works with governments, U.N. agencies, international organizations, NGOs and other companies to improve the implementation of humanitarian aid and development programming projects.

Mike’s work is at the self-described intersection of ‘old-school’ M&E and ‘new-school’ ICT, working with partners to build on established M&E strategies, streamline data flows and analysis systems, facilitate access to key information, knowledge, reporting and data in a fast, reliable and secure manner.

Q: It’s an exciting time to be an M&E professional. Do you see a big need for young professionals who are trained to take on this kind of work?

M: Yes, it is definitely an exciting time to be in my field. Just looking at the types of conferences and forums being held on the subject, it’s really clear that M&E is a rapidly developing focus in our field. Specifically, I see M&E growing in two separate, but overlapping, areas.

Standard M&E careers: If you were to search job opportunities listed on Devex or Idealist, they are the typical of M&E positions you would find, ones that take a traditional approach (i.e M&E personnel are used to provide managers the with analysis and data they request.) These opportunities are certainly growing, as organizations will always need highly trained staff help to address their M&E.

Beyond the standard label of M&E: Just as in other sectors, skillsets such as analytics, knowledge management, data collection, and information sharing are highly valued in M&E, and the field is increasingly embracing individuals who have these skill sets. This is especially true for people who understand analytics, and how data can be collected, used, and analyzed.

As more and more players in the field are using new technologies and tools for data collection and analysis, a college graduate or young professional entering the field of M&E has a great opportunity to make his or her mark on the industry by leveraging his or her digital knowledge to provide guidance to some of the most prominent development organizations.

Q: What do you look for when hiring?

M: First off, I think it’s important to have a passion, an area of expertise that you enjoy. As a profession, M&E can take you in myriad directions, and it is important to identify what type of work in which you most want to engage. A strength of my team at ISG is that everyone has differing professional interests, ranging from gender equality to how ICTs can catalyze development. Having these range of specialties strengthens what we can offer clients, and when looking at new hires, I look for individuals who are already established or are on their way to becoming an expert in a field or sector.

Aside from passion and subject-matter expertise, people need to appreciate the big picture. When we look to make hires, we want an individual who understands what organizational performance means, ranging from back-office activities such as business development and marketing, all the way through to front-line programming at the field level. If you’re interested in M&E, you have to understand that organizational effectiveness is made possible by a complex interplay of many elements. Having that general appreciation for how organizations function, the types of struggles they face, and how you can improve upon their performance, are keys to success in this field.

isggroupMike with his group at ISG

Q: What does career progression look like for someone in M&E?

M: The reason I was drawn to this field is that there is no set path. Before working in M&E, I worked in Mergers & Acquisitions (M&A), which attracted me for similar reasons. When I worked in M&A, my clients represented a wide range of sectors, and I worked with management to help these organizations restructure, fundraise, find new investors, and generally position themselves to be more successful at what they were already doing (sound familiar?). Monitoring and evaluation is no different. People come to the field from a variety of different backgrounds—such as corporate finance like me, IT, agriculture, and plenty of others, including academic programs focused on M&E—and they serve clients representing the same diversity.

Because everyone enters the field with different skills, it is hard to say exactly how one progresses in the field. Someone entering the field after studying M&E at a university will likely have a fairly technical background may take a M&E support role within a larger organization and begin to assume more responsibility over the years. Whereas, if you’re transitioning into the field with an already established skill set from a different sector, you’re likely to take a different direction and provide either consultancy services or specific project-level guidance related to your expertise.

Q: What role do you think technology plays in M&E?

M: My personal take is that technology in and of itself is not necessarily transformative. Whether M&E is done on pen and paper or done using state-of-the-art IT solutions, good M&E is good M&E. However, technology has allowed organizations to quickly come up to speed with regards to implementing more robust approaches to M&E. If an organization is just shaping their M&E approach, using a technology solution can offer ease and expediency. The way most of these tools are constructed is based on industry best practices, so their structured, hierarchical approach is what we think of as good M&E. Clients who use these tools are then trained to capture their data in a systematic way that has the end in mind.

A lot of our clients initially say, “I really like that heat map,” or cite another specific visualization they see proved by an M&E tool and ask how they can create it for their program. This then launches into a discussion about how to collect data in a way that can deliver these types of reporting and highlight what is most important. It is much harder to go the other way.

Q: Any other pieces of advice you have for people considering the TechChange diploma in Tech for M&E?

M: Know how your study applies to what you want to accomplish and set some goals for yourself before you get into the nuts and bolts of the diploma program. If you come to these courses with a general idea of what skill gaps you want to address, you’re going to be very well placed to make the best use of the TechChange diploma program.

When I was taking one of TechChange’s M&E courses, I knew that I wanted to leverage the new skills I was developing to enhance my company’s marketing efforts. Thus, when I studied data visualization, I created multiple infographics for ISG, using tools that I would not have come across on my own. This was one of the best parts of the class: discovering cutting-edge tools in the field that I could utilize immediately.

———————————

That’s all for this installation in our M&E Professionals Series! Be sure to check out our Technology for Monitoring and Evaluation diploma program – deadline to enroll is September 4!

TechChange Graphic Designer Rachel Roth explains to TechGirls Ghada and Nataly how to draw characters using Adobe Photoshop and Illustrator.

Yesterday, TechChange hosted a job shadow day with Nataly Ayyad and Ghada Missaoui, two incredibly talented young ladies participating in this year’s TechGirls program with the U.S. State Department. This selective exchange program encourages Middle Eastern and North African teenage girls to pursue science, technology, engineering, and math (STEM) careers. Within the next few weeks, they are travelling throughout the U.S. for their first time to visit technology companies such as Google, Facebook, Yahoo, and Instagram.

Nataly and Ghada, hailing from the Palestinian Territories and Tunisia respectively, were selected to participate in this program and spent the day at the TechChange headquarters to learn what it’s like to work at an international edtech social enterprise. Nataly hopes to go into web design and believes that technological skills can be empowering to underprivileged people, especially women. She looks forward to sharing her new skills with them. Ghada is very passionate about astronomy and one day hopes to pursue studies in astrophysics. She has built her own website called “Go Girls Engineering” using Adobe Muse to showcase engineering role models for young students.

2014 TechGirl Ghada and OscarTechGirl Ghada shows Oscar Chen her website, Go Girls Engineering, she created using Adobe Muse. They discuss the pros and cons of using design tools like Muse as opposed to coding by hand with WordPress. 

Throughout the day, Nataly and Ghada met with different members of the TechChange team to learn about digital animation, graphic design, digital photography, web design and programming, interactive infographics, data visualization, search engine optimization (SEO), customer relationship management (CRM) software such as SalesForce, instructional design for eLearning with Articulate Storyline, and more.

Upon arriving to TechChange, the TechGirls were eager to know when we would show them how we do our animation videos.

Alon Askarov explains animation to TechGirls Ghada and NatalyCreative Director Alon Askarov explains how he uses Adobe Edge Animate and After Effects to create TechChange animations.

“My favorite part of my day at TechChange was definitely learning about the animation process!” said Nataly.

Emily Frutcherman guides TechGirls Nataly and Gadha through ArticulateEducation Technologist intern, Emily Fruchterman, shows the TechGirls the online learning courses that TechChange developed with the Asian and Pacific Islander Wellness Center to scale HIV prevention

TechGirls Ghada and Nataly learn about photography with CharlieCharlie Weems demonstrates how to change aperture settings on a DSLR camera.

Cathie Chen explains Articulate to TechGirls Gadha and NatalyCathie Chen shows the TechGirls how to create a hotspot on Articulate Storyline.

Eric and Emily go over a project with TechGirls Nataly and GhadaEric Bihl and Emily Fruchterman do a hands-on session to show how to create triggers and slide layers on Articulate Storyline.

“Everything I learned at TechChange was very interesting, from learning about animations, digital photography, eLearning software, to understanding the diversity of online courses available,” commented Ghada.

TechGirls Group photo

Thanks for joining us, TechGirls!

See a summary of last year’s TechGirls Job Shadow day at TechChange here and click here to learn more about the TechGirls program.

Want to change the world with socially minded and innovative ideas, but bills, student loans and lack of investment opportunities are pushing you away from entrepreneurship and towards jobs that are unfulfilling and not utilizing your education? According to Accenture, you are not alone.

A staggering 41% of college graduates within the past two years are underemployed, which means they are either in jobs that are not full-time or have nothing to do with their degree. The 2008 global financial recession changed the employment landscape, drying up credit for start-ups and diminishing confidence in large institutions. The tepid recovery makes them cautious in hiring. Yet there is a growing demand from within the large organizations for employees who help them remain competitive through new ideas and a start-up mentality. The need has increased the demand for intrapreneurs, innovators from within the company that will move it forward.

Nick Hughes is an example of a social intrapreneur in action. Hughes was a middle marketing manager for Safaricom, the largest mobile network in the East African nation of Kenya when he developed an idea that would help millions of disadvantaged people while driving the bottom line of his company. Hughes’ concept, which became M-PESA (m for mobile, “pesa” is Swahili for money) has become a wildly lucrative, socially beneficial, and intrapreneurial idea which has led Safaricom to become the banker for the poor and rural in Kenya who can not get accounts from traditional banks. As of 2013 the M-PESA service has 17 million members, providing a critical socio-economic service for Kenya’s poor and profits to Safaricom.

The idea of social intrepreneurship is becoming particularly enticing to young people who want stable careers, but care less and less about making a fortune in corporate America and more about making a difference in the world around them. Personally, intrapreneurship has been an incredibly rewarding career choice. In 2009 I lost my job, and spent a year trying to become a social entrepreneur, sleeping on couches, living unemployment check to unemployment check, and trying to launch a company that would deliver social purpose. In 2010, UNICEF picked up on one of the company’s ideas and offered me a short-term consulting opportunity to carry-out an innovative idea. Thus began my path to learning the process of becoming a strong source of new ideas and innovation for the established institution. Three years later I’ve been a part of an amazing international team, got the opportunity to travel to Rwanda, Zambia, and Brazil, and had a consistent paycheck.

Innovating from within UNICEF provided stability and opportunities that I would not have had otherwise. In 2011, from within UNICEF, I led the launch of unicef-gis, a mobile and web application for youth-led digital mapping of risks, resiliency, and vulnerability. The app was deployed successfully into countries (Brazil and Haiti) where I knew nobody and did not speak the language. Without the UNICEF network, the idea would have never come to fruition or realized the amazing social impacts.

I personally understand today’s harsh jobs reality for graduates, interns, and young professionals. The truth is that a six-figure education and degree is only enough to level the playing field, but not enough to secure you a rewarding job that allows you to be independent and pay back student loans. That’s why learning how to practice intrapreneurship in your field can become a novel employment option for those who will be entering the workplace or are already in it, looking to move up into more senior positions. These large institutions like the World Bank, United Nations, and Google are actively looking to hire these intrapreneurial individuals.

People often ask me, how do you become gainfully employed doing cool, innovative projects for a large international organization? The truth is, intrapreneurship is not something learned in the traditional classroom or from a textbook. It’s understood by trial and error, failing forward, and failing fast. These are skill sets that can be practiced and learned, and lead to rewarding and stable jobs. Interested in learning about intrapreneurship or know a student, intern or young professional that might benefit from learning about it? TechChange.org is providing a limited number of individuals the opportunity to learn about intrapreneurship from guest experts and interactive, self-paced online learning modules. Learn more:

TC108: Social Intrapreneurship – Innovation Within Institutions