Evaluation/Program Assessment

The ability to measure the processes and outcomes of organizational initiatives is vital to understanding whether an organization has achieved its mandate. But determining the best method to measure the impact of volunteers and how to successfully report on those findings are distinct challenges in volunteer administration.

In this excerpt from a new, yet-to-be-published book called Measuring the Impact of Volunteers: A Balanced and Strategic Approach, authors Christine Burych, Alison Caird, Joanne Fine Schwebel, Michael Fliess, and Heather Hardie review the limits of statistical reporting and suggest ways to measure and communicate more useful data once it is collected. Using examples and outlining strategies, the authors present the basic touchstones of reporting results and note that it’s important to consider your audience, how and who should deliver the reports, and what to do if the news is not good. 

“Utilizing the strategies we have outlined for reporting on the engagement of volunteers can play an important role in articulating the impact and contribution of volunteers,” the authors write. “Reporting in a strategic way helps us to focus our priorities and to communicate to volunteers, key stakeholders, our executives, and boards that we are achieving our mandate to support the mission and vision of our organizations.”

This quarter’s Research to Practice contributes to the theme of Volunteer Centers. Reviewer Laurie Mook looks at a study of the direct and secondary impacts of the HandsOn Network and its affiliated member organizations: Measuring the Impact of HandsOn Network: An Evaluation of Direct and Secondary Impact from the Stakeholder Perspective.

Based in Atlanta, GA, HandsOn calls itself “the largest network of local volunteer centers around the world,” with 250 affiliates. Whereas previous studies relied on self-reports from affiliates to provide evidence of their impact, the researchers in this study used surveys and interviews to collect data directly from affiliates, community partners, and volunteers. Based on the perspectives of these groups, the researchers developed a conceptual framework that other Volunteer Centers and other intermediaries can use to measure the impact of their work.  

The Volunteer Program Assessment (VPA) provides consulting services to nonprofit volunteer programs across the United States and Canada. VPA consultants are graduate students and faculty from five universities who volunteer their time to work one-on-one with leaders of volunteers. VPA was established with the help of funding from The Humane Society of the United States (HSUS), which continues to provide financial support for volunteer programs – within the animal welfare sector and beyond – to receive VPA at no cost. Currently, VPA is seeking to expand its outreach efforts and diversify its nonprofit client base.

In this e-Volunteerism feature, four authors from the University of North Carolina at Charlotte provide a brief history of VPA, an overview of the process, and a summary of its success to date. Based on normative data from over 100 nonprofits, the authors describe three common areas for improvement among volunteer programs. They also provide additional information for those interested in participating in VPA.

In the last quarter century, nonprofits have increasingly been held accountable for the resources that have been entrusted to them. For some organizations, accountability mechanisms have been imposed left, right, and center, as funders and donors seek to monitor the use of the funds they provide. While a reasonable amount of accountability is beneficial all around, too much emphasis on this measure can be stifling and may indeed have exactly the opposite effect of what it intended to promote: efficient and effective use of resources.

While we generally can find information on the financial resources used by nonprofits, public disclosure of the amount and significance of volunteer contributions to those organizations is far less common. In this Research to Practice, Laurie Mook presents the findings of researchers in Australia who set out to determine just how organizations account for volunteer services. The study involved over 400 nonprofit organizations. The researchers systematically reviewed these organizations’ websites and annual reports for any disclosure of volunteer contributions, including their acknowledgement; how they supported mission and the wider community; human resources measures such as the number of volunteers and hours contributed; and policies guiding their involvement.

How well did the organizations do in discharging their accountability for their reliance on and use of volunteer contributions? Too much, too little or just right? As Mook reveals, the results are enlightening.

In 2009, Jeffrey L. Brudney and Lucas C.P.M. Meijs proposed a new way of thinking about volunteer resources: as a natural resource which must be managed sustainably or it will be exhausted. In their article, "It Ain’t Natural: Toward a New (Natural) Resource Conceptualization for Volunteer Management" (published in Nonprofit and Voluntary Sector Quarterly, Volume 38, Issue 4, and reviewed in e-Volunteerism by Steven Howlett), the authors argued that volunteer energy should be viewed as a human-made, renewable resource that people can grown, recycle and influence positively as well as negatively.

Many of us, especially readers of this journal, have long defied the skeptics and believed that volunteering has multiple benefits. Now there is growing research to prove that volunteering is good for society as a whole and that it needs to be nurtured and fostered.

In this Along the Web, we round up some of the emerging research that explores the impact of volunteering as a producer of social capital and well-being that goes beyond any individual volunteer, agency or organization. Benefits for individual volunteers – such as skill development or building personal confidence – are of course very well-known and documented. But as this article shows, there is a fascinating evidence base developing that also strengthens the case for encouraging more volunteers and better, more widespread volunteer programs. The timing is perfect. Just as policy makers and funders of volunteer programs increasingly want to know that their actions or investments will be effective over the longer term, the research linking volunteering to social capital and well-being is emerging.   

There has been quite a bit of research on volunteer satisfaction, but not so much on volunteer manager satisfaction. How satisfied are volunteer managers with their jobs?  Do their working conditions differ from those of managers of paid staff? What are the implications for nonprofit organizations and human resources departments? In this issue, reviewer Laurie Mook looks  at a study of 314 volunteer managers, conducted by a group of Canadian researchers who analyzed job-related and organizational factors such as co-worker respect, supervisor support, closeness to volunteers, and the nature of the work as an expression of personal values. Their final model, Mook explains, predicted job satisfaction for both short-term and long-term volunteer managers. 

We hear over and over again how volunteers are indispensable to many organizations. While we have previously covered articles on different methods used to estimate a value for volunteer contributions, a new study out of New Zealand looks at how volunteer value is communicated, both internally and externally. In this issue, reviewer Laurie Mook examines how a team of researchers conducted a qualitative study of local and national medium-sized health charities, and provides some thought-provoking insights into the barriers and drivers to communicating volunteer value for these organizations. An interesting aspect of the study, Mook explains, is that the researchers interviewed the executive director, fundraising manager and manager of volunteers from each organization, providing for a more holistic look at how volunteer value is communicated. Mook also provides her insights into the practical implications of the study, encouraging readers to reflect on the implications of making volunteer contributions visible while also considering the impact of keeping them invisible.

This issue’s Research to Practice provides great food for thought on organizational factors affecting volunteer management. For example, how do the goals of the organization, area of activity, or degree of bureaucracy impact the role that a volunteer management program can take in the strategic achievement of an organization’s mission? How can organizational settings be “assessed and aligned to the needs of volunteers, but also to those of the organization and society at large?”

In this article, reviewer Laurie Mook shares insights from Sibylle Studer and Georg von Schnurbein, two researchers from the Centre for Philanthropy Studies at the University of Basel in Switzerland. They reviewed and synthesized academic literature relevant to volunteer management from 1967 to 2011, looking for evidence to help explain how nonprofit organizational factors supported or restricted volunteer management. As the researchers found, there are many studies looking at ‘who volunteers’ and ‘why people volunteer,’ but studies from the organizational perspective are not as prevalent. Mook presents their findings and other conclusions about organizational factors and how they impact strategic volunteer management.

In this issue, author Laurie Mook looks at an interesting case study of the Philadelphia Ronald McDonald House as an example of how to measure the direct impact of volunteer service on the organization, clients and volunteers themselves. The research – conducted by Debbie Haski-Leventhal (Australia School of Business), Lesley Hustinx (Ghent University, Belgium), and Femida Handy (University of Pennsylvania) – is based on a series of surveys, informal interviews and observations through the researchers’ own involvement as volunteers in the organization.

As Mook explains, volunteer managers often view the monetary value placed upon volunteer service as one way to gauge the relative importance of volunteer resources as compared to other resources in delivering the services of a nonprofit. And, according to Mook, volunteer hours are also used as a proxy for impact.  But through this case study, Mook explores a few more tools. For example, the findings in this study reveal several areas of impact that can be measured. From the perspective of the client, three categories of impact emerge: tangible impact (providing services), attitudes (satisfaction and perceived altruism) and future behavior (willingness to volunteer). From the perspective of the volunteer, intrinsic and tangible benefits are identified. Overall, the researchers are able to communicate the distinctive and unique impact that the volunteers had for the organization.