BY Thomas Hubert
19. Sep 2013 - BOGOR, Indonesia (19 September, 2013) – Forest certification is considered a promising tool to improve forest management, but there needs to be a robust evaluation of a complex web of factors to establish its true value for various stakeholders, according to a new analysis. Certification is a market-based scheme under which companies or communities managing forests voluntarily submit to audits by independent inspectors. If they meet responsible forest management criteria, their timber products can then carry a recognized branding such as that of the leading certification scheme, the Forest Stewardship Council (FSC).
“However, the often-claimed environmental and social benefits of certification remain to be empirically evaluated,” scientists from the Center for International Forestry Research (CIFOR) have said in a paper proposing a framework for such an evaluation.
The scientists analyzed existing literature on forest certification. Although much information has already been collected, they found that none of the approaches they reviewed resembled what could be considered a well-designed, independent evaluation.
“Only 14 employed designs that appropriately considered the confounding effect of selection bias, and only one looked at forest management,” said University of Florida biologist Claudia Romero, one of the paper’s authors, referring to an earlier survey of 134 publications on the certification of various agricultural and forestry products.
This means that almost all the existing knowledge on certification is potentially skewed by a variety of factors influencing why and when forest managers choose to enter the scheme, and where scientists chose to conduct their research.

WEIGHING THE BENEFITS
Scientific evaluation of forest certification is important to inform supporters whether they are getting value for their money, the CIFOR paper states.
All stakeholders in tropical forests, from local communities to policymakers and international markets, would benefit from knowing whether certified forests yield greater socio-environmental, economic, and policy benefits than other options such as uncertified logging concessions or conservation areas.
Charitable foundations and governments that have been supporting certification also need to know whether the funding they provide is having the desired impacts.
“We need to determine whether being certified or not has any effect,” said Manuel Guariguata, a CIFOR principal scientist and co-author of the paper. “From a donor’s perspective, we’re talking many millions if not dozens of millions of dollars that have been invested in certification, and so far there’s no evidence that they have been invested efficiently.”
To assess the impacts of certification, the research suggests a dual-evaluation roadmap looking at social, economic, policy,  and environmental impacts expected from certification as well as the way the process is conducted.
The difficulty with impact evaluation is that forests are subject to many influences, and certification is only one of them. Measuring indicators is therefore not sufficient to determine whether success or failure in reaching certain goals is attributable to certification.
“Someone needs to say: OK, you have achieved those targets, but how did you do it?” Romero said.

FRAMEWORK FOR IMPACT
As an example, the paper explains that fewer work-related injuries among forestry workers – a desirable outcome of certification – may well result from improved safety regulations rather than from certification itself.
“We need a theory of how the goals of certification might be achieved. This is how we are going to separate the effect of certification from other factors such as government policy,” Romero said.
Researchers call such an intended path towards better forest management a “theory of change”.
“It’s a mental model that helps evaluator to see where specific activities are supposed to generate the desirable outcomes, and how certification interacts with other factors,” Guariguata said.
While the paper provides the building blocks needed to construct a theory of change anywhere in the world, it warns researchers that they will need to follow a participatory approach with local stakeholders before they can clearly lay out a model against which to check the impact of certification on the ground.
“You essentially bring people around the table and discuss what it is that makes a certified forest become more desirable in social, economic and environmental terms,” Guariguata said.
In addition to this evaluation of certification’s impacts against an expected theory of change, researchers will also need to look at the quality of the certification process itself.
“There are administrative aspects: the reports must be there, etc. We may be in a case where we don’t achieve responsible forest management not because the theory was wrong, but because it was poorly implemented,” Romero says.

AIMING FOR IMPARTIALITY
Another challenge in evaluating certification is to detect and avoid bias when choosing sample areas to study. Because certification is a voluntary process, certified and non-certified forests are not randomly allocated to each treatment  the paper warns. The choices behind a company’s decision to enter or leave a certification scheme result from a range of factors, which can also influence its outcomes.
“The main risk would be to go into a country and compare a certified forest with the nearest non-certified one. Even though they may be very close in ecological terms, they may be very different in other ways,” Guariguata said, adding that discrepancies can also arise from how long forests have been certified.
Finding the right comparison points for scientists studying the impact of certification – the so-called counterfactuals – seems to be one of the main challenges going forward.
Romero says that more research will help flesh out the evaluation model, including ongoing studies on variation in size, experience and other characteristics among forest management units as well as on the dynamics leading them to join or abandon certification and changes in contextual factors that affect decision-making regarding forest management.
“We’re in the early stages,” Guariguata said. “We have a framework that we think can be applicable globally, but it’s generic and needs to be adapted to country circumstances. It cannot be called a protocol or anything that is written in stone but we think is a good step forward.”