FAQs
INFFER™ Frequently Asked Questions (FAQs) are available for downloading (226KB pdf) or view them below.
Contents
Section 2: Goal, works, technical feasibility
Section 4: Delivery mechanisms, risks and costs
Section 5: Project information
Section 6: Project assessment form
General
1. What is an “asset-based” approach?
We refer to INFFER as an “asset-based approach” to assessing and prioritising environmental and natural resource projects because the process starts with the identification of assets. We find that this is an effective way to get people thinking clearly about what is required to achieve outcomes.
2. What is the difference between an asset-based and a threat-based project prioritisation approach?
In principle, very little. If both approaches include a comprehensive set of criteria, they should come up with more or less the same priorities. We prefer to start by identifying the assets because we think it helps shape the mind set of users, and it helps with communication.
In practice, the threat-based approached with which we are familiar are not sufficiently comprehensive, often omitting key factors such as technical feasibility and practice change/adoptability from the assessment.
3. How does INFFER add value?
Compared to how prioritisation decisions are often made by environmental managers, INFFER provides a number of important advantages.
i. It allows systematic comparison of investment options across all types of NRM issues, not just within any one category (such as biodiversity).
ii. It helps users ensure that their projects are defined in a way that is internally consistent. This means that projects can be compared validly, rather than favouring the project that has been exaggerated the most.
iii. It highlights the need to assess the technical feasibility of achieving a specific goal.
iv. It highlights the need to assess the adoptability or otherwise of works at the required scale.
v. It helps choose the most appropriate class of policy tools using the Public: Private Benefits Framework.
vi. INFFER includes all of the relevant criteria in one system. Most other systems we have looked at omit crucial elements, or allow decision makers to proceed without ensuring that they have all the essential information.
vii. It integrates all the information in a way that gives appropriate weight to different issues in considering priorities. Environmental managers often don’t have a way to do this, and may end up choosing priorities based on an assessment that gives too much weight to, say, environmental threat or asset value, and too little weight to technical feasibility, project cost, the adoptability of works, or the timing of benefits.
viii. It leads to selection of more realistic target outcomes and better targeted monitoring and evaluation.
ix. The full INFFER process scans all investment options available to an environmental manager and, through a series of steps, narrows the focus onto a smaller set of projects that are likely to deliver the most valuable outcomes. It results in a highly strategic and well-considered set of projects being identified.
4. How is INFFER related to the Australian Government’s “NRM Program Logic”?
The basic idea of NRM Program Logic is to help make sure that actions are likely to lead to outcomes. INFFER has exactly the same aim, although it provides a much more structured and guided process. The information requirements of INFFER are more comprehensive, and it has a stronger focus on value for money.
5. What skills are required to complete the Project Assessment Form and formulate a recommendation about the project?
The person completing the form needs to have good knowledge of the asset(s) that are the focus of the project, or at least have access to people who do. They need to know about existing management actions and projects relating to the asset. They need to be able to engage with other experts with specialist knowledge about the asset and its management, and to capture their advice and interpret it appropriately. They need to be able to handle a range of different types of information (community knowledge, expert scientific knowledge, economic information).
6. How long does it take to complete the Project Assessment Form for one project?
It depends on the level of knowledge of the person completing the form. If the person has very good knowledge across all aspects of the form, it may take only a day or so. More likely it will require time to collect and collate additional information, probably through consultation with experts and perhaps with local community members. Reading of past reports related to the asset will probably be required. We estimate that a relatively new user will take 3-5 days of work to complete this task where there is a lot of information to capture and report. Users become faster at the process with experience.
Some people have commented that this seems like a long time. In our view, it is not long considering that you are likely to spend millions of dollars of public money in the funded projects. It is an appropriate level of due diligence.
7. How much economic expertise is needed?
Not a great deal. The process is designed to inject a Benefit: Cost Analysis mindset into the process without requiring users to actually conduct a Benefit: Cost Analysis. It is a highly structured process, with a lot of detailed guidance provided. The person completing the form needs to be able to estimate the cost of the project actions, and may need to estimate the level of payments to landholders that would be required to achieve particular levels of adoption. Discussion with economists with local knowledge would be helpful in answering these questions, particularly where the project requires actions by commercially oriented landholders.
8. What if we don’t have the required information?
In the process of completing the Project Assessment Form it is common to find that there are substantial knowledge gaps. INFFER asks you to assess whether these knowledge gaps need to be filled before proceeding with the project, whether they should be filled as part of the project, or whether there is no need to fill them.
The information used to guide prioritisation does not have to be perfect. For example, the questions in Section 3 probably require a fairly high degree of subjectivity and judgment in most cases. We recommend two strategies to allow for this: (i) A feasibility assessment phase at the beginning of large projects; (ii) Adaptive management of projects, in which filling the key knowledge gaps is an active strategy, and is used to update decisions about project design and about whether the project should proceed.
9. How can I search for a particular piece of text across the whole Project Assessment Form?
By printing the complete PAF into a pdf file.
To do this you need to have a pdf printer driver installed. For example, see pdf995.com for a free one.
Once you have a pdf printer driver installed, use the button at the bottom of Section 5 to print the whole Project Assessment Form. Select the pdf printer driver as the print destination.
Then read the pdf file into your pdf viewer software and use its search facility to locate the text you are seeking.
What INFFER can and can’t do
31. What sorts of projects can INFFER assess?
INFFER is used to assess projects that aim to prevent or repair damage to natural assets, such as water resources, biodiversity and land. The projects must be linked to specific, identifiable natural assets.
It is not designed for general capacity-building projects for which results cannot be linked to specific assets, but if capacity building is what is needed to manage a particular asset, then the project can be assessed using INFFER.
INFFER provides a lot of information that is useful in considering investments in research (see FAQ36). However, it is not designed to allow a comparison between projects that mainly have the aim of collecting information about an asset (e.g. monitoring projects, and projects collecting basic data) and projects that aim to improve the management of the asset by influencing or directly undertaking works and on-ground actions.
32. Can INFFER be used to evaluate an existing project?
In principle, yes, provided that the existing project is internally consistent (i.e. the interventions specified will actually generate the required level and types of works and on-ground actions to achieve the project goal). In practice, we find that few existing projects are internally consistent. This means that users wishing to evaluate an existing project need to be willing to be flexible in how they design the project. Users who have not been so willing have sometimes become frustrated with INFFER, when the real problem was that their project was not internally consistent.
A risk when analysing an existing project, even if it is internally consistent, is that it will introduce constraints on what actions can be considered, or make assumptions about which actions are desirable, and these constraints or assumptions can reduce the project’s value for money.
To get as much as possible out of the process it is better to start with an asset, rather than an existing project. One should have as few preconceptions as possible about the project, and allow the project design to emerge in response to the questions that INFFER asks.
If you do wish to retrofit a project, you need to be prepared to adjust the project design to ensure that it is internally consistent. We would emphasise that this is a strength of INFFER, not a weakness. If the project is not internally consistent then we are misleading ourselves and others about how worthwhile it is. For example, it may show a high Benefit: Cost Ratio, but be based on works that are actually not adoptable on the assumed scale.
A crucial step when specifying an existing project is specifying the specific goal for the project. The goal needs to be realistic and highly achievable with the planned works. Then the goal needs to be accurately reflected in Q2.4 about the effectiveness of works at protecting asset value.
33. Can INFFER assess projects of different scales?
An example could be where one project deals with a single localised wetland, and another deals with a group of wetlands across the region.
Yes, the questions in the Project Assessment Form are scalable to any level. Comparisons between large scale and small scale projects are valid (in principle) because the Benefit: Cost Ratio expresses the benefits of the project per dollar of project cost.
However, the reality is that a very small scale project and a very large scale project have some intrinsic differences. The main one is that it is more difficult to give precise answers to the questions of the Project Assessment Form for a very large-scale project. There is likely to be heterogeneity within the area covered by a large-scale project, but in a number of cases INFFER asks you to provide a single value for the overall asset (e.g. technical feasibility or adoption). You have to give a response that best represents the area overall. This makes it a bit more difficult to complete a good assessment for a very large-scale project.
34. If there is an overriding threat that cannot be mitigated, does that rule out any project?
Not necessarily. It depends on the goal that is set for the asset. For example, the goal may be to preserve some biodiversity within a changed environment, or to support farmers to adapt their production systems to a degraded environment. For a project to be supported, the goal needs to be realistic given the realities of threats affecting the asset.
35. Can threatened species projects be assessed using INFFER?
INFFER can be used to assess projects for individual threatened species or for groups of species that have similar characteristics. Unlike the statutory processes that are in place for recognized threatened species, INFFER does not assume that all species can or must be protected. It helps investors to weigh up the value for money of investments in threatened species relative to other opportunities to invest in other environment or natural resource assets.
36. Does INFFER have a role in guiding investment in Research, Development and Extension?
Yes, INFFER can make a significant contribution to providing guidance about investment in Research, Development and Extension.
Research: In Sections 1, 2 and 3 of the Project Assessment Form (Step 3), we ask users to rate the quality of information provided, and capture key information gaps that may require additional research or data collection. There are always numerous information gaps around environmental issues, but INFFER identifies those gaps that are most crucial to decision making about management and prioritisation. In Step 4, users are asked to consider how information gaps should be handled.
Options include:
(i) Information is adequate to proceed to project implementation;
(ii) There are key information gaps that can be addressed as part of the project;
(iii) The project, if funded, should start with a feasibility assessment phase for say the first year. There should be an additional decision made after that phase about whether the full project should proceed, and any necessary modifications.
(iv) Information gaps are so pervasive and serious that they this asset needs a research/investigation project, rather than an implementation project.
Development: The Public: Private Benefits Framework (which is embedded in INFFER) provides guidance on the choice of delivery mechanisms to protect or enhance environmental assets. One of the options considered is “technology development”, meaning R&D to develop new sustainable practices that will be more adoptable by landholders, or more effective for environmental management. [See Pannell, D.J. (2009). Technology change as a policy response to promote changes in land management for environmental benefits, Agricultural Economics 40(1), 95-102.] Technology development is recommended as an appropriate investment response in cases where (a) the best available environmentally beneficial land management practices have negative private net benefits, and (b) there are good prospects to develop improved technologies.
Extension: The Public: Private Benefits Framework also identifies projects for which extension should be the front-line tool to pursue natural resource outcomes. Such projects should generate positive private net benefits for landholders, as well as positive public environmental benefits. Extension might also play a supporting role in other projects where the main mechanism for change is technology development or incentive payments.
How does it work?
61. How does INFFER handle climate change (or other changes)?
INFFER is forward-looking. The project assessment form asks what will happen to the asset in future (e.g. over 25 years) with or without the proposed project. In either case, the answer to that question can factor in the impacts of anticipated climate change. It is up to the environmental manager to do this, if they consider it appropriate, when they are responding to the questions in the form.
62. How is leasehold land dealt with?
If a project requires changes on leasehold land, it may be that the land managers would respond differently to a proposed project or intervention than they would if they owned the land freehold. If so, this would be reflected in answers to the questions in Section 3 of the Project Assessment Form relating to practice change (Q3.3 and Q3.4). It may also be that a project requires engagement with the land owner or the body responsible for the land (e.g. a government department). If so this would be captured in the questions relating to changes required of other organizations (Q2.2(c) and Q4.3(a)).
63. How can INFFER contribute to our strategic thinking about the whole catchment?
Some users are concerned that INFFER may tend to focus attention onto particular parts a catchment, rather than the whole catchment.
At least in Australia, the level of funding available in environmental and natural resource programs is not nearly sufficient to manage the issues comprehensively.
Also, there is a lot of spatial heterogeneity in the problems these programs address. There is variation from place-to-place in the values of assets under threat, the severity and urgency of threats, the technical feasibility of reducing those threats, and the adoptability of the required works. So the environmental benefits from investing in works will vary widely not just between catchments, but within catchments.
Thus, the usual reality is that the best use of program funds from an environmental perspective requires prioritisation of effort within a catchment. INFFER helps managers to identify what those within-catchment priorities should be. Of course there may be some situations where efforts to promote action over the whole catchment are warranted. For example, there may be a worthwhile practice that is both relevant and highly adoptable across the whole catchment.
INFFER does not require users to define their projects based on small parts of catchments. It is quite possible to define a project for the entire catchment, or an entire state for that matter. (See the question above about scales of projects.) If the project is indeed cost-effective at the whole-catchment scale, then this will be reflected in the INFFER analysis. However, it may be that a particular catchment-scale project is not cost-effective, while a more targeted project working on parts of the catchment would be. Users might therefore choose to define their project in a more targeted way. This is not a requirement or limitation of INFFER. It is just reflecting the reality that more targeted efforts often generate more valuable environmental outcomes per dollar spent than do less targeted efforts.
Given the limited funds available and the spatial heterogeneity outlined above, we generally start by asking users to focus on key assets within catchments, rather than the whole catchments. Again this is not a limitation or requirement of INFFER. It is just a recognition of where the best value for money outcomes are likely to be achieved.
In reporting back at the whole catchment scale, we would just aggregate the outcomes from activities that are occurring within the catchment. INFFER actually is a tool for thinking strategically about investments across the whole catchment. If well used, it provides an integrated assessment that helps managers decide which parts of the whole catchment are worthwhile investing in. This is more strategic than just assuming that investment should occur across the whole catchment.
Sometimes, when people talk about achieving outcomes at the catchment scale, they seem to put it on a pedestal, as if undertaking works on a smaller scale is inherently inferior. Actually, one shouldn’t be apologetic if activity is not occurring across the whole catchment. It may be a signal that you’ve got your thinking straight.
64. Does INFFER bias investments towards localised assets rather than dispersed assets?
No, it should not do that. The Project Assessment Form works for both localised and dispersed assets. What we tend to find is that the available information is stronger for some localised assets than for typical dispersed assets. This may mean that there is more confidence to proceed with investment in those localised assets in the short term. However, this is not a bias! It is an appropriate response to the availability of required information, which should be a factor whether one uses INFFER or some other process. INFFER encourages users to identify information gaps and recognises that for some assets the short-term priority is for projects to fill those gaps. Once that is done, the competitiveness of a dispersed asset just depends on how it stacks up in relation to the factors that INFFER considers: value, threat, technical feasibility, socio-economic feasibility, urgency, cost, etc.
Another possibility is that, in some cases, the technical feasibility or socio-economic feasibility of a project is lower for a dispersed asset than for some localised assets. Certainly not in all cases, but in some. In that case, the dispersed asset would be less competitive, just as any project with low feasibility should be. This is about good decision making, not bias.
For those dispersed assets that have modest value per hectare and extend over large areas (e.g. agricultural land), a project would need to be able to deal with threats at low cost per hectare to be able to compete. If it cannot do that, it will have a low Benefit: Cost Ratio value. Again, this is about good decision making, not bias.
Overall, INFFER is making the comparison between localised and dispersed assets more transparent and so, hopefully, less biased.
There may be implications for the choice of policy mechanisms to be employed in projects for dispersed assets. For example, in a case where agricultural land is your asset, the framework brings to the fore the option of technology change or industry development. If it is feasible to develop technologies or industries that deliver win-win outcomes for farmers and the environment, these have a good chance of being cost-effective. Use of traditional incentive payments on a broad scale is much less likely to be cost-effective in such cases (although each case has to be assessed on its merits, of course).
65. I have noticed that INFFER tends to encourage people to focus on particular components of larger assets. Isn’t there a risk that this may not lead to worthwhile outcomes?
What INFFER strives to do is to ensure that the project assessment is internally consistent: the stated goal would really be achieved by the specified on-ground changes; the project interventions would really deliver the on-ground changes; the budget would really be sufficient to deliver the project interventions. The reason for emphasising this is so that, when you compare projects, it is on an equal, realistic basis, rather than favouring those projects that have been exaggerated the most.
To achieve this internal consistency, if the budget for a particular project is too low to achieve what you’d really like, you have to do one of the following:
• Lower the goal so that it is achievable (e.g. by seeking a lesser change in environmental condition, or focusing on a sub-component of the overall asset);
• Increase the budget;
• Perhaps look at options that might deliver outcomes at low cost per ha (possibly technology change or industry development).
• Maintaining an ambitious goal with an inadequate budget is not a viable alternative.
All INFFER is doing here is making all this transparent and getting it onto the table so that we can make sensible, well informed decisions based on what proposed projects are actually likely to achieve, rather than what we’d like to imagine they could achieve if we had bigger budgets.
If the outcomes of a project are not worthwhile, this will be reflected in the Benefit: Cost Ratio (BCR). A project that achieves trivial changes to an unimportant asset will have a very low BCR.
In some situations, focusing the effort onto sub-components of the landscape is actually the sensible thing to do. They could be the components that are most valuable/most threatened/most feasible to protect. If so, the process of breaking the landscape down into components will lead to the most meaningful outcomes, not less meaningful ones.
66. How would one assess a project that provides a small part of a larger long-term project, or one of a series of projects?
An example would be a project to improve riparian vegetation in one small part of a river, as part of a plan to do it for the whole river. Or a project to protect and enhance an area of native vegetation in a region, with the aim of doing it for a much larger area over coming decades. A potential problem with small projects that are really part of a larger project is that the degree of change to the overall asset that they contribute may be very small and hard to estimate. It may require the full-scale, long-term project to achieve the goal that is really sought for the asset.
There are two possible ways to handle this type of project in INFFER.
(a) Evaluate the full-scale, long-term project. Specify the goal, the technical feasibility assumptions, and the costs to match this much larger project. This then gives an indication of whether it is worth proceeding with any constituent project. The advantage of this approach is that it may be easier to get realistic information about technical feasibility than it would be for smaller individual constituent projects. A potential disadvantage is that there may be constituent sub-projects that are better (more cost-effective) than the overall project on average, and this would be obscured by the approach. In entering costs for this project, the current project costs would relate to the small-scale project that is currently under consideration, and the ongoing maintenance costs would be the average annual investment in the large project over time.
(b) Evaluate the individual smaller-scale project on its own. To capture the fact that its overall contribution to protection of the larger asset will be small, you would specify a small value for W in the technical feasibility section (e.g. it might be 0.01). This would reflect the proportion of the full-scale project that this smaller project represents, times the expected impact of the full-scale project. For example, if the full-scale project would be expected to enhance the value of the asset by 50%, and the small project is about 2% of the scale of the large project, then the value of W would be 0.5 x 0.02 = 0.01. Using this approach the ongoing maintenance costs would be those only for the individual sub-project, not the full-scale, long-term project.
67. How does INFFER deal with projects where there is a long time frame until the benefits are generated?
INFFER is designed to be able to assess a proposed project that might have a duration of say one to five years, but will generate benefits in subsequent years. For example, the project may help to avert degradation of an environmental asset that is not expected to happen for some decades, or it may take decades for current actions to repair an already-degraded asset.
In the INFFER Project Assessment Form, we collect information about the likely time lag until benefits occur. Then, the Benefit: Cost Ratio includes a discount factor so that projects with more immediate benefits get a higher weight than projects with distant benefits (other things being equal).
68. How does INFFER deal with projects that require investment over a long time frame?
In the Project Assessment Form, we ask for an estimate of ongoing annual maintenance costs, which are factored into the assessment of cost effectiveness. Maintenance costs are converted to a “present value” using standard discounting methods.
69. How does INFFER compare a one-year project with a five-year project?
On the basis of overall value for money. We ask, which of the two projects has the greatest environmental benefits per dollar spent? In both cases, we ask for information about the need for ongoing expenditure (beyond the project) and factor that in.
70. How does INFFER compare large and small projects?
On the basis of overall value for money. We ask, which of the two projects has the greatest environmental benefits per dollar spent?
71. How does INFFER handle uncertainty?
In Sections 1, 2 and 3 of the Project Assessment Form, we ask users to rate the quality of information provided, and capture key information gaps that may require additional research or data collection. The project assessment report provides estimates of probability of project failure due to several factors (practice change, lack of cooperation from other essential institutions, socio-political factors constraints, technical feasibility, and long-term funding). These factors or included within the calculation of the Benefit: Cost Ratio.
We recommend that large projects include a phase of detailed feasibility assessment (both technical and socio-economic feasibility) as their first phase.
72. What if there is too little information about an asset to make a good investment decision about it?
If the available information is too weak, the project should not be funded.
This should be identified in the process of completing the Project Assessment Form. For some such projects, it is likely to be worth investing in new research of data collection to fill key knowledge gaps.
73. How does INFFER relate to Benefit: Cost Analysis and Multi-Criteria Analysis?
INFFER aims to introduce a Benefit: Cost Analysis (BCA) mind-set into environmental decision making, supporting environmental managers to ask the right questions and bring them together in the right way.
INFFER involves the user providing a score for the significance or value of each asset, encompassing both financial (market) values and intangible (non-market) values. So that the results of different assessments can be compared, we standardise the score to $20 million per point. So, for example, an asset with a value score of 10 is considered to be worth $200 million. If the score provided is consistent with that, then the resulting Benefit: Cost Ratio is fully consistent with that produced by a full Benefit:Cost Analysis.
On the other hand, users could choose to use the scoring system in a relative way, indicating the relative value of the different assets they are concerned with, rather than calibrating the scores to $20 million per point. In that case, the resulting Benefit: Cost Ratio indicates the relative merits of investing in the different environmental assets being assessed, but not the absolute merits (i.e. it would not necessarily be the case than any investment with a Benefit: Cost Ratio greater than one would be worth investing in). This would also mean that the results would not be comparable with results from analyses conducted by other people who had not used the same scoring system. Only those analyses conducted with the same scoring system would be comparable.
An advantage of INFFER is that it is specifically designed to apply to investments in the environment and natural resources, whereas BCA is a more general technique. This means that INFFER can provide a lot more detailed and structured guidance, which is essential for non-economists to be able to use it.
INFFER also has some things in common with Multi-Criteria Analysis (MCA). Like MCA, INFFER involves collection of various types of information, and bringing them together in a structured and transparent way. Key differences to MCA include that INFFER specifies in advance which information is relevant to the decision, provides detailed guidance about the collection of that information, and integrates the information in a specific way that is consistent with BCA, which often is not achieved in an MCA. It is quite possible to choose to omit crucial information in an MCA, or to include it in an inappropriate way, but the more structured and guided approach provided by INFFER does not allow this.
74. Is INFFER consistent with the concept of environmental triage?
Yes. The idea with environmental triage is to split up the potential investments into three groups: (a) those where the assets are in such good condition and facing so little threat that they don’t really need investment, (b) those where investment can make a worthwhile difference, and (c) those where the assets are so damaged or protecting them is so intractable that it is not worth investing. This is highly consistent with what INFFER does. It effectively helps you allocate any particular project to one of these three groups. Further, it helps you judge which of the projects in group (b) should be the highest priority, and what form the intervention should take.
75. How do funder priorities influence the assessment of a project using INFFER?
Assets that are higher priorities for investment by funders would have higher scores for asset significance (Q1.2), other things being equal.
They may also have lower socio-political risks, in the sense that the funding organisation may facilitate cooperation from other relevant organisations (Q4.4).
Assets that are higher funder priorities may have higher prospects of long-term funding being available for the asset (Q4.6(b) and Q4.6(d)).
76. If there is a project that has good prospects of delivering benefits but has a long time lag until benefits, wouldn’t the use of discounting unfairly disadvantage this project?
Discounting means that if you have two projects that would deliver exactly the same benefits but one would do so more quickly than the other, the one that would do so more quickly is favoured. This is about choosing the most valuable outcomes. It is a standard approach in economics and would be considered essential by, for example, Departments of Treasury.
The further into the future that the benefits of a project occur, the lower the BCR score, so yes, projects which only deliver benefits in the distant future would find it more difficult to compete with projects that deliver more quickly. This is appropriate. Importantly, the time lag is only one of a large number of factors influencing the BCR. It is still quite possible for a project with a short lag to benefits to have a low BCR and for a project with a long lag to have a high BCR, depending on asset value, impact of works, adoption, risks, etc.
77. If a project would generate benefits in the form of carbon credits, how would these be captured in the INFFER analysis?
They would be recorded as spin-off benefits in Q 2.6.
78. How does INFFER handle a situation where the project needs to be supported by changes in another natural resource management organisation?
Three questions in INFFER are relevant to this issue:
Q2.2(c) asks what works and on-ground actions would need to be undertaken by other organisations, and which other organisations would be responsible.
Q4.3(a) asks which policy mechanisms would need to be implemented by other organisations.
Q4.3(b) asks you to specify the delivery mechanisms that will be used to encourage the other organisation to adopt the required actions and policy mechanisms, as specified in Q2.2(c) and Q4.3(a).
Q4.4 requires you to quantify the risk of non-cooperation by the other organisations.
These questions build on each other in a logical order: What has to happen? How will you make it happen? What is the risk of it not happening?
79. I have previously used “risk and consequence” to assess the seriousness of project risks. How does the INFFER process of scoring “threats” relate to that approach?
The “risk and consequence” approach breaks down the assessment of project risks into two components: the probability that an adverse outcome will occur (“risk”, scored as almost certain, very likely, likely, …, rare) and the seriousness of that outcome if it does occur (“consequence”, scored as catastrophic, major, moderate, minor, insignificant).
While this can be a useful vehicle for structuring thinking about the riskiness of a project in certain circumstances, it does not suit the needs of INFFER. One reason for this is that it characterises the consequence as a binary variable: a certain level of damage, versus no damage at all. It is assumed that a certain probability can be assigned to the specified level of damage, and it implicitly assumes that the rest of the probability is assigned to no damage.
In reality, a particular threatening process may have a range of possible damage consequences, each with a subjective probability level. Zero damage is probably very unlikely in most natural resource projects, even if a major project is implemented. In INFFER, our assessment of threat is effectively a subjective weighted average or expected value, weighing up different possible levels of damage and their probabilities. This assessment is conducted simply, by scoring the most likely level of damage as: Very high (76-100% loss of asset value), High (51-75%), Medium (26-50%), or Low (0-25% loss of asset value).
This treatment of threats encompasses both risks and consequences, and is both more realistic and more flexible than the usual “risk and consequence” approach.
Another requirement of INFFER is to be able to quantitatively assess the reduction in damage as a result of the project. This is conceptually straightforward using our approach: it is the reduction in the scored damage level (most likely, or weighted average) due to the project. This assessment would be more problematic in a “risk and consequence” approach. It would probably involve either a reduction in the probability of a fixed adverse outcome, or a reduction in the adverse outcome with a fixed probability. Either of these approaches would be less realistic, and both would implicitly include zero damage as an outcome with a significant probability, whether or not the project was implemented.
80. In the Project Assessment Form (step 3) the categories for some of the parameters appear to assume non-linear relationships. What’s the reasoning behind this?
For three of the questions, the response categories offered are not evenly spaced: 2.5 (Risk of technical failure), 3.3(c) (Private adoption of works), and 3.4 (Adoption of adverse practices). There are different reasons for this for each question. They are explained after each question in the Instruction Manual for the Project Assessment Form.
81. By allowing for the risk of technical failure and emphasising knowledge gaps, is there a risk that INFFER will steer environmental managers away from innovative projects?
The question implies that innovative projects are likely to have higher risks of technical failure. On the other hand, one would hope that they also have benefits in other areas (e.g. higher likely adoption by landholders, or greater expected impact). The overall assessment of a project involves consideration of all the relevant factors. Thus, even if an innovative project is judged to have a relatively high risk of technical failure, this can be offset by other benefits. If other offsetting benefits are not expected, then of course the project would not be assessed favourably, but obviously that is appropriate. It would not be sensible to fund a project with higher risks unless there were also grounds to expect that it could have higher benefits. This is basic risk management.
Looking at the risk issue in a different way, imagine that you must choose one out of two projects that are identical in every respect except that one has a higher risk of technical failure than the other. Clearly you would choose the less risky project. You would not choose the more risky project just because it was more innovative because, on the balance of probabilities, it would not deliver such valuable outcomes. It would need to have other offsetting benefits in order to justify funding. Thus, a risky project is not ruled out by any means, but it needs to justify its resources, because funding it will mean that a less risky project is not funded.
The question also implies that innovative projects are likely to have greater knowledge gaps. INFFER does not rule out funding projects with knowledge gaps, but as with risk, they need to be weighed up against other factors. Step 4 of the process asks decision makers to consider knowledge gaps and decide how they should be handled. The suggested options are:
(i) Information is adequate to proceed to project implementation;
(ii) There are key information gaps that can be addressed as part of the project;
(iii) The project, if funded, should start with a feasibility assessment phase for say the first year. There should be an additional decision made after that phase about whether the full project should proceed, and any necessary modifications.
(iv) Information gaps are so pervasive and serious that they this asset needs a research/investigation project, rather than an implementation project.
For categories (ii) and (iii), the knowledge gaps are addressed within the project. If an innovative project is judged to fall in category (iv) then it would not be responsible to proceed with an implementation project without undertaking further research first.
We have also been asked a similar question about projects that are risky in a socio-political sense. Such a project would be scored down in Q4.4 (Socio-political risks). An environmental management body suggested that it may wish to “make a stand” by backing an important project even though it faced high socio-political risks. If the socio-political risks only affected this project, we would hold that they need to be fully factored into the project assessment and would need to be offset by other benefits if the project is to be worth supporting. That is, the project would need to have a competitive Benefit: Cost Ratio.
There may be a case where changing perceptions or attitudes in other organisations or at the political level would have benefits for other projects or activities as well. In that case, we would suggest that these other benefits be documented in Q2.6 as a positive spin-off from the project, and then weighed up along with all other relevant information about the project.
82. In the Benefit: Cost Ratio, some of the values included in the Project Assessment Form seem to have particular numbers, like 0.92 or 0.87. Why these numbers? Where have they come from?
Numbers like that are simply the mid point in a range that has been selected by the user.
The Benefit: Cost Ratio is calculated based on responses from the user. To keep things simple, in the Project Assessment Form we tend to provide several categories of response to choose from. For example, in Q2.5 (Risk of technical failure), there are five response options offered:
0-5% Very low risk of project failure due to poor technical feasibility. (F = 0.97)
6-10% (F = 0.92)
11-15% (F = 0.87)
16-20% (F = 0.82)
21-100% High risk of long-term project failure due to poor technical feasibility. (F = 0.4)
The user might choose 11-15% (i.e. there is an 11 to 15% risk of technical failure). The mid-point, 13%, has been subtracted from 1, giving an 87% probability of technical success as the value for F in the Benefit: Cost Ratio.
The user can either use the suggested numbers in the Project Assessment Form or suggest their own. For example, if you estimate that the risk of technical failure is 40% then the value of F would be 0.6, or if you think there is no risk of technical failure, F would be 1.
Section 1: The Asset
101. What is an asset?
An asset is the thing we hope to protect, improve or manage better through a proposed project. It could be large or small, degraded or pristine, localised or dispersed. An asset could be a single localized thing (for example, a particular wetland or river), or it could be a collection of smaller assets, such as remnant vegetation on farms in a region, or agricultural land in a region. If the asset is defined to be very large (e.g., the Murray River, Great Barrier Reef), it is unlikely that the available funding will sufficient to manage it, unless the goal specified for the asset is very modest.
102. What is the difference between an asset and a project?
When we describe INFFER as “an asset-based framework”, we mean that projects are built around assets. Each project assessed is based on a particular asset or set of assets. The output from the INFFER process for an asset is an assessment of a particular project related to the asset(s), rather than an assessment of the importance of the assets per se. The significance of the asset(s) is identified as one part of the process of assessing the project.
This focus on assets does not mean we neglect other aspects of the issue. For example, the “eco-system services” that are enhanced by the project are estimated as part of the process. The focus on assets simply provides an effective way to structure the process.
103. Can a project address more than one asset?
Yes. A project might be based on actions that will protect or enhance a group of assets that are located in a cluster, or a set of similar assets that are in different places but require similar management.
104. Can INFFER assess projects that are not focused on specific assets (e.g. capacity building, information collection, monitoring)?
The current INFFER process is strongly asset-based. It can readily assess projects in which capacity building is used as a strategy to protect or enhance specific assets, but it is not suitable for projects that focus on untargeted capacity building.
INFFER helps to highlight which information is needed to make good decisions about an asset and it helps to identify key information gaps. It is quite possible to include information collection as an element with a project that is focused on a particular asset. However, INFFER is not suitable for assessing an information or research project that is not focused on a particular asset.
INFFER helps to define projects in a way that greatly facilitates monitoring and evaluation focused on those projects. As with information, monitoring and evaluation activities can be included within a project that is focused on a particular asset. However, INFFER would not be suitable to evaluate a project that consists solely of monitoring and evaluation activities.
105. Is it appropriate to treat the community as an “asset” in INFFER?
The community is central to the INFFER process, but it is not appropriate to treat it as an asset in the same way as we define a wetland or river as an asset. We assume that the purpose of the public funding is to improve environmental and natural resource outcomes, and while the community plays a number of essential roles in that (see below), we are not investing in the community for its own sake. (There are other government programs that do that.) Rather the program would support the community to pursue environmental and natural resource outcomes that are important to the community.
The process can capture positive spin-off benefits from the project for community capacity if these are significant.
See also several FAQs from Part 3 for more information about the community and its role in INFFER.
106. How do you value intangible environmental benefits in INFFER?
Environmental managers are asked to assess the overall significance of each asset. When they do this, they can consider whatever mix of tangible and intangible (e.g. aesthetic, educational) factors that they consider appropriate. Each asset is given a score which indicates its overall value relative to other assets. After that, other factors such as degree of threat, feasibility of protection, adoptability of works, and cost are factored in. These are applied to intangible benefits in the same way as they are to tangible benefits.
107. The process of providing a value score for each asset is subjective. Is there a risk that this will introduce errors into the assessment?
Yes, the process of selecting a value score for the asset is subjective. However, it just makes explicit what we already do implicitly, so that it can be scrutinised and the decision process can be transparent.
The inclusion of the asset value score in this way has significant advantages. In particular, it facilitates the broadest possible range of comparisons between projects of different types (short-term vs long-term; large vs small budget; tangible v intangible benefits; biodiversity vs wetlands vs agricultural land)
In judging the merits of the approach used in INFFER, it needs to be compared to the alternatives. Two of those alternatives are to assume that all assets are of equivalent value, or to invest in a more detailed valuation process.
• Ignoring the asset valuation problem and implicitly assuming that all assets have the same value is not a viable strategy. For example, we would not want to assume that the Great Barrier Reef has the same intrinsic value to the Australian community as a small lake in the Western Australian wheatbelt.
• If time and resources permitted, it would be possible to undertake a more sophisticated and elaborate valuation process. For example, we could use what economists call “non-market valuation” methods, and if we did so, the results could slot directly into INFFER. It’s a question of balance and the importance of accuracy of particular numbers.
In practice, looking at the dozens of INFFER analyses now completed, we find that the asset value is rarely the decisive factor driving the overall value for money of projects. Its influence is no more important than factors such as technical feasibility and the adoptability of works.
Whatever tool or method is used (not just INFFER), the actual decision process should include a strong element of quality assurance and review of assumptions by competent experts. If the value score (or any other parameter) has been exaggerated, this can be picked up.
The decision making body doesn’t have to accept the asset values provided. It would be possible to substitute alternative preferred values and examine the consequences for project value for money.
108. Why do we need to specify a benchmark asset condition?
The idea of the benchmark asset condition is to help you to be consistent when thinking about the value of the asset (Q1.2) and the effectiveness of works (Q2.4). In Q1.2(b) we ask about the overall significance of the asset, but the answer to that depends on the condition of the asset. The question is, what condition for the asset should you be thinking of when you specify its significance? The answer is: the benchmark condition.
When answering the questions about the effectiveness of works in Q2.4, the answers are a proportion of the asset condition. But again, which condition? The benchmark condition.
109. Does it matter what condition I specify as the benchmark condition?
In principle, you can choose any asset condition as the benchmark condition, but we have some recommendations.
Firstly, it may help to minimise potential confusion if the benchmark condition is at least as good as the condition resulting from the project. If you are assessing more than one project for this asset and they have different goals, it is important to use the same benchmark condition for all projects.
Overall, our recommendation is to use the condition that is expected to result from successful achievement of the goals of the project. If you are assessing several different projects with different goals for the same asset, we suggest using the condition that would result from successful achievement of the most ambitious set of goals.
Section 2: Goal, works, technical feasibility
200. Setting “SMART” goals for environmental and natural-resource projects is challenging. How should this task be tackled?
The establishment of a “SMART” goal is a crucial part of developing and evaluating a project. We have prepared a document with suggestions and advice on how to approach it. See http://www.inffer.org/using-inffer/setting-smart-goals/
201. Each asset has a different goal, with different levels of ambitiousness. How is the degree of ambitiousness factored into the assessment of the project through the INFFER process?
A more ambitious goal means you get more benefits from the project (i.e. a bigger number for T), but also it becomes harder to achieve your adoption target (a lower A), perhaps entails more risks (possibly lower P), and definitely higher costs (higher C and M). The system is set up to be able to compare projects with different degrees of ambitiousness, because ambitiousness is captured in those parameters, which form part of the Benefit: Cost Ratio.
For the same reason, it is possible to use INFFER to compare the net benefits of managing a particular asset with more or less intensive management (and correspondingly more or less ambitious goals). You can prepare several Project Assessment Forms for the same asset, with different types and intensities of management in each, and compare their Benefit: Cost Ratio values.
This only works because, in completing the Project Assessment Form, we require your project to be internally consistent. When you change the goal, internal consistency requires that you vary those other parameters as well. Without the emphasis on internal consistency, the comparison of projects with more or less ambitious goals would be meaningless.
202. How ambitious should the goal for the project be? Will more ambitious or less ambitious projects tend to be more cost-effective?
It is not obvious in advance what the best level of ambitiousness is. For some assets, the project that generates the most benefits per dollar spent might be very conservative, while for others it might be very ambitious. This largely depends on how tractable/intractable the problem is (tractability encompassing both technical and socio-economic issues). More intractable problems would lead you to having more conservative goals. Beyond a certain level of intractability, even a project with a conservative goal would not be competitive with projects for other more tractable assets.
This table illustrates how you could rank projects with different levels of ambitiousness, including different projects for the same asset, and that the more ambitious projects may or may not have a higher Benefit: Cost Ratio. In these examples, management of Asset A is quite tractable, and for Assets B and C it is relatively intractable.
| Project | Benefit: Cost Ratio |
|---|---|
| Asset A ambitious | 20 |
| Asset A conservative | 15 |
| Asset B conservative | 6 |
| Asset C conservative | 5 |
| Asset B ambitious | 3 |
| Asset C ambitious | 1 |
If you could only afford two of those projects, and your main criterion was value for money, you would choose “Asset A ambitious” and “Asset B conservative” (assuming that “A ambitious” and “A conservative” are mutually exclusive).
Section 3: Practice change
301. How is the community involved in the process?
The community plays several crucial roles in the INFFER process:
(a) The community values different environmental assets differently. We capture community valuation of various assets in community workshops (or draw in information from past workshops or surveys).
(b) Particular members of the community provide important local knowledge about assets, such as the degree of current degradation, and the impacts of current management actions.
(c) For some assets, it is primarily up to members of the public to implement the works that would be required to manage the asset. We ask about likely landholder responses to the project in the Project Assessment Form, and this information plays a key role in both the Public: Private Benefits Framework (for selecting appropriate policy tools) and the Benefit: Cost Ratio.
302. Which social factors are considered in the assessment?
The questions in Section 3 are about the behaviour of the landholders whose behaviour you hope to influence, the behaviour of people in other relevant organisations from whom you need cooperation, and the risk of community opposition to the project, or lack of sufficient community support. It obviously encompasses a very broad range of social issues. If any other social issues are relevant to the feasibility of the project, you can bring them in here: Question 4.4 asks for “Socio-political risks”, including non-cooperation by other organisations, and social, administrative or political constraints on the acceptability of a project.
In answering Questions 3.1 to 3.4 you need to weigh up social issues such as: past adoption of the desired practices or similar practices in the area; the likely goals of the relevant people (e.g. mainly commercial or mainly lifestyle); their capacity to change (e.g. the resources they have available); the strength of local networks and community groups that might encourage or support the types of changes you’re hoping for; the sorts of peer pressures landholders are likely to come under from their peers (could help or hinder); and the level of trust and cooperation that can be expected in that community. The section should be completed in consultation with local experts who have some feel for these issues and can bring them into consideration when proposing answers to the questions.
Away from the feasibility section, social issues are also reflected in the determination of the asset value: why, and to what extent, is this asset important to the community. And then social issues may be relevant when you consider the choice of policy interventions that you will implement. These need to be socially acceptable and effective.
303. How is community knowledge included in the assessment?
The recommended process for applying INFFER comprehensively includes workshops with community members to elicit the values they assign to particular assets, the overall significance of different assets, and knowledge of threats
304. How is community capacity included in the assessment?
From the point of view of achieving outcomes, the key benefit of community capacity is that it may enhance the capacity and willingness of land/water managers to adopted changed practices. If you judge that this would apply in your project, it would positively influence the score provided for practice change in Q3.3.
A more positive rating for practice change increases the Benefit: Cost Ratio for the project, and moves the project to the right in the Public: Private Benefits Framework.
In some cases, community capacity may also reduce the risks of project failure due to other socio-political risks (Q4.4). On the other hand, a highly motivated and networked community may be more effective at opposing a particular project, and so increase the risks of project failure. This needs to be judged on a project-by-project basis.
305. How does INFFER take into account the role of local groups (e.g. catchment groups, Landcare groups) in enhancing knowledge and capacity, and maintaining works over time?
Increased knowledge may lead to increased adoption of changed practices, which would be captured in Q3.3.
Increased capacity of land/water managers may also lead to increased adoption of changed practices, which would be captured in Q3.3 (see FAQ 304).
Enhanced maintenance of works over time would be a positive for the project, and may be reflected in responses to Q4.6(b) and Q4.6(d).
306. How should you handle a case where you believe that a particular practice would be in the interests of land/water managers, but you expect them to continue to resist uptake of the practice?
An example of this could be a project which requires reductions in fertilizer rates to reduce nutrient off-site pollution. We intend that responses to the questions about practice change should be realistic and should reflect actual landholder behaviour, not the behaviour that you would like them to follow. In general, people close to environmental and natural resource policy in Australia tend to be too optimistic about how much behaviour change will be prompted in the long term by any given policy intervention. If farmers are not currently adopting practices that you think would be in their best interest to adopt, there are several possible explanations:
(a) It is actually not in their interest to adopt (i.e. you are mistaken to think that it is). Perhaps there are features of the practice that you haven’t appreciated that are important to the farmers. Perhaps the practice is slightly in their interest, but not enough to be compelling, so that it is not worth the cost, the time and the risk of adopting. Or perhaps you have an inaccurate perspective on what constitutes their interest. e.g. they may have risk attitudes that you are not considering. It is also important to be aware that in some cases a practice can be quite attractive at a small scale but very unattractive at a large scale.
(b) It is actually in their best interest, and they are on the pathway to adopting it, but the diffusion/adoption process is slow. Except for simple technologies such as a new wheat variety, the adoption process is likely to take at least a decade and potentially quite a lot longer.
(c) It is actually in their best interest, but there are factors inhibiting them learning about the technology. It may be difficult to trial, or trials may be difficult to learn from due to poor observability or complexity.
A type of project that tends to fit this category is one that requires reduced rates of fertilizer or chemical inputs. This is one that frustrates scientists all over the world. See a brief article here which argues that explanation (a) is often the case and discusses reasons why non-adoption of reduced rates is often completely understandable and reasonable from the farmer’s perspective.
For the purposes of completing the INFFER Project Assessment Form, if a practice has been promoted to farmers but still not adopted, you should assume that net benefits from practice change are neutral, at best, and more probably slightly negative.
Section 4: Delivery mechanisms, risks and costs
401. How are public and private benefits defined?
‘Private net benefits’ refer to benefits minus costs accruing to the private land/water manager as a result of the proposed changes in land management. They exclude payments which are part of the policy intervention, so that we can compare landholder behaviour with and without the intervention. In principle, private benefits are broader than financial benefits, and include the broad range of factors that influence the relative advantage of the new land use options (as perceived by the landholders) such as riskiness, complexity, social considerations, personal attitude to the environment, and farming-systems impacts of the land-use practice.
‘Public net benefits’ means benefits minus costs accruing to everyone other than the private land/water manager. They exclude any costs borne by the environmental manager in the process of intervening to encourage the change in land management. This will allow us to compare the benefits of an intervention with its costs.
402. How does one estimate public net benefits?
The Public: Private Benefits Framework requires an assessment of the public net benefits of a project, not the public net benefits of the environmental asset in question. For an environmental project, estimating public net benefits has at least three components: the value or significant of the environmental asset, the level of threat or damage to that asset, and the technical feasibility of reducing that threat or damage. Ignore the adoptability of works and the cost of the works (at this stage). If you answer all the questions in the Project Assessment Form, INFFER will automatically calculate the public net benefits and use the results to advise of the choice of delivery mechanism and the project’s value for money.
403. How does one estimate private net benefits?
Private net benefits is another way of saying, the “adoptability” of the works that would be needed to achieve your goal for the asset. The adoptability of works needs to be considered carefully. Here we are concerned with scoring the attractiveness of the required works and on-ground actions to the target group of potential adopters, recognising that there will be variation due to individual perceptions and circumstances. Strategies that could assist you to assess adoptability include:
• reviewing the existing literature about the farm-level economics of proposed land-management changes in the relevant areas;
• considering the available land-management options in the light of research evidence about the adoptability of different practices (Pannell et al., 2006);
• observing actual adoption behaviour of land managers for the relevant practices, with and without extension and/or incentive payments; and
• talking to farmers and local experts.
An additional option could be to conduct a conservation tender, to see how large a payment landholders require in order to be willing to adopt a certain practice at a certain scale.
Section 5: Project information
Nil
Section 6: Project assessment report
601. The potential exists for investment decisions to be made by ranking on the BCR score alone. Isn’t that a problem?
Any tool or system can be abused. Preventing this relies on the institutional arrangements and the individuals involved in decision making. We emphasise that the BCR is just an input to decision making. You would certainly want it to have a strong influence, but not to dominate other important considerations. To encourage decision makers to use the BCR in a balanced way, it is provided as one part of a Project Assessment Report. Also included in that report is information about time lags, risks, spin-offs, information quality and key knowledge gaps.
We note that if other factors influence decision makers to prioritise a project with a lower BCR, it is valuable to be able to see what is being given up. Thus, the use of the BCR improves accountability and transparency.
602. Why are the variables in the benefits part of the ratio multiplied rather than weighted and added?
The BCR has been carefully designed to be logical and conceptually sound. In summary form, it is calculated as follows:
Starting with V as the asset value, W is multiplied because of the way it is defined as the proportional increase in value (so that overall benefits are proportional to W); A is multiplied because it is defined as a proportion of the required adoption and it is assumed that benefits are proportional to W, B, P and G are multiplied because they are probabilities; and DFB(L) is multiplied because it is a proportional discount factor.
This BCR formula avoids a serious problem that arises if variables are weighted and added up – a common approach in Multi-Criteria Analysis and in many ad hoc scoring systems. The big problem with additive systems is that a low score in one critical variable can be compensated for by high scores in other variables.
For example, if a project is not technically feasible, there is no way it should be supported, but using an additive system it may be. Indeed, a project that would achieve no outcomes whatsoever could be scored relatively highly within an additive system, even if the variables are scored accurately. This cannot happen in the BCR – if the impact of works (W) is zero, then the score for the whole equation is zero (as it should be).
In general, the problem with additive systems is that, they often do not accurately reflect the benefits of the project, even if the numbers provided are completely accurate. Indeed, in principle, they cannot be consistently accurate if used for the sort of problem that INFFER addresses. The example above with zero impact of works provides one illustration of this. As another example, suppose there are two projects that are identical except that in one case, the works make twice as much positive difference to asset condition as in the other. Logically, the benefits index for the first one should be twice as high as for the second. This would be accurately reflected in the BCR, because the value for W would be twice as high, and therefore the BCR would be twice as big. However, in an additive system, doubling the value of any variable cannot double the value of the index – the impact will always be less than the proportional change in the variable.
The following numerical example illustrates what can go wrong with a weighted additive system. Suppose the following values are assigned to the variables for four assets/projects.
| Asset/ project | Value (V) | Impact of works (W) | Technical feasibility (F) | Adoption (A) | Adverse adoption (B) | Socio-political risks (P) | Long-term funding (G) | Lag (L) | Cost |
|---|---|---|---|---|---|---|---|---|---|
| 0-1 | 0-1 | 0-1 | 0-1 | 0-1 | 0-1 | 0-100 years | $ million | ||
| Wilson Wetlands | 15 | 0.25 | 0.88 | 0.7 | 1 | 0.98 | 0.8 | 20 | 5 |
| Rogers River | 80 | 0.01 | 0.83 | 0.5 | 1 | 0.85 | 0.9 | 15 | 3 |
| Riley Reserve | 5 | 0.5 | 0.93 | 0.6 | 1 | 0.98 | 0.5 | 10 | 2 |
| Patterson Park | 5 | 0.25 | 0.93 | 0.6 | 1 | 0.98 | 0.5 | 10 | 2 |
For the purposes of illustration, suppose the following weights are used in a weighted additive scoring system.
| Value (V) | Impact of works (W) | Technical feasibility (F) | Adoption (A) | Adverse adoption (B) | Socio-political risks (P) | Long-term funding (G) | Lag (L) | Cost |
|---|---|---|---|---|---|---|---|---|
| 0.2 | 15 | 10 | 10 | 5 | 10 | -0.5 | -5 |
These weights have been chosen to reflect the sorts of weights that decision makers might actually choose. They are not crafted to generate the anomalies pointed out below.
The table below shows the results for the BCR and for the weighted additive scoring system using the above weights.
| Benefit: Cost Ratio | Weighted additive scoring | ||||
|---|---|---|---|---|---|
| Asset/ project | Score | Rank | Score | Rank | |
| Wilson Wetlands | 2.8 | 3 | 14 | 4 | |
| Rogers River | 0.8 | 4 | 33 | 1 | |
| Riley Reserve | 12 | 1 | 32 | 2 | |
| Patterson Park | 6 | 2 | 28 | 3 | |
Note that it does not matter that the BCR scores are lower than the weighted additive scores. This just reflects the different scoring systems. It is only meaningful to compare scores within a column.
Given the way that these variables have been defined, the Benefit: Cost Ratio accurately reflects the relative cost-effectiveness of the different projects – the environmental benefits per dollar spent. If you wanted to maximise environmental benefits you would prioritise the funding of these projects in the rank order of their BCRs.
In the example above, the ranking of projects using the weighted additive scoring system is very different. In particular, the project that provides by far the worst value for money (Rogers River) is given the highest score! This is mainly because the project has a very low impact on asset condition, but the additive scoring system does not adequately reflect this. In addition, although the Riley Reserve project is actually twice as good as the Patterson Park project, using the weighted additive scoring system, their scores are little different.
We have done some investigation of the costs (in terms of lost environmental benefits) of using addition when you should use multiplication. Generally, the costs are very high. Even if you have perfect information about the projects, you are likely to lose around 50% of the potential environmental benefits just by using an additive index to assess priorities. For details, see here. Fortunately it is easy to avoid the problems by using the BCR or something similar.
603. How does the BCR assist with comparing assets/projects of different scales, durations and types?
Comparing assets/projects of different scales, durations and types has always been a challenge for managers of natural assets. Because of the way it is designed, the BCR assists greatly with this problem.
In principle, the ideal decision method is one that results in the most valuable outcomes overall, given the available budget. The BCR is theoretically consistent with this ideal.
In the BCR, benefits are measured in a way that allows comparison across different types of assets. A key factor in allowing this is standardisation of asset value scores (V) to $20 million per point. To assist with selecting scores, a table of example assets with suggested value scores is provided. The other variables in the benefits index (the numerator of the BCR) are all expressed as proportions or probabilities, so that the unit of measure for the benefits index is essentially the same as for V. The main differences is that V is a particular deterministic value, whereas the benefits index is an expected value, or weighted average, based on various probabilities that the investment will not fail for one reason or another. The benefits index is scaled up by a factor of 20 since each point represents $20 million. Because benefits are measured using the same scale for every asset type, it is valid to compare benefits for projects of different types of assets (e.g. rivers vs land vs native vegetation). Costs are also measured in the same way for each asset type (dollars), and so they too can be validly compared.
In the BCR, the benefits index is divided by total project costs to provide an estimate of the expected level of benefits per dollar spent. This allows us to compare the merits of projects of different scales. The decision rule is to choose the projects with the highest BCRs, down to the point where the budget is exhausted (exactly the same as with Benefit: Cost Analysis, where one chooses those projects with the highest Benefit: Cost Ratios). At least that is the starting point, to be modified following discussion of other relevant considerations (see FAQ 601).
We need to be able to compare projects of different durations. For example, some projects only require a short period of funding, whereas others require ongoing funding over, say, 25 years. We achieve this comparability by discounting future costs back to their present levels (using standard economic discounting methods). The costs included in the BCR account for this discounting process. (For simplicity, costs within the first three- to five-year project are not discounted).
In summary, the standardisation of V and the use of multiplicative proportions allow comparison of projects for different types of natural assets, and dividing by costs allows the comparison of projects of different sizes and durations.
604. I’m looking at two projects that are mostly very similar, except that one has higher costs. INFFER gives them quite different values for the Benefit: Cost Ratio (BCR). Is that right? Doesn’t this bias investment against the project with the higher costs?
Suppose you were looking at buying either a Toyota Corolla or a Honda Civic, and you liked them both equally, but the Toyota was much cheaper. Which would you buy? To buy the more-expensive Honda, you’d have to think it was superior to the Toyota by enough to justify the extra expense. That is exactly what the BCR is capturing for environmental projects.
The BCR represents the benefits of a project divided by the costs of the project. If you have a lot of projects to choose from, and a fixed total budget to invest, the way that an investor can maximize their overall benefits is to choose the set of projects with the highest benefit: cost ratios.
For example, suppose there were 10 projects, each with gross benefits of $10 million, and each with a different level of costs: $1, $2, $3, $4, $5, $6, $7, $8, $9 and $10 million. If you have a budget of $10 million, the best strategy is to choose the first four projects, which have benefit:cost ratios of 10, 5, 3.3 and 2.5. Your total gross benefits would be $40m and your net benefit would be $30m (= $40m – $10m). If you chose the fifth project (benefit:cost ratio = 2), you would have to drop, say the fourth and the first projects to pay for it. The gross benefit would fall to $30m, and total costs would stay at $10m, so your net benefit would fall to $20m (= $30m – $10m). Clearly, projects with the highest BCRs are the priorities.
Note that if one project is just a scaled up version of another one, and the benefits and costs are scaled up by the same proportions, then the BCR will be identical for the two projects.
605. How does the size of an asset affect its Benefit: Cost Ratio score?
If asset A is larger in scale than asset B, this influences the INFFER assessment in several ways, and these influences flow through to the Benefit: Cost Ratio. For the purposes of illustration, assume that, apart from their sizes, the two assets are basically identical.
• Asset A would have a higher score for asset significance (Q1.2(b)).
• Asset A would require more extensive resources to be maintained at the same quality (e.g. more extensive works and actions specified in Q2.2).
• If the project for asset A provided the required extra resources, the score for technical effectiveness would be similar for both projects. If the extra resources were not provided, asset A would have a reduced score for technical effectiveness.
• If asset A requires more extensive changes in land management than asset B, then asset A may have a lower score for the attractiveness of changed practices.
All of these factors would feed into the calculation of the Benefit: Cost Ratio. The BCR for asset A may be higher or lower than for asset B, but on balance is more likely to be lower.
See also FAQs 64, 65, 70, 106.
