- Decision Support Systems for water resources management: current state and guidelines for tool development
1.1 Water Management DSS
There are many different understandings of what Decision Support Systems are, what they are composed by, what they do (or should do) and how. They include a vast variety of methods and tools developed for diversified purposes and contexts and for that reason providing a unique definition is practically impossible. A computer-based tool is surely one important component, but more and more DSS is intended as a broader combination of the tool(s) and the process of structuring problems and aiding decisions (Section 2 provides an introduction to DSS concepts and terminology).
DSS in the water management sector usually consist of simulation models, and/or of techniques and methods for decision analysis, recently extended to include the support to participatory processes. Therefore, a DSS typically integrates multi-source geographically referenced data and data management systems, a variety of models and elaboration procedures within a customised user interface. Emphasis is given to hydrologic models accompanied by environmental assessment and/or socio-economic evaluation. The models include both those aimed at reconstructing and simulating the physical reality, and those constructed to manage divergent objectives and to find a compromise among the expectations of different actors in a participatory process. In an idealised view DSS should act as mediators between science and policy/decision making and as catalysts of trans-disciplinary research.
Methodological proposals and tools have been developed since the 1970s. Unfortunately, DSS have found very limited implementations in the real world, thus demonstrating that most of the DSS tools developed so far have failed to meet the objective of being used in the real world (in Section 3 a brief methodological overview and a literature review are provided).
The question arises then, whether realism could allow us to make proper and effective use of
improved
DSS, or whether abandonment should be preferred. We will not provide a definitive answer, but we will identify a set of best practices to be implemented with realism, to provide the ground for more effective future developments and applications. (see Box A below and Section 7 for details).
1.2 Models, DSS tools and Integrated Water Resources Management (IWRM)
As the conflicts for water have exacerbated and the policies have become more articulated and complex, more scientifically robust methods are needed by managers and policy makers. The challenges imposed by the implementation process of the Water Framework Directive (WFD) and the putting in practice of the principles of Integrated Water Resources Management (IWRM) are emblematic in this regard. Those challenges may represent an important triggering factor for the development of improved DSS tools and, indeed, the 5th and 6th Framework Programmes of European research mobilised significant financial resources, targeting that process.
Unfortunately, in the attempt to cope with issues of increasing complexity, methods and computer tools (simulation models in particular) have shown a tendency to become more and more sophisticated and complicated, and there is a growing gap between the specialised knowledge of the DSS developers and the application of this knowledge in decision making.
1.3 The perspectives of DSS end users
In order to investigate the reasons for the frequent failures of DSS tools in being adopted by the intended end users (i.e. water managers and policy makers), the report includes substantial contributions from outside the academia and from representatives of the community of DSS end users who were involved in the writing and compiling of an ad hoc questionnaire.
The scope of the questionnaire was to acquire insights into actual and potential users needs, expectations and satisfaction, during the tools development and application.
As a prerequisite for consideration in this report the responses to the questionnaire given by DSS users had to be distinct from those given by DSS developers. This dramatically limited the choice of tools, since it became clear very soon that DSS applications are seldom used outside/after the original development process and/or by users different from the software developers. The objective of examining the ten tools and collecting at least two end-users questionnaires for each tool was very challenging and was met only after great efforts and after several selected DSS were discarded (details about the results are provided in Section 4).
Neither the selection of DSS nor the information collected through the questionnaires are intended as representative samples of the universe of tools and application cases, but at least this work is probably the first attempt to involve potential end users in assessing the outcomes and the impacts of recent research efforts in developing water management DSS.
Most of the respondents reached were specialised in the field of water planning and management; just a few were experts in modelling. In general they were advisors or policy makers, although a significant portion consisted of researchers involved in DSS applications in the context of case studies.
Most of the users reached were at their first experience with a DSS. The main motivations for that new experience were acknowledged to be a proposed partnership from research institutions and the emerging needs or inability of the previous management system to approach increasingly complex decisions. Most frequently, the main aim for developing and acquiring the DSS was to encourage and simplify stakeholders involvement. Other frequent motivations mentioned by the end users were the need for an enhanced identification of alternative policy options and for providing transparency to previous policy decisions. The satisfaction of specific regulatory obligations and requirements, i.e. the implementation of the WFD, was also pointed out as a target of the tool application. Section 5 goes deeper in the systematic analysis of the needs and the potential role of science.
In the users opinion, the DSS resulted useful in a broader context, not just informing the choice of policy, but facilitating the planning process and learning among the actors, contributing to finding a compromise among different expectations and interests.
The adoption of DSS to simplify stakeholders involvement did not necessarily make the decision making process easier. In some cases this was due to the inadequate technical features of the systems. In other cases the use of the DSS evidenced the poor quality of the input data, or failed to provide an effective user interface or to communicate hidden uncertainties. Moreover, the lack of cooperation between the partners beyond the development project was pointed out as one reason for limited use and, in some cases, even the failure of the project to deliver the tool.
1.4 Bridging the science and policy gap?
The results of the survey evidenced the role played by research in DSS implementation in the real world. One can easily say that the adoption of a DSS tool by a potential end user, who is not involved in a research activity, is, to say the least, occasional. Therefore, even assuming that more effective dialogues between the science and policy spheres provide the ground for more effective DSS tools in the future, one of the main issues for future efforts in the fields of training and capacity building is the crucial role of mediators to encourage intended end users to adopt research outputs.
Indeed, science has been increasingly called to inform environmental policy making. Appeals to provide 'useful' knowledge, i.e. one with direct policy implications, is a fundamental ethical principle for scientists. At the same time however, the dominant authority of science as the most privileged source of knowledge is being increasingly challenged. Moreover, it is crucial to realise that science and policy making, despite their interdependency, are rooted in different cultures and embodied in distinct frameworks of values, incentives and concerns. These differences have frequently led to frustrating experiences at the interface between science and policy.
Section 6 analyses the known factors of DSS success or failure based on the evidence of the international literature. Despite their crucial importance, the identification of DSS success factors and their measurement is a difficult task, since the development and application of DSS entail multiple potential benefits. The unambiguous detection of DSS failure is at least as difficult as the measurement of its success. From the experiences gathered so far it appears clear in general that the process of policy making is at least as important as its outcomes. Therefore, one important criterion is the degree of change to the usual management introduced by the implementation of the DSS. This could have a dual meaning, since an acknowledged limiting factor is represented by the resistance of managers to changes in conventional practices.
1.5 The Guidelines for DSS development and implementationThe final part of the report (Section 7) includes the Guidelines for the development, implementation and application of DSS tools. That section includes a series of recommendations that have been developed to increase the probability of future successes in DSS developments for the practical implementation of the IWRM principles. They have been organised according to a generic temporal sequence, by identifying three main phases:
Details about the results of the DSS and questionnaire surveys are reported in the Annexes.
There are many different understandings of what Decision Support Systems are, what they are composed by, what they do (or should do) and how. They include a vast variety of methods and tools developed for diversified purposes and contexts and for that reason providing a unique definition is practically impossible. A computer-based tool is surely one important component, but more and more DSS is intended as a broader combination of the tool(s) and the process of structuring problems and aiding decisions (Section 2 provides an introduction to DSS concepts and terminology).
DSS in the water management sector usually consist of simulation models, and/or of techniques and methods for decision analysis, recently extended to include the support to participatory processes. Therefore, a DSS typically integrates multi-source geographically referenced data and data management systems, a variety of models and elaboration procedures within a customised user interface. Emphasis is given to hydrologic models accompanied by environmental assessment and/or socio-economic evaluation. The models include both those aimed at reconstructing and simulating the physical reality, and those constructed to manage divergent objectives and to find a compromise among the expectations of different actors in a participatory process. In an idealised view DSS should act as mediators between science and policy/decision making and as catalysts of trans-disciplinary research.
Methodological proposals and tools have been developed since the 1970s. Unfortunately, DSS have found very limited implementations in the real world, thus demonstrating that most of the DSS tools developed so far have failed to meet the objective of being used in the real world (in Section 3 a brief methodological overview and a literature review are provided).
The question arises then, whether realism could allow us to make proper and effective use of
improved
DSS, or whether abandonment should be preferred. We will not provide a definitive answer, but we will identify a set of best practices to be implemented with realism, to provide the ground for more effective future developments and applications. (see Box A below and Section 7 for details).
1.2 Models, DSS tools and Integrated Water Resources Management (IWRM)
As the conflicts for water have exacerbated and the policies have become more articulated and complex, more scientifically robust methods are needed by managers and policy makers. The challenges imposed by the implementation process of the Water Framework Directive (WFD) and the putting in practice of the principles of Integrated Water Resources Management (IWRM) are emblematic in this regard. Those challenges may represent an important triggering factor for the development of improved DSS tools and, indeed, the 5th and 6th Framework Programmes of European research mobilised significant financial resources, targeting that process.
Unfortunately, in the attempt to cope with issues of increasing complexity, methods and computer tools (simulation models in particular) have shown a tendency to become more and more sophisticated and complicated, and there is a growing gap between the specialised knowledge of the DSS developers and the application of this knowledge in decision making.
1.3 The perspectives of DSS end users
In order to investigate the reasons for the frequent failures of DSS tools in being adopted by the intended end users (i.e. water managers and policy makers), the report includes substantial contributions from outside the academia and from representatives of the community of DSS end users who were involved in the writing and compiling of an ad hoc questionnaire.
The scope of the questionnaire was to acquire insights into actual and potential users needs, expectations and satisfaction, during the tools development and application.
As a prerequisite for consideration in this report the responses to the questionnaire given by DSS users had to be distinct from those given by DSS developers. This dramatically limited the choice of tools, since it became clear very soon that DSS applications are seldom used outside/after the original development process and/or by users different from the software developers. The objective of examining the ten tools and collecting at least two end-users questionnaires for each tool was very challenging and was met only after great efforts and after several selected DSS were discarded (details about the results are provided in Section 4).
Neither the selection of DSS nor the information collected through the questionnaires are intended as representative samples of the universe of tools and application cases, but at least this work is probably the first attempt to involve potential end users in assessing the outcomes and the impacts of recent research efforts in developing water management DSS.
Most of the respondents reached were specialised in the field of water planning and management; just a few were experts in modelling. In general they were advisors or policy makers, although a significant portion consisted of researchers involved in DSS applications in the context of case studies.
Most of the users reached were at their first experience with a DSS. The main motivations for that new experience were acknowledged to be a proposed partnership from research institutions and the emerging needs or inability of the previous management system to approach increasingly complex decisions. Most frequently, the main aim for developing and acquiring the DSS was to encourage and simplify stakeholders involvement. Other frequent motivations mentioned by the end users were the need for an enhanced identification of alternative policy options and for providing transparency to previous policy decisions. The satisfaction of specific regulatory obligations and requirements, i.e. the implementation of the WFD, was also pointed out as a target of the tool application. Section 5 goes deeper in the systematic analysis of the needs and the potential role of science.
In the users opinion, the DSS resulted useful in a broader context, not just informing the choice of policy, but facilitating the planning process and learning among the actors, contributing to finding a compromise among different expectations and interests.
The adoption of DSS to simplify stakeholders involvement did not necessarily make the decision making process easier. In some cases this was due to the inadequate technical features of the systems. In other cases the use of the DSS evidenced the poor quality of the input data, or failed to provide an effective user interface or to communicate hidden uncertainties. Moreover, the lack of cooperation between the partners beyond the development project was pointed out as one reason for limited use and, in some cases, even the failure of the project to deliver the tool.
1.4 Bridging the science and policy gap?
The results of the survey evidenced the role played by research in DSS implementation in the real world. One can easily say that the adoption of a DSS tool by a potential end user, who is not involved in a research activity, is, to say the least, occasional. Therefore, even assuming that more effective dialogues between the science and policy spheres provide the ground for more effective DSS tools in the future, one of the main issues for future efforts in the fields of training and capacity building is the crucial role of mediators to encourage intended end users to adopt research outputs.
Indeed, science has been increasingly called to inform environmental policy making. Appeals to provide 'useful' knowledge, i.e. one with direct policy implications, is a fundamental ethical principle for scientists. At the same time however, the dominant authority of science as the most privileged source of knowledge is being increasingly challenged. Moreover, it is crucial to realise that science and policy making, despite their interdependency, are rooted in different cultures and embodied in distinct frameworks of values, incentives and concerns. These differences have frequently led to frustrating experiences at the interface between science and policy.
Section 6 analyses the known factors of DSS success or failure based on the evidence of the international literature. Despite their crucial importance, the identification of DSS success factors and their measurement is a difficult task, since the development and application of DSS entail multiple potential benefits. The unambiguous detection of DSS failure is at least as difficult as the measurement of its success. From the experiences gathered so far it appears clear in general that the process of policy making is at least as important as its outcomes. Therefore, one important criterion is the degree of change to the usual management introduced by the implementation of the DSS. This could have a dual meaning, since an acknowledged limiting factor is represented by the resistance of managers to changes in conventional practices.
1.5 The Guidelines for DSS development and implementationThe final part of the report (Section 7) includes the Guidelines for the development, implementation and application of DSS tools. That section includes a series of recommendations that have been developed to increase the probability of future successes in DSS developments for the practical implementation of the IWRM principles. They have been organised according to a generic temporal sequence, by identifying three main phases:
- the phase prior to the actual development or acquisition of the DSS tool;
- the phase of development (in case of new tools to be implemented) or acquisition/adaptation of existing ones;
- the phase of implementation and application in the decision case.
- The most important messages and keywords to be extracted for the Guidelines are:
- accurate preliminary exploration of the problem and identification of actors involved;
- framing of the process and the DSS tool within the existing institutional setting;
- early involvement of end-users;
- match between DSS requirements and data availability and local knowledge;
- match between local practices and the DSS procedure;
- flexibility;
- communication within and outside the group of users;
- documentation;
- training and capacity building;
- dissemination and maintenance of the tools.
Details about the results of the DSS and questionnaire surveys are reported in the Annexes.
Tweet this
Print
