Evaluation and reporting Baseline review Target setting Political commitment Implementation and Monitoring Integrated management system Organisational setup Involvement and communication

Introduction to Implementation and Monitoring

With the implementation of the strategic programme and the action plan, the management cycle reaches its very core: all the preceding assessment and planning has the overall objective of improving the way the city functions in terms of sustainable development. The implementation is where it shows. The implementation is a demanding task in terms of organisation and coordination of all the parallel actions that will take place. A crucial condition is a solid communication and involvement approach and the organisational setup. Cooperation with and between various stakeholders assures that the different actors buy in to the implementation process. Therefore, implementation is based on the “foundation” which is a combination of the action plan, the organisational setup and above all – communication and involvement. The approval of the action plan by the city council may be a determining success factor, as it legitimises actions and gives them a required priority.

In parallel, and for the purpose of being able to measure and report the results, the implementation of the strategic programme and its action plan should be monitored in an appropriate way and fed back to the politicians. It allows for being able to see if actions are implemented with good results. If not, it allows for taking corrective measures while implementation is in progress. In order to be able to engage in monitoring, the actions need to rely on targets based on indicators defined in the strategic programme.

Refinement of the action plan

With the strategic programme and action plan approved by the city council, an important step has been taken towards improvement of the sustainability performance. The action plan serves as a basis for the implementation of the strategic programme, but needs to be further refined. Outline and segment the objectives and targets to concretise measures and projects to be realised by the relevant departments and stakeholders involved.In order to serve as a good basis for day-to-day work, ensure that the action plan answers the following questions:

  • What is to be done? (title, short description of measure)
  • Who is responsible? (person, department or stakeholder, contact details)
  • Who should support implementation and how (person, department, contact details)
  • By what time should the measures be completed? (timetable, date of realisation)
  • Which personnel and financial resources are assigned? (staff, budget)
  • Is public information legally required? (which information? to whom? through which instruments? when? how often?)
  • Are green procurement procedures in place in case of a call for tender or purchase by the local government? (yes/no (If not in place yet, green procurement procedures should be introduced.)/under development)

For the majority of measures, stakeholder involvement will be necessary for realisation. This requires substantial and formal agreements with stakeholders. Describe the measures taken by stakeholders in the same way as the actions worked out by the municipal departments.

Concrete measures could also include the training of personnel in specific aspects and/or new legislation. Further information regarding staff training – for example, the responsibilities for training and an analysis of training needs – is provided in the “Organisational Setup” section.

The heads of the department and the involved stakeholder organisations are responsible for the concretisation of measures. The coordinator is responsible for compiling the information on the refined measures and to document them in the “refined action plan”.

The responsible department documents and completes the refined action plan and makes it available at one spot for all actors involved. It serves as a basis for the monitoring of realisation of activities through the coordinator and responsible persons assigned.

Monitoring

Good management practises include regular monitoring on both a short- and long-term basis. An effective monitoring process provides ongoing, systematic information that strengthens project implementation. The monitoring process provides an opportunity to:

a) compare implementation efforts with original goals and targets,
b) determine whether sufficient progress is being made toward achieving expected results, and,
c) determine whether the time schedule is observed.

Monitoring is not an “event” that occurs at the end of a management cycle, but rather is an ongoing process that helps decision-makers to better understand the effectiveness of the action or system. An effective monitoring and evaluation programme requires collecting and analyzing important data on a periodic basis throughout the management cycle of a project. This process often involves collecting baseline data on existing conditions, reporting on progress toward environmental/sustainability improvements, making connections between actions and intended outcomes, and making mid-course changes in program design.

An effective monitoring and data management system records the performance of all institutions with implementation responsibilities. It provides a system of accountability for all responsible parties on how well they are achieving the goals and targets established in the IMS. The responsibility of appropriate application of the monitoring system lies with the responsible persons/organisations/authorities assigned to this activity and has to follow the reporting duties as outlined during the “organisational setup” phase.

Implementation together with monitoring show how important it is to work with indicators and SMART targets from the very beginning of the system implementation. The work with indicators and measurable data has to start with the baseline review. Key data based on indicators have to be mapped in addition to analysis and recognition of missing indicators on the occasion of the baseline review of the existing situation. Within the next step of the system, these key data and indicators are used to formulate SMART targets in the strategic programme and as a result will form the basis for the action programme and therefore the basis for implementation processes. Finally, the implementation can be further controlled and monitored, referring to the clearly defined indicators and thus SMART targets. Effectiveness monitoring is thus very much dependent on a baseline recognition and reference to selected indicators.

It is important to have a look at the current situation of monitoring to adapt the monitoring system as much as possible to the structures in place and to avoid duplication of work. Several of the EU based legislations require monitoring (e.g., Strategic Environmental Assessment, Clean Air Directive, etc.). Very often, monitoring activities are very dispersed and not linked with each other. It would, however, be very useful if one unit could make use of data collected in another unit. As an example, the monitoring results of the traffic control department could provide useful insights for the health department. The IMS delivers a platform to connect the various data sources.

An effective monitoring and reporting system ideally includes the following elements:
  • Clearly articulated targets and a set of indicators to measure performance;
  • A schedule and set of guidelines for all responsible parties to report to each other;
  • An opportunity for responsible parties and stakeholders to periodically meet to coordinate actions and review each other’s performance
  • A link between the evaluation reports and relevant statutory planning cycles of the municipality, such as annual budgeting and capital planning, so that the municipality can adjust its plans as based on the actions taken by other sectors.

Collecting Data

In preparing the monitoring setup, it is good to check the following questions:

  • For which indicators are data currently being collected?
  • What are key information sources? Are representatives from these information sources already involved in the IMS process?
  • How valid and accurate are the data?
  • Are the data easily accessible and available?
  • Are there any costs associated with acquiring the data?
  • For those indicators where no data currently exists, which steps should be taken to collect new data? How expensive would a new data collection effort be?

Ideally, most monitoring processes include collecting both quantitative and qualitative data.

Quantitative data is information that can be counted and measured. Quantitative environmental data focuses on actual environmental improvements, such as the amount of waste reduced or energy saved. Mechanisms for collecting quantitative environmental data are usually programme-specific, such as using water meters to measure actual water consumption.
 
 
 
Qualitative data is a more difficult measurement of programme success. It includes assessments of problems encountered, stakeholder satisfaction, and unanticipated benefits. Qualitative data can give a real understanding of the actual impact the actions are making on people’s lives. It is usually collected through instruments such as surveys and personal interviews. In order to have a better understanding of the successes and challenges, it is advisable to collect both types of data. For example, to address persistent water shortages, a town may decide to implement a pilot water conservation programme to install low-flow showerheads in residences. A quantitative data collection effort would focus on how much water has actually been saved, while qualitative data would reveal how satisfied consumers were with the performance of the new showerheads. Both types of information are imperative to determine whether the program was successful.

“Pressures – state - impact – response”

When applying the analytical framework of “pressures – state - impact – response” to monitoring, it becomes obvious that all of the four areas need to be monitored. Ideally, this has already been taken into account when choosing the indicators. If not, the indicators can still be sorted at this point according to the schematic to facilitate the analysis:

Image 1

To take a concrete example, if you want to monitor the effectiveness of an action, say a noise protection wall, you need to measure

  • whether the wall has been erected (response),
  • whether the noise level has been reduced as a result of the wall (state),
  • or if the source of the noise has decreased (independently of this action, maybe due to related measures in the action plan) (pressure),
  • and finally, if the affected people living there are less disturbed now (impact).

In collecting data, it is also important to distinguish between compliance monitoring versus effectiveness monitoring — both types of monitoring are important. Compliance monitoring measures whether the implementing institution did what it said it was going to do (e.g., install 5,000 low-flow showerheads, or to use the above example, the noise protection walls have been installed), while effectiveness monitoring measures whether the actions achieved their intended result (e.g., reducing water usage by 20% per household, or the decibels behind the wall are a certain rating lower). Of course, the real measure of success is effectiveness, i.e., how well conditions are improving. However, compliance monitoring is a critical piece of the evaluation process to help determine whether implementing institutions have fulfilled their commitments, and it helps to analyse why goals were not reached where this is the case.

Systematic data collection

This system can be as simple as using standardized reporting forms to facilitate the collection and compilation of data up to an entirely computerized data-sharing system. Nevertheless, what counts is not the level of high-tech computer application that is installed to manage your data, but whether the indicators chosen and the items monitored accurately reflect the progress of implementation and allow for an analysis of deviation from targets and goals.

Data management
Observe Set up a system and tools that allow keeping track of changes (based on the indicators and targets, considering PSIR).
Set time intervals for measuring
Record Organise the way you store data in a logical way

Be sure to collect data on the indicators prior to beginning the implementation of the strategic programme. Ideally, you used the same set of indicators already for the collection of data for the baseline review, upon which the impacts of implementing selected actions will be measured.

In practical terms, data collection is less troublesome if it is organised systematically and in a reproducible way, i.e.,:

  • Establish overviews of existing data sources in the form of inventories and a meta-database;
  • Prepare templates and guidelines on data handling, technical documentation of the data and analytical methods used;
  • For each dataset and indicator (diagrams, tables and maps), maintain a paper or electronic fact-sheet with description of data source, data quality, methods for data compilation, other relevant information and its graphical draft.

Each institution submits information to the responsible coordinator, who is responsible for assuring the system and who compiles this information into a report. The more specifically the coordinator formulates what he/she expects the different departments to monitor, the easier it will be to systematically store and later retrieve this information, to make it accessible to a larger audience and to evaluate the results.

A good monitoring and evaluation process engages all stakeholders and is useful to those ultimately responsible for improving the project. Evaluation is also an important public awareness and educational tool.

The objective is to use synergies, avoid double work and report as efficiently as possible. Please consult the section “organisational setup”, which has further information on reporting routines.

To monitor the realisation of the action in accordance with the action plan is easier than monitoring their effects. The coordinator needs to receive feedback from the responsible persons regarding the

  • realisation of the activity - yes/no?
  • agreed timetable
  • assigned resources
  • modifications

With this information, the coordinator is able to control the compliance of the implementation process with the action plan and to elaborate one part of the interim report or internal audit report.

Corrective measures

A well-organised monitoring system is able to detect deviations from the set direction promptly. The mistake can be analysed immediately, corrective measures taken as soon as possible, and damage or loss minimised.

Communication and involvement

The effectiveness of implementation is very much dependent on the partnerships developed and the involvement and cooperation of various stakeholders. This is the step of actual action, which most often creates lots of challenges, mainly due to the fact that it requires the cooperation of different sorts of groups with various stakes. Planning is usually much less complicated than the actual implementation of the action plan.

At this stage, questions about cross-sectoral cooperation and multi-stakeholder involvement and cooperation appear. Is the organisational set-up cross-sectoral enough? Are those to implement working together for a common objective? Is the common objective of sustainability clear? Are departments cooperating with each other and are the relevant stakeholders involved? Are the targets SMART? Are indicators available and measurable? Is the timetable realistic? In order to make things happen, address all these issues in line with the development of the integrated management system in the city.

The public needs to know what is ongoing. To inform the public about the implementation of the action plan may play the decisive role as to whether the implementation will be successful or not happen at all. Very often the effectiveness of the implementation depends on citizens' involvement. In this case, combine information about the ongoing implementation processes with the call for cooperation and involvement. The examples are various: waste separation, rational energy use (decreasing energy consumption in households) or water-saving measures, to name just a few.

To find out more, please read the section on communication and involvement.

Sources

Regional Environmental Center (REC): “Guide to Implementing Local Environmental Action Programs in Central and Eastern Europe” (author: Paul Markowitz, 2000)

Annexes

Annex 1

Example of a monitoring form for a specific action
Title of action, number in action plan
Summary of action and results
Description of target Summary excerpt from relevant part of Strategic programme
Description of action Summary excerpt from relevant part of action plan
Data collection information Indicators, reporting requirements, data collection methods
Results Qualitative and quantitative impacts
Difficulties encountered Description of problems encountered both internal and external and how these problems were dealt with
Lessons learned What knowledge has been gained?
Recommendations for adaptation of targets and /or actions What recommendation is given for further implementation effort?