The Parliamentary Committee on comprehensive crisis management, which finished its work this spring, noted that the Finnish participation in crisis management lacked clear and measurable objectives against which the effectiveness of the participation could be evaluated. The committee recommended the establishment of clear quantitative and qualitative objectives for participation in crisis management, and the development of indicators to evaluate these objectives and the effectiveness of the participation. What then is meant by the effectiveness of crisis management, which aspects of effectiveness may need to be considered, and what kind of difficulties may its measurement entail? And what kind of effectiveness research is currently being carried out at CMC Finland?
Effective for whom?
With studies of effectiveness related to crisis management, the review often focuses on an operation and its mandate, and effectiveness is considered against the mandate’s fulfilment. Reviews focusing on the fulfilment of mandates are a natural way to study the effectiveness of an operation, because the success of the operation in areas which it does not even seek to affect cannot really be expected.
The most important objectives of civilian crisis management are related to changes and developments that the activities support at the local level. A crisis management operation cannot be genuinely effective if it does not prioritise the needs of the local community. Other kinds of operational planning lead to an inability to recognise the central issues of a crisis or conflict and respond to them, which in turn leads to a waste of resources. However, the mandates of operations can sometimes one-sidedly reflect the needs of those other than the local people. It must thus be emphasised that a positive effectiveness evaluation related to the fulfilment of an operation’s mandate does not necessarily mean that the operation was equally effective from the perspective of the local population. The operation may ultimately very effectively carry out the wrong things if the mandate does not correspond to the needs of the local population.
However, it is clear that alongside objectives related to local challenges and developments, crisis management as political activity always also entails other reasons and motives, and these different aspects are not necessarily mutually exclusive. It is possible to serve the objectives of local communities so that crisis management simultaneously produces concrete positive results more broadly, e.g. in Europe or Finland. This idea is specifically referenced in arguing that participation in civilian crisis management also improves the safety of people in Finland, as stated e.g. in the latest Government Report on Finnish Foreign and Security Policy.
The idea is based on the assumption that internal and external security are inseparable in an interdependent world, and we cannot affect one without considering the other. At the level of practical policies, the connection between internal and external security is seen in the effort to deepen the cooperation between the European Union’s CSDP operations and operators in Justice and Home Affairs (JHA), for example. However, if we want the discussions e.g. on the improvement of internal security through participation in crisis management to be based on more than just an assumption, we need also to consider the means by which we could verify and develop this side of effectiveness, alongside the review of changes at the local level.
For this requirement, the Ministry of the Interior initiated a research and development project at the end of last year to evaluate the achievement of the objectives of civilian crisis management from the perspective of the administration of internal affairs. The project is being coordinated by Crisis Management Centre (CMC) Finland, and key partners include the Finnish Immigration Service, the Police University College and the Ministry’s unit for international affairs.
No effectiveness without objectives
Our project studies the effectiveness of civilian crisis management from the perspective of the administrative sector of the Ministry of the Interior, taking three aspects into account. First, we review the strategic and political level, i.e. how the participation of employees of the administration of internal affairs in civilian crisis management duties advances the objectives for the participation of the Ministry of the Interior’s administrative sector, such as the improvement of internal security. For example, we examine whether we have succeeded in sending sufficient expertise with respect to the objective to geographically or thematically critical locations. In this case, for instance, the number of specialists in civilian crisis management duties in a certain area, or the thematic competences they have, can be used as an indicator. The objective is thus to find more qualitative indicators than just those measuring the total number of specialists.
Challenges to the work have been caused especially by an observation also confirmed by the Parliamentary Committee that the objectives of participation in civilian crisis management have not been very precisely defined historically, except perhaps for the total amount of participation. To make the review of effectiveness sensible, concrete and measurable objectives are needed against which the activities can be evaluated. In terms of effectiveness, it is also important to carefully consider why an objective is important already at the stage of establishing the objective. In other words, the objectives of the activities must be defined, alongside what is sought with each objective from the perspective of effectiveness. Once established, the objectives should also be critically reviewed from time to time. Does the participation or action that follows the objectives lead to the desired kind of effect? Would another operation or job function perhaps be more significant from the perspective of a specific effectiveness objective? Resources can thus be allocated to those functions that best support the achievement of effectiveness.
The governing idea in the project has been that we are not so much seeking an answer on the effectiveness of specialists who were working abroad at a specific time, as is typical of effectiveness evaluations. This type of ex-post review does indicate success at a specific time, but it does not guarantee that the situation cannot change rapidly. Instead, our goal has been to build a real-time and lightweight model that acts more like an early warning system, i.e. to provide signals if our effectiveness objectives seem to escape reach.
A learning organisation, a developing specialist
In addition to the fulfilment of political and strategic objectives, we also review the impact of the experience that specialists accrue from civilian crisis management missions on the specialists’ background organisations; i.e. whether this experience is utilised in the organisations, or whether there is room for improvement in this respect. Finally, we review effectiveness from the perspective of the specialist’s own professional development, i.e. what kind of effects the participation in civilian crisis management has had, e.g. on the specialist’s career development within their organisation.
Information on themes related to the background organisation and professional development has previously been collected through feedback events and surveys aimed at CMC’s specialists, as well as individual larger projects. In these, most specialists have mentioned the development of language skills, cultural knowledge and interaction skills during civilian crisis management duties, regardless of the particulars of their assignment. In addition, specialists have accrued knowledge related to project management, the subject matter of their duties in the homeland and management skills during civilian crisis management. However, there have been challenges with the systematic collection of information on effectiveness in the different organisations. It has also been unclear how effectively the information received from the specialist is deployed within the background organisations. During the project, the process is intended to be developed to ensure information on effectiveness is collected where it will also be used. This ensures that information is collected and effectiveness is evaluated appropriately from the perspective of the background organisation. This also facilitates the implementation of the necessary corrective measures if any problems are detected.
With the sections concerning the specialist and their background organisation, the main responsibility for advancing the project currently lies with the Finnish Immigration Service and the Police University College, which study these questions for their own personnel. It still remains possible and desirable for other organisations to join the project. In addition to the administration of internal affairs, we have also already had discussions with the Ministry of Justice on extending our review to the specialists of the judicial administration for the effectiveness experienced by the organisation and the specialist.
The coronavirus situation has made the situation more difficult: because CMC Finland’s feedback events are currently suspended, we have thus far been unable to accrue interview material on how specialists feel their assignments are advancing the achievement of the current objectives. Nevertheless, the plan is to construct a model for how to evaluate effectiveness from the perspective of the administration of internal affairs and Finland in general during the autumn at the latest, when it is hoped the implementation of the recommendations of the Parliamentary Committee on crisis management will also start in earnest.
This project, which reviews effectiveness from the perspective of the administration of internal affairs, is only part of the effectiveness research carried out at CMC Finland. For example, we are part of the international EPON network, which studies the effectiveness of peace operations, emphasising the previously mentioned local perspective. It is important to evaluate effectiveness at different levels and from different perspectives, because civilian crisis management and its objectives are also far from being one-dimensional. The work carried out at CMC Finland to develop the evaluation of effectiveness will receive additional resources in the summer, when a new specialist focusing on effectiveness starts working in the Research, Development and Situational awareness unit.
Jyrki Ruohomäki
Johanna Hakanen
