For a better user experience please update your browser or use Chrome or Firefox browser.
Utilities
Publication detail
HomePilot and feasibility study on the sustainability and effectiveness of results for European Social Fund participants using counterfactual impact evaluations
Final report
Publication metadata
This study is aimed at assessing the feasibility of Counterfactual Impact Evaluation (CIE) as an advantageous evaluation method of European Social Fund (ESF) interventions, to be used at EU level, relying on data collected at national level. The study is based on a mix of desk analysis, field analysis, questionnaires and interviews with Managing Authorities and Institutions holding relevant data, as well as econometrics. First, an exploratory analysis of available administrative data in each Member State of the EU was carried out. Then, three CIE pilot evaluations were produced, focusing on ESF operations for long-term unemployed (in Italy, Slovakia and Spain). A further CIE pilot, concerning ESF interventions for early school leavers in Belgium was planned but it was not possible to obtain the necessary data in due time. The pilot evaluation on the Veneto region provides an analysis of an ESF supported operation consisting of training and/or internship. It found a positive and statistically significant effect of the support: in fact, the probability of participants to be employed 18 months after the enrolment in the operation is 14% higher than for the control group (who have not participated in ESF operations). In Slovakia, the pilot study concerned an ESF intervention aimed at supporting jobseekers by providing paid jobs in public sector. It found a positive effect on employment: treated people have a probability of being employed 2.2 percentage points higher than the control group after 18 months the completion of the intervention. Also the pilot carried out in Catalonia found a positive impact on employment: 16 months after launching the interventions, ESF operations increased the employment rate by 12 percentage points among the participants in comparison to the control group. The findings of the study confirm the possibility of conducting CIE of labour market operations using administrative data, in the considered 26 Member States. CIE can be also used for assessing education operations, in many but not all the considered Member States. Notwithstanding the feasibility of using CIE, in many cases several improvements with respect to data accessibility and usability should be implemented by Member States, for instance by applying a ‘one-stop-shop’ approach to data delivery, as well as unified and simplified protocols for privacy rules. Improvements in data quality can also extend the application of CIE techniques and improve the robustness of the results. It is also recommended to anticipate planning of CIE in the design stage of the ESF operations and to promote dedicated communication for disseminating evaluation results. At EU level, the implementation of a unique European CIE of similar ESF operations, implemented in several countries, is a hypothetical possibility which, however, for theoretical and practical reasons is not feasible. In order to produce a valid policy assessment at EU level, another approach would consist of coordinating, comparing and summarising a pool of national or regional CIEs that examine very similar interventions. However, given the heterogeneity in actual programme implementation, beneficiary groups and data contents, such an approach is challenging. A meta-analysis is a third possibility which, potentially, can be implemented more directly. This approach would be feasible, based on current and evolving CIE practices, but it is likely to produce better results when the data collection is systematic and the number of individual CIEs is large enough. A CIE policy by the Commission should aim at ensuring adequate coverage of the different ESF operations and set quality standards, with a view to increasing comparability among national CIEs.
- Published:
- Corporate author(s): Directorate-General for Employment, Social Affairs and Inclusion (European Commission) , IES- Ecorys , Ismeri Europa
- Themes: European Social Fund
- Subject: data collection , dropout , EU Member State , European Social Fund , feasibility study , job application , labour market , long-term unemployment
|
PDF
|
|||
| ISSN | ISBN 978-92-76-08840-0 | DOI 10.2767/39339 | Catalogue number KE-01-19-580-EN-N |
- Released on EU publications website:
Document viewer
Note:
The publication cannot be displayed in the document viewer. Open web version
You may access it from: Download and languages