Measures to Support a Shift in Research Assessment Culture

Successful science and research require suitable framework conditions. The German Research Foundation (DFG) ensures these conditions by regularly conducting analyses, providing relevant information and adapting its procedures accordingly. In May 2022, the DFG issued a position paper on academic publishing, declaring both the academic community as a whole and itself as a funding organization as responsible for initiating a cultural shift towards research assessment that is geared more towards equal opportunity and attaches even greater importance to the substance of research. It is also up to research funding organizations to broaden the spectrum of accepted publication formats, to attach greater value to content-based evidence of achievement and to strengthen the recipient side of publishing. In the interests of bringing about such a shift, the DFG has launched a comprehensive and far-reaching package of measures in order to fulfil this mandate, including the introduction of a mandatory CV template. The DFG hopes that this refocusing – away from quantitative indicators and towards the substance of scholarship – will lead to improved equality of opportunity and a higher-quality basis for review overall.

Binding CV template across all funding programs
In order to strengthen qualitative evaluation criteria over quantitative indicators, the DFG will be introducing a curriculum vitae template that will be mandatory across all programs from 1 March 2023 (the template will be adapted shortly for proposals under the Collaborative Research Centre and Research Training Group programs; information will be provided separately). The template adopted by the DFG Senate allows applicants to provide both narrative and tabular information, thereby facilitating a holistic view of the applicant’s academic career in the review and evaluation process. In addition to mandatory information required in order to assess eligibility, applicants may also provide details of special circumstances or additional services to scholarship such as committee activities or the establishment of research infrastructures. As such, the template provides the basis for a qualitatively sound assessment of academic performance that takes greater account of the respective stage of the individual’s life and career. Accordingly, reviewers are now instructed to consider applicants’ academic performance in the context of their individual CV and career stage.

Publication details in applications and CVs
Performance assessment based on content-related qualitative criteria also explicitly includes ensuring that the entire spectrum of academic publication types is equally displayed and acknowledged in funding proposals and CVs. In addition to a maximum of ten publications in the more common publication formats, the CV can therefore now list up to ten further sets of research outcomes and findings that have been publicized in a variety of other ways, including articles on preprint servers, data sets or software packages. In DFG proposals, a project-specific list of publications will be included in the general bibliography. The intention here is to shift the focus of the review and the evaluation of a proposal away from the list of publications and towards the substance of the applicant’s accomplishments. No information on quantitative metrics such as impact factors and h-indices are required in the CV or the proposal, and such information is not to be considered in the review. The relevant details are included in DFG forms and review instructions.

DFG’s position paper on academic publishing
The fundamental functions of academic publishing are to publicize, quality review and document academic findings and to attribute authorship and reputation. A publishing system that is appropriate to its purpose includes free choice of the form and venue of publication, the securing of exploitation rights by those publishing, and ensuring open access to the published material. In its position paper, the DFG lays out the tasks of academic communities as follows:

• Establish, use and recognize new forms of quality review for publications
• Expand the notion of addressee orientation in academic publishing
• Strengthen alternative systems of reputational attribution
• Ensure that scholarship has control over its own data

Deriving from the above, the funding agencies – including the DFG – have a responsibility to:

Broaden the spectrum of accepted publication formats
In terms of publication formats, it is crucial for a good match to be achieved between the content to be conveyed and the target group. Responsible scholarship assessment should provide support by explicitly accepting the entire spectrum of academic publications. At the same time, there should be no incentive for giving preference to certain publication formats or venues simply because they promise an advantage in terms of peer review or evaluation.

Attach greater importance to proof of achievement that is geared towards content
Responsible assessment of scholarship is based on the content of academic output. It is not derived from any standardized procedure for publishing academic findings, so it should deliberately refrain from setting any incentives to align academic activity and publication types with the assessment procedure.

Strengthen the recipient side
Readers should be able to search for and find academic publications in an appropriate way and select them according to content-based criteria. However, the large-scale commercial search systems currently in use do not come close to reflecting the publication system in its entirety. For this reason, it is important to continue to support scholarship-driven activities that seek to develop services for specialist research, make academic information available and develop the infrastructures required for this purpose.

The DFG’s positioning is based on the work done by an Executive Committee working group which looked into the subject of academic publishing and the assessment of scholarship in 2020 and 2021. Its position paper was adopted by the DFG Senate in March 2022.

To read the full position paper Academic Publishing as a Foundation and Area of Leverage for Research Assessment. Challenges and Fields of Action in both English and German, please consult the DFG website.

All information on the DFG’s Measures to Support a Shift in the Culture of Research Assessment can be found here.