EVALUATIONS OF SCIENTIFIC PRODUCTIONS: CHALLENGES AND MOTIVATIONS OF EDITORS AND REVIEWERS

Objective : This investigation was to understand the challenges and motivations of editors and reviewers in the evaluation process of productions submitted to scientific journals. Methodology : To this end, this research was conducted through a methodological approach focused on the qualitative approach, using the exploratory research method and questionnaires directed to reviewers and editors of journals in Information Science and Education, to collect data to be treated through content analysis with the establishment of categories. Results : The results show that most of the responding editors and reviewers of this research, showed to be experienced in the perception of editorial work, and that in Brazil those involved with scientific editing are volunteers, and still little recognized for academic merits. Conclusion : It is concluded that, with the complexity, relevance and the controversies around the evaluation process, which involves all the actors of the scientific production (editors, reviewers and authors), perhaps these role experiences help in the understanding of the need of a well elaborated and timely opinion, but also, should contribute so that the several actions suggested by the research subjects, were really put into practice, collaborating to the development of science and the scientific community.


de 25
open review (all those involved in the process have their identity revealed and the reviews should be published together with the article with the identities of the reviewers).It can be seen that in the areas of Information Science and Education in Brazil, there is still a predominant use of double-blind evaluation.Gradually, some journals are starting the process of open peer review, some of which leave it up to the author to choose whether or not to publish the opinion of their article, but without identifying the reviewers.
There are advantages and disadvantages to all the types of evaluation listed above, but it is not the purpose of this article to detail them, but rather to raise a reflection regarding the double-blind system, which allows evaluators to make critical analyses without fear of being rejected or being the target of some kind of 'academic revenge'; but at the same time this type of evaluation can provide a feeling of freedom for some evaluators, to write unfounded, aggressive, in-depth and biased evaluations, obviously in these cases the editor must intervene.
There is also criticism of this evaluation model, as it is possible for an experienced evaluator to discover the identity of an author, if they have been working in that field for years and have other writings on the subject, or if it is an article from a dissertation or thesis already deposited in an institutional repository.The right thing to do would be for the reviewer to refuse if he or she sees a conflict of interest in assessing the article.However, sometimes this doesn't happen, which further reinforces the need for training and preparation of evaluators, who perform a dual function in their work: evaluating scientific productions, helping the editor to choose what will or will not be published through their opinions, as well as guiding authors of submitted articles to optimize their writing through notes based on suggestions for improvements in structural, theoretical, epistemological, conceptual and methodological terms, among others.
In this way, this evaluation activity plays an important role in the communication of scientific knowledge and in the growth of science, and therefore requires recognition and proper preparation to perform this function, since, theoretically, a peer-reviewed article carries with it a certification of quality for having been analyzed by experts in the field.
In the relationship between evaluator and author, the editor acts as a mediator, receiving criticism from authors regarding superficial, incomprehensible, dubious, fragile, disrespectful, lengthy and inconsistent opinions.This situation of evaluations in this format can damage the image and credibility of the journal and the editor, by sending articles to evaluators who are potentially unprepared or perhaps uninterested in a job that is mostly voluntary, which in no way justifies the lack of interest.According to Stumpf (2008, p. 23), The criticism should help them, above all, in the preparation or writing of subsequent works that they wish to publish".However, there are reviews that do not achieve this goal, some are poorly prepared, or telegraphic, focusing, according to Gross (2020), on extensive observations directed at grammatical errors, specific terms or works not cited, when they should focus on the theoretical, conceptual, methodological, epistemological and structural aspects of the production.Cabral (2018, p. 434) also emphasizes that some opinions are considered "unfair and useless [...] but should be seen as a result of the maturing process of academia, which is certainly better today than it was 10 or 20 years ago." For this reason, the relationship between reviewers and editors can be described as delicate, because on the one hand there are dissatisfied authors who may feel wronged by an inconsistent review, and on the other hand there are volunteer reviewers with whom editors may not want to get involved.Of course, this is not to say that these situations occur in all journals.In a study carried out by Maia, Farias and Farias (2022), the authors analyzed the perception of knowledge sharing among reviewers from the point of view of scientific editors and, based on the data analyzed, observed that the latter perceive the relevance of knowledge sharing among reviewers, but have not yet put such initiatives into practice in the journals in which they work.It is therefore clear that there is still a long way to go in order to build and strengthen this delicate relationship.As Rigo (2017, p. 513) explains, sometimes we are authors and sometimes we are reviewers, because "the reviewer has their own article being reviewed by a colleague in another journal, or sometimes in the same journal".For this reason, the author points out that in order to remedy the various weaknesses of peer review, something simple and complex is needed at the same time: empathy, in order to "guarantee more legitimacy to the review system and also to reinforce attitudes such as impartiality, transparency and cordiality, capable of generating and consolidating trust within the scientific community".
Another point for reflection is certain attitudes in the peer review process when reviewers violate the rules, such as misconduct demonstrated in the form of plagiarism of the article being reviewed or indicating the reviewer's own work to the authors of the work in order to obtain citations.According to Stumpf (2006, p. 4), "the delay in evaluation can also be seen as facilitating plagiarism, because by prolonging the evaluation, evaluators can incorporate the results or methods of the work they are evaluating into their own research or publications.These attitudes are referred to by Rigo (2017, p. 513) as misconduct, which "when perceived, creates a sense of injustice in the review process and calls into question the legitimacy not only of the process but also of the journal".According to this author, there are other aspects to consider in the process of evaluating scientific productions with regard to the reviewers: In the peer review process, the reviewer becomes "powerful", as they have been assigned the task of building an opinion on the work that will guide the editor's decision-making.Their power of persuasion can either lead the editor to ruthlessly reject the work or approve it immediately.However, more often than not, reviews are constructed in such a way as to offer "salvation" to the work, leading to a more cautious decision to proceed with it to future rounds of evaluation until it reaches what the reviewer sees as suitable or ready for publication.(RIGO, 2017, p. 512).
As noted in the author's reflections, evaluators have a crucial task of deciding what they are evaluating, thus requiring not only technical skills in the act of evaluating, but also behavioral skills, acting consciously and ethically, without impregnating the opinion with personal opinions or indicating their own publications as research sources for the authors of the articles being evaluated.

METHODOLOGY
The methodological path that led to this research is based on the qualitative approach, which was used because it "does not intend to apply pre-existing concepts, and the research instruments and techniques are designed based on what the researcher feels when getting to know the subjects and the reality that surrounds them in the field of research", as Farias (2014, p. 112) explains.In terms of method, exploratory research was used, as it is understood that this type of research allows for the expansion of the knowledge to be studied and can also increase the researcher's experience with a specific problem, as well as provide an overview of a subject and/or concepts surrounding the topic in focus.
Two Google Forms questionnaires were used to collect data -one for editors1 and the other for reviewers2 .Both questionnaires were provided with an informed consent form that explained what would be done with the data collected.the Field of Education (FEPAE), and the Information Science researchers email list.The target audience for this research was editors and reviewers in the fields of Information Science and Education, chosen for convenience, as the researchers in this research had experience as editors and reviewers in these two fields of knowledge.
Before the questionnaires were made available, two pilot tests were conducted: one with an evaluator and the other with an editor, with the aim of improving this survey instrument.Both pilot test respondents did not suggest any changes to the questionnaires, stating that they were clear and objective.The responses from the pilot tests were considered in the data analysis, which was carried out using Bardin's (2009) content analysis, since, according to this author, it is a set of techniques for analyzing communication, using different tools simultaneously to achieve the objective of classifying and describing the data collected in an analytical and systematic way.

Profile of the research subjects
In order to organize and clarify the data in the light of the research objectives, categories of analysis were created called evaluators' perceptions and editors' perceptions, which are described in section 4 of this article.Before presenting the data collected and analyzed, a profile of the 41 evaluators who took part in the survey is presented.The majority of respondents (51%) work as reviewers for scientific journals in the field of Education, while 49% work for journals in Information Science.Regarding the age group of the respondents, Table 1 shows that most of the respondents (41%) are between 41 and 55 years old, followed by the age group between 58 and 68 years old (27%), then the age group between 24 and 40 years old (22%), and finally the age group between 71 and 75 years old with 10% of the evaluators subject to the survey.This data could mean that the age group between 41 and 55, which had the highest number of respondents, may also have the highest number of evaluators, as it is understood that there is an indication of stabilization in teachers' careers, but in order to make this claim, it would be necessary to have exact figures of the total number of evaluators in Brazil, in the two areas in focus.Regarding the length of experience of the research subjects as reviewers of scientific articles, Table 2 shows that the majority of respondents (73%) have been involved in this activity for more than five years.Only 5% of the evaluators are still new to this role, with one or two years' experience, while the rest have more than two and up to five years.Regarding the number of scientific journals in which the subjects work as evaluators, Table 3 shows that the majority (61%) work in more than five journals, followed by three journals (15%), two (10%), and one and four journals tied at 7%.With regard to the profile of the 40 editors who took part in the survey, the percentage of participants is as follows: 72.5% work in journals in the field of Education and 30% in Information Science; the total was more than 100%, as one editor answered that he worked in journals in these two fields of knowledge.Table 4 shows that the majority of respondents are aged between 41 and 49 (32%), followed by between 50 and 59 (30%), between 60 and 66 (15%), between 71 and 81 (13%) and between 33 and 40 (10%).As you can see, most of the respondents are between 41 and 59 years old, a similar age range to the evaluators.
Bearing in mind that in many journals of higher education institutions (HEIs) in the field of Education and Information Science, lecturers also act as editors, it can be inferred that this is a predominant age group, as lecturers must feel more secure and active in their professions in order to accept the challenge of acting as editors.The editors were also asked how long they had been working in this role for scientific journals.In Table 5, the data show that 63% of the editors have more than five years of experience, followed by a tie of two to three years and three to four years with 12% each, and four to five years with 8%, and the least experienced with only one to two years is only 5% of the respondents.In the end, it was observed that the editors and evaluators from both areas (Education and Information Science) who took part in this survey are considered experienced in their roles and can contribute to strengthening the process of evaluating scientific articles with their answers, which are analyzed in the following section.

DATA ANALYSIS AND DISCUSSION
The data collected through the questionnaires were tabulated to be analyzed in two The first category begins with information on whether or not there are initiatives by journals to reward the work of reviewers.According to the data collected, 27 respondents indicated that there is no reward system, twelve responded that there is (citing the review statement or certificate), and two responded that there is no financial reward, with A38 emphasizing this: Most journals don't have financial reward initiatives.I wouldn't do it out of financial interest either.The initiative is usually implicit and not explicit, such as being a reviewer for the journal that is best rated by the indexers.An explicit initiative I always receive is to put my name on the editorial board, send a statement of evaluation and a message of thanks.But there is an initiative I receive from a foreign journal that I think is very important for the reviewer: the editor sends me messages about the progress of the review of the article, showing how my opinion was used to improve the article.An excellent initiative!
The respondent above mentioned one reward initiative that is considered common, which is the issuance of a statement or certificate of evaluation, and another that was not mentioned by any other respondent, which would be to obtain information on the progress of the article within the editorial process.Another initiative mentioned by Gross (2020) would be a list of the best reviewers of the year, with the aim of valuing the efforts of reviewers to maintain the quality of scientific productions.Some other measures to reward and motivate reviewers are mentioned in the subcategories below.
In the same questionnaire, reviewers were also asked if they knew of any measures taken by the journals for which they work to "attract" new reviewers and to motivate those already on the editorial board to continue reviewing.The majority (80.5%) of the reviewers did not know of any type of action, while 19.5% said they knew of these actions, which were detailed in the open-ended responses, which were analyzed in the same category of reviewers' perceptions, divided into subcategories, namely: a) feeling rewarded and motivated to review; b) analyzing the relationship between editors and reviewers; c) biggest challenges in the reviewing process; and d) actions that could be implemented to reward/winning/motivate reviewers.
The first subcategory begins with whether there is a reward and motivation to evaluate.There are several reasons why evaluators feel motivated and rewarded, listed below, which were similar among some of the respondents, some of whom also identified themselves as editors and were aware of the difficulties in obtaining evaluations.The motivations cited by the evaluators were: contributing to the development of the field; contributing to the dissemination of knowledge; exchanging experiences; helping journals; commitment to the collective; guaranteeing the quality of the field's content; getting to know the production of the research field; getting to know how issues are dealt with from different perspectives; solidarity with editors; and reducing the waiting time for authors and editors.
Table 1 shows some of the responses to illustrate the evaluators' thoughts on this subcategory.
Chart 1. Impressions of reward and motivations for evaluation.
It's important to give this feedback to the authors.It's a commitment of those in academic careers to collaborate with research.If everyone did a bit, there wouldn't be so many problems and wear and tear.In addition, the number of evaluations could be reduced.(A15) Yes, because it's something I enjoy doing, it's a pleasure to contribute to scientific communication.But if there were some form of reward, it would certainly increase my motivation even more.(A13) The motivation is intellectual, in the learning that an evaluation offers the evaluator and the possibility of internalizing this learning in their research practice, in addition to the possible possibility of projection at curriculum level.(A19) From the point of view of scientific academic altruism, promoting science and encouraging researchers, yes.But I feel that other motivators are lacking (more traffic to publish in journals, slow response from some of them, bureaucracy that prevents progress, as well as the accumulation of requests over several periods).(A22) Yes.I understand that it's a partnership, because as an author myself, I want my articles to be carefully evaluated.The basis of science is peer review.(A26) The invitation to an evaluation of this nature denotes acceptance of your expertise in the subjects and of the titles you hold as recognition of your academic and scientific importance to the areas for which you collaborate.(A29).As can be seen in chart 1, the evaluators' motivations are diverse, ranging from their commitment to science, to scientific communication, to the possibility of learning, to the return of their article, which is also submitted, and is analyzed in the same careful way as the responding evaluator, but there are those who deal with the lack of remuneration and the accumulation of duties of the teacher, who is also an evaluator.For this reason, "scientific academic altruism" is perceived, as A22 reports, in the work of the evaluators in the face of all the difficulties, such as the "lack of other motivators (more traffic to publish in journals, slow response from some of them, bureaucracies that prevent progress, in addition to the accumulation of requests in various periods)".According to Gross (2020, p. 6), one of the possibilities to encourage the "preparation of good reports would be precisely the valorization of this activity by Capes in the evaluation of postgraduate programs", but obviously, other long-term planned actions would be needed to encourage more motivated and engaged evaluators.
The survey data also shows that some respondents, such as A2, A11, A14 and A24, were categorical in their response, writing only that they "don't" feel rewarded or motivated."No.I only get points on my CV and when I receive an evaluation statement" (A31); "I don't feel motivated, just a 'moral' obligation" (A35).As observed, some evaluators do not feel motivated, but continue to evaluate articles because they count points on their CV or because they feel they need to contribute to the communication of science, which is corroborated by Stumpf (2008, p. 23), when he states that evaluation "does not bring public recognition, much less remuneration for evaluators.But many scientists see the task as part of their obligation to science".
With regard to the subcategory analyzing the relationship between editors and evaluators, the majority of survey respondents say it is good, respectful, cordial, calm and appropriate, built on trust, as can be seen in the responses in Chart 2.
Chart 2. Relationship between editors and evaluators I find it efficient in most cases.In many situations it's even possible to contact them outside the platform (whats, email) to speed up the revision process.And even when communicating via the platform, it works adequately, usually with a response within a fairly quick timeframe.(A1) I think that the relationship between editors and reviewers depends on several factors, for example: the prestige of the journal, the resources available to reward reviewers, the length of the reviewers' careers.Chart 2 shows a positive overview of the relationship between reviewers and editors from the perspective of the reviewers who participated in this research.The view of these subjects is that it is a collaborative, positive relationship, or even impersonal, as A5 states: "It only changes when you have bonds of friendship", which A25 also emphasizes by stating that "reviewers are 'captured' by proximity and personal affinity".A2 emphasizes that factors such as the prestige of the journal and the length of the reviewers' careers can influence the relationship between editors and reviewers, which leads to the perception of differences in thinking between the respondents, as it depends on the experience and length of experience of each.In contrast, nine of the 41 respondents considered the relationship between reviewers and editors to be: distant, reasonable, cold, non-existent, weak, and lacking dialogue.
According to E11, there is no relationship, and if there is one, it is distant.This could be due to various factors, such as the accumulation of tasks for editors who are professors and researchers, and who may not have a reduced workload to devote to the position of editor, causing, for example, as Cabral (2018, p. 436) explains, "neglect of their own research agenda, a reduction in the quality of their teaching, or even a reduction in the time they devote to their loved ones."No attempt is made to justify the lack of dialogue reported by the evaluators, only to point out possible causes.
Regarding the subcategory "biggest challenges in the evaluation process", 15 respondents said that the biggest problem is having the time to develop a careful analysis and a consistent opinion of the articles, because they combine several teaching activities with the role of evaluator, as A13 points out, who acts in other roles in the journals and feels pressure from both the evaluators and the editors.In addition, some reviewers said that it was difficult to review texts with grammatical errors, no content, and poor writing that took time to understand.Other responses are listed in Chart 3, which shows the evaluators' reflections on the challenges they experience in this task.

Chart 3 -Challenges in the evaluation process
In some situations where the articles are based on statistical and even empirical data, I believe that incorporating the scientific data that underpins the research has increased the demands on the evaluators, since they need to check that the results are correctly represented.In this case, it requires more time and competence in the subject being assessed.(A1) The constant updating of the themes made available.(A3) Not using Publons and/or ReviewerCredits accreditors (A6) Decide whether or not the article should be rejected or if it can be redone and resubmitted.Another precaution is to know how to choose your words so as not to discourage or attack the author.Knowing how to make suggestions within the theoretical and methodological framework chosen by the author.This requires empathy and study on the part of the referee.(A8) Finding the courage to evaluate, to give a quick and quality response, knowing that our own submissions take months to be evaluated.(A11) It takes a long time to send the evaluation voucher.It's not uncommon for me to have to ask, and that's very bad." (A12) I'm speaking as an editor -it's a very disparate quality of evaluation.Sometimes they are opposite, which means more commitment, time and delay.Another problem is finding evaluators who agree to take part.(A16) Aiming to evaluate without belittling the evaluated, paying attention to the characteristics of the submitted manuscript and, at the same time, looking for ways to contribute to its improvement.(A19) Being fair and understanding that sometimes flaws can be pointed out, fixed and the writing still has merit for dealing with the subject in a way that contributes to the area of knowledge to which it was submitted.(A20) There are still many evaluators who don't have the maturity to give complete opinions that really state the reason for acceptance or rejection.(A27) At the moment, the biggest challenge is self-plagiarism.There are submissions that have aspects of "rehashed" work, which forces me -as a reviewer -to read work by that author in order to copyedit that work with what the author has submitted.(A29).
Evaluating an article presents the same difficulties for the evaluator as those involved in evaluating any academic-scientific work.Demanding linguistic adequacy, scientific rigor, originality and obedience to the technical standards of reference records and formatting is becoming almost impossible in the editorial space of Brazilian education.Evaluating work that fails to meet these requirements is always very difficult.(A30) The illegibility of texts with problems of the author's mental organization; problems of plagiarism; unfinished research that presents poorly-written results in order to say that it has been completed; texts from areas of knowledge that are not in the evaluator's domain.(A38) Lack of normative standardization in line with ABNT.There are different standards for each journal.(A40) Source: survey data (2021).
Chart 3 gives a glimpse of the day-to-day life experienced by evaluators with a wide range of challenges, such as self-plagiarism, illegibility of the text, lack of originality, scientific rigor, the delay on the part of editors in sending proof of the evaluation, as A12 states "It's not uncommon to have to ask, this is very bad", among other problems, as A38 explains "plagiarism; unfinished research that presents poorly written results to say it has been completed; texts from areas of knowledge that are not in the evaluator's domain."Even so, there is a concern to be careful when issuing an opinion, to be fair, so as not to discourage the author, as A8 points out, when explaining that it is necessary to "know how to make suggestions within the theoretical and methodological framework chosen by the author.This requires empathy and study on the part of the referee."Gross (2020) emphasizes this when he points out that having well-prepared opinions allows for the promotion of quality and consistency in the productions analyzed, promoting the growth of the actors involved in this process, i.e. evaluators and authors.
In the subcategory of actions that could be implemented to reward/conquer/motivate reviewers, the respondents suggested the following: sending statements/certificates to evaluators quickly or automatically through the system; opinions should be scored in postgraduate programs, in competitions, project submissions to calls for proposals from funding agencies, in internal evaluations carried out by universities (career progression); sending articles in the evaluator's area of interest; institutions should value and consider evaluations as an academic activity; adoption of tools such as Publons 3 and ReviewerCredits 4 for evaluator registration and recognition; invite the evaluator to 3 Publons is a free resource for the global multidisciplinary academic research community, powered by Clarivate Analytics.Publons links directly to the Web of Science (WoS), allowing researchers to claim and manage their publication history in WoS and display their history alongside their peer review and journal editing history (CLARIVATE, 2023). 4ReviewerCredits is a system for obtaining academic recognition and rewarding peer review efforts.ReviewerCredits allows you to redeem credits for publication discounts, editorial and translation services, subscriptions, training courses, conference attendance and much more (REVIEWERCREDITS, 2023).
Encontros Bibli, Florianópolis, v. 28, 2023: e92858 Universidade Federal de Santa Catarina.ISSN 1518-2924.DOI:https://doi.org/10.5007/1518-2924.2023.e92858/5402315 de 25 publish in the journal in which they work, in a special issue or thematic dossier, for example; have some kind of tribute on the journal's website to the most active evaluators, publicizing their work and curriculum; try to train postgraduate students as future evaluators; keep the option of registering new evaluators on the journal's website; promote courses related to the activity; increase the deadlines for accepting and submitting evaluations; evaluators could have preference in publishing, and in proposing thematic dossiers; remunerate evaluators.
Chart 4 shows more actions that editors can use, as well as those already mentioned, to implement in the journals in which they work.
Chart 4. Actions to reward/conquer/motivate evaluators I feel there is a lack of natural incorporation as a reviewer into the Scientific Committee of journals after a considerable number of reviews.It is notorious that some journals add evaluators due to the representation of the curriculum, seeking to bring greater credibility to the journal.This model is recurrent in journals seeking internationalization and is repeated even among the names of researchers at a national level.This practice seems understandable and even necessary to me, but I think it's important for journals to adopt a balance between the quantity and quality of reviews, in order to incorporate new members of the scientific committees, i.e. regular reviewers who make a significant contribution.I don't know if any journals already do this, but I think it would be a satisfactory way of rewarding reviewers.(A1) No.Most Brazilian journals are not professionalized, so volunteering not only increases the risk of thematic or theoretical conflicts of interest on the part of editors, it can offer nothing to technical work that could not be volunteered.What's more, Sucupira doesn't consider an opinion to be noteworthy.Technical work is worth almost nothing in terms of teaching output.(A8) I'm in favor of training in evaluation, whether through Anped, Coneped, courses (A25) Publication of a newsletter at the end of each editorial year with a body of evaluators; publication of the average evaluation time at the end of the cycle.Obviously, always preserving the impersonality of the journal in terms of the evaluation process.That's the basics (A31) In addition to listing the evaluators who collaborate the most, evaluate with quality and meet deadlines, perhaps highlighting the 3 most prominent evaluators in this regard in the Official Report.(A33) Sending messages to the evaluators to follow up the process of accepting or not accepting articles, so that they know what has been used and what has not; tutorials on evaluation style; good evaluation practices for the journal.(A38) Source: research data (2021/2022).
As noted in the A8 reflections, there are a number of issues related to the evaluation process that require the attention of editors, journal editorial boards, scientific publication promotion agencies, undergraduate and postgraduate regulatory bodies, the colleges and deans of the courses with which the journals are associated, editorial boards, journal portals, etc., with the aim of motivating and rewarding the work of reviewers, as well as of the editors themselves, who often carry out this activity on a voluntary basis, as seen in the next category, which deals with the editors' perception.
In this category, the editors are referred to as E1 to E40, and the questions have been The first subcategory aimed to find out whether the journal where the researchers (editors) work has initiatives to reward reviewers.The data shows that 77% of the respondents answered that there are initiatives and 23% answered that there aren't any.
The one most often mentioned by the editors is the issuance of a certificate/declaration of evaluation.According to E9, "Issuing a statement is the least an editor should do.I don't even consider it a reward for the reviewer.In the case of E33's journal, a list of reviewers who have contributed is published in the issues, "regardless of the editorial decision (but with the process completed during the period)".E12 says that the journal he works for uses ReviewerCredits as a reward, while E18 is paid for reviewing the journal he edits.
Looking at the responses of the reviewers to the same question of whether or not there are reward initiatives, where 27 reviewers (the majority) said that there is no reward system, and twelve said that there is (the review statement), it can be seen that there is no consensus between reviewers and editors on what rewards would be in this process, since the majority of editors responded positively to the same question.Of course, the respondents to this survey may not work for the same journals, i.e. the reviewers and editors who answered the questionnaire are from different institutions, but this can be considered a small sample of the overall picture.
With regard to the subcategory recurring problems in relation to evaluations, the editors point to the delay in evaluation times (46%) as the biggest challenge, followed by inconsistencies in opinions (28%), rudeness (9%) and conflicts of interest (3%), as can be seen in Table 6.The other problems cited by the editors were: evaluators not answering in the system whether or not they agree to carry out the evaluation; insufficient, poor quality, summarized, poorly written and poorly prepared opinions; complaints about the evaluation received; difficulty in finding evaluators for all themes/approaches; evaluators who poorly evaluate articles with opposing theoretical tendencies; evaluators who do not follow up the evaluation in all the rounds, requiring constant replacement of evaluators and delaying the process.Analyzing Table 6 with the problems and percentages presented, it can be seen that the time taken for evaluations and inconsistencies in opinions are the biggest challenges in the evaluation process.According to Stumpf (2006, p. 4), a quality opinion takes time, and the evaluators, who are usually teachers and have a variety of activities, "undertake to carry out the task in good time so as not to interfere with the editor's schedule", but they may not be able to do so within the stipulated timeframe.Another fact warned by the author is that the "delay in evaluation can also be seen as facilitating plagiarism, because by prolonging the evaluation, evaluators can incorporate the results or methods of the work they evaluate into their research or publications", and furthermore, even with the delay in the time of evaluations and the possibility of plagiarism, there are opinions that may not contribute to the improvement of the article, and even contain an aggressive/disrespectful content.

Encontros
Still on the subject of time, there is the subcategory time to receive evaluations.
The most common timeframe for receiving evaluations was one month (43%), followed by other timeframes (39%), detailed below, and then three weeks (4%), with two and four weeks tied at 7% each.Other deadlines cited were two to three months, one semester, and other editors said that it depends on the situation.According to E35, the time it takes to evaluate articles also depends on whether there are technical problems, or, for example, if there are ongoing events in the area: "In general, it is possible to receive an evaluator's assessment within a week, but the minimum period is one month, which can always be changed according to need."As noted, this deadline will vary depending on issues involving the evaluation process itself, technical aspects and even the professional demands of the evaluators.However, Gross (2020, p. 7) warns that the "problems observed are not restricted to the limited availability of evaluators or the limited time and attention dedicated by some evaluators to the opinions".There are evaluations that focus on grammatical aspects and/or standardization, when they should make real contributions to the advancement of the scientific knowledge in question, dealing, for example, with the adequacy of research methods and techniques and conceptual problems.
In relation to the subcategory that reflects on whether there are actions in the journal to select new evaluators and to keep those already on the editorial board motivated to evaluate, 80% of the editors answered yes, and 20% no.Among the most cited actions are: giving more time to evaluate the manuscript (40%); sending a direct message to the evaluator convincing them to stay on the list of evaluators (32%); contacting the evaluator via WhatsApp (11%), as shown in Table 8.Having more time to evaluate scientific productions is also a demand from evaluators who carry out other activities such as teaching, research and administrative issues at the universities where they work.The indicators listed above were used to organize the actions suggested by the research subjects (editors) with a view to selecting new evaluators, as well as keeping those already on the editorial board motivated to continue evaluating.As can be seen in Chart 6, the indicator 'Adoption of institutional policy' includes actions aimed at universities, research centers, etc. where journals are located.This indicator suggests that tools should be created to encourage researchers to become evaluators, and those who already are should be motivated to continue this work, as E39 points out when he explains that there should be "information exchange between editors in the sense of sharing information.For example: a list of reviewers with their contacts, etc.I think the IBICT could promote this integration"; or according to E18's reflections when he suggests strengthening "symbolic remuneration for evaluation"; and also as E17 points out when he ponders that "perhaps it could be better scored in evaluations such as RAAD and in Lattes itself to be considered a relevant task that contributes to the program's score, if it is a professor linked to a postgraduate program."Chart 6. Actions to select new evaluators and to keep those already on the editorial board motivated to evaluate

Suggested actions Indicators
Motivation policies for evaluators to be adopted by publishers.(E3) It would be important if these participations were taken into account in the institutions where the researchers work.(E4) In the case of private HEIs, organize institutional publishing labels and charge the price of books to be published administered by the HEI treasury so that evaluations are remunerated.It's a very complex issue because we generally rely on the kindness of professionals for this task.It is known that article evaluation is more of a demand and there is no remuneration.Perhaps it could be scored more highly in assessments such as RAAD and in the lattes themselves, being considered a relevant task that contributes to the program's score, if it is a professor linked to a postgraduate program.(E17) Adoption of institutional policy Strengthen symbolic remuneration for evaluation (E18) It was discussed at the last CAPES Mid-Term Seminar (2019) that CAPES area coordinators should sensitize colleagues to contribute to the area's sister journals.I think this should be reinforced."(E26)

Policies for valuing journals, editors and evaluators (E28)
There should be an exchange of information between editors in the sense of sharing information.For example: a list of reviewers with their contacts, etc.I think IBICT could promote this integration.(E39) National base of evaluators (E8)

Creation of an accreditation system for assessors
Searching databases, searching for themes-researchers, requesting nominations for evaluators from the organizers of thematic sections, from well-known researchers.(E8) Every year, we turn to the list of the Association for Research and Postgraduate Studies in Information Science -ANCIB -for new contacts and to colleagues in the area of Communication and Information, through e-mails, with a view to inviting colleagues to be referees for the journal.(E26) I've thought about structuring a formal report for them to fill in so that they have more evaluation criteria that aren't based on their practices alone.We're still evaluating it with the team."(E31)

Editorial policy
I do almost all of the above, but I don't think it's innovative, I think it's part of the editor's role to have more direct contact with the reviewer.Thanking them for their opinions, giving them clear guidelines on how to prepare the opinion, passing on a statement.Unofficially, we give a slight precedence in the evaluation to articles by reviewers who are always collaborating.Precedence in evaluation, but a small one, due to the demands of the journal, and in publication in aheadof print AOP.
When it comes to publication in AOP, i.e. after it has been evaluated and approved for publication, I always ask when the author, who is also an evaluator, needs the text to be published in SciELO.(E9) The Journal's Editorial Board always invites new evaluators and adopts the Ad-Hoc Evaluator.(E27) For new reviewers, we make personalized invitations to authors who have published with us and are not yet part of the reviewer team.There are also contacts made at events.(E34) The Editorial Board is periodically re-evaluated and only people who actively contribute are retained.Unfortunately, this does little to motivate reviewers.(E35) Evaluation training for new doctors; mini-courses at scientific events for evaluation training.(E23) We commend the work of the reviewer (who has already contributed his work to the Journal) and say that the opinion of such a reviewer is very important for improving the manuscript and for science itself.(E5) categories: The first, reviewers' perceptions, aims to understand, from the point of view of the research subjects, what initiatives they are aware of aimed at rewarding and motivating reviewers, how they perceive the relationship between editors and reviewers, what the main challenges are in the review process, and whether they know of measures to "win over" new reviewers and to keep those already on the editorial board; As for the second category, perceptions of editors, it aims to understand, from the point of view of the editors of scientific journals, what initiatives the journal where they work has taken to reward reviewers, as well as what are the recurring problems related to reviews, what is the average time to receive reviews, and what actions the journal would take to select new reviewers and retain those already working in the journal.The respondents were named from A1 to A41, i.e. reviewer 1, reviewer 2, and so on.
Yes and no.Yes, because it's a way of entering the educational field.No, because this is just another unpaid job that teachers do.(A30) I always feel motivated to evaluate and collaborate with scientific communication and the journal.In addition, I value the work of the reviewer, as I am a journal editor, and I know how much we need reviewers to boost the editorial process.(A33)First of all, it's a way of finding out about the various research projects underway in my area of research, although at no time have, I used this knowledge for my own benefit.But it is a way of keeping up to date.I like evaluating because it's something I can do to improve publications and teach authors how to improve their writing and organization of results.(A38) Source: research data (2021/2022).
Others justified not being motivated or rewarded: "Not much.It depends on whether the topic is interesting for my research or that of my students" (A8); "I don't feel rewarded.But what motivates me is a certain amount of recognition for the subject of the articles" (A9); "No, I only do it to help colleagues because I used to be an editor and it's a difficult task" (A28); 12 de 25 general it's impersonal.It only changes when you have friendships.(A5) Unprofessionally in general.Contacts are by auto-response, the better qualified the magazine, the less tolerance the editor has for delays.In my experience, therefore, I would ask you not to generalize.(A8) I see them as two partners, each with their own role, and they need to work together to ensure the high quality of the journal's work.(A13) I have a good professional relationship with the editors of the journals I work for.(A19) Nowadays, evaluators are "captured" by proximity and personal affinity.(A25) Relax, the evaluator has a certain amount of decision-making power, that's interesting!"(A28)As an evaluator, I receive the invitation, followed by a thank you.Nothing beyond those formalities."(A29) I think the relationship is collaborative and also requires understanding.(A33)Positively.The editors depend on the reviewers and they know it.We're partners, but I think the editors in general should give the evaluators feedback on what happened with the articles.(A38) Source: research data (2021/2022).
divided into the following subcategories: a) initiatives to reward reviewers; b) recurring Encontros Bibli, Florianópolis, v. 28, 2023: e92858 Universidade Federal de Santa Catarina.ISSN 1518-2924.DOI:https://doi.org/10.5007/1518-2924.2023.e92858/5402316 de 25 problems related to reviews; c) time taken to receive reviews; d) measures in the journal to select new reviewers and to keep those already on the editorial board motivated to review.

Table 2 .
Length of time working as a scientific article reviewer

Table 3 .
Number of journals the evaluators work for

Table 8 .
Actions in the journal to select new reviewers