1 Introduction

European study about informatics programs in the high schools (Eurodyce, 2022) has found that this topic is still considered as a new discipline. But the major surprise from the study outcomes was that the cybersecurity topics are not included in the curricula for informatics or computer science subjects. The same study revealed that the teachers of informatics topics have never studied cybersecurity topics as a separate discipline during their education, or as a part of the computer-science subject, although the importance of cybersecurity for society’s digitalization and sustainable development is widely recognized elsewhere as being very important. A way to build an educational ecosystem with relevant topics like cybersecurity that reaches target audiences is not yet included as mandatory content in the current European actions for building cybersecurity skills among the Member States’ citizens. Cybersecurity subject is only offered within the higher-education institutions (HEIs) or in courses offered by the professional education market (Dragoni et al., 2021).

Today’s young population is changing its internet habits very quickly, in response to considerable societal, market and technological innovation. They are frequently engaged with online videos, music, gaming, messaging. In general, their internet use is broadly positive, contributing to the building of digital skills. When enjoying the benefits of the internet, it is important to be aware of the potential risks and to apply safety precautions as young people now have internet access at a very early age. Studies claim that cybersecurity education should be applied in schools, because there are many benefits if a school is able to offer cybersecurity education (ENISA, 2021) since the adult participants of seminars organized on the market are reluctant to spend more time and money for learning. Schools should provide programmes with cybersecurity topics to the larger community as cybersecurity education is changing the minds of individuals and is contributing to a safer digital environment. According to Amankwa (2021) schools in the USA are provided with grants and financial allocations from the federal government to ensure support for implementing cybersecurity education. In addition, camps for high-school students dedicated to cybersecurity topics are regularly organized each year (Chen & He, 2021). One of the examples is the GenCyber (2015) summer camp organized for teachers and students within an NSA/NSF cybersecurity support programme. Important study on the issue, based on a systematic literature review of more than 20 very relevant papers reports that a well-designed curriculum in cybersecurity education at the K-12 level could successfully motivate students at this level to pursue cybersecurity pathways and careers ln the future, but today such a curriculum is still missing (Jerman Blažič & Jerman Blažič, 2022).

Children start using the internet at an early age, and more than 90% of young Europeans are online every day. The use of the internet by teenagers brings many benefits linked to information and communication, but it comes with risks too, including privacy violation, identity theft, ransomware, and the fraudulent use of debit cards, etc. It is, therefore, of paramount importance that school education includes cybersecurity topics that will help these new generations to acquire skills that make them aware of major on-line threats and encourage them to adopt appropriate individual and collective behaviour to reduce the risks they are exposed to. The major issue when introducing cybersecurity topics at the school level is the teachers’ education, which should ensure the transfer of new knowledge to their students and build the necessary skills with modern delivery methods enriched with interactive tools like educational games. The identified gap in skilled cybersecurity professionals all over the world is also due to the lack of appropriate education at the high-school level.

Recommendations on an institutional level how to address cybersecurity topics or curricula in the educational programs of European high schools are missing, as was proven by a survey involving high-school management in 14 European countries in 2021 (Caulkins et al., 2022). In that context, digital skills, including the cybersecurity skills built through cybersecurity programs are understood to represent a combination of abilities, knowledge and experience that enables an individual to successfully complete a task when working in a digital environment and using digital services and being safe from attacks and threats (Catal et al., 2023). The lack of sufficient attention given to acquiring cybersecurity knowledge by European children and young people has a negative impact on the capacity of a modern, digitized society capable of efficiently reacting to the increasing number of cybercrimes (Adams and Makramalia, 2017). Cybersecurity skills are becoming especially important for economic development, as most experts claim that the digital economy’s winners and losers will be determined by who has these skills (Jerman Blažič, 2022). One of the papers addressing this issue found that a lack of cybersecurity education is a global problem, and that it could take up to 20 years to address the existing skills gap at all levels of education (Leune & Petrilli, 2017). The key to produce more cybersecurity graduates for reducing the labour shortage is in establishing an educational pathway in the educational process with cybersecurity programs. The study of Jin et al. (2018) pointed out that students receiving computer education in high schools are eight times more likely to major in a computer degree and become a cybersecurity professional.

The problem addressed in this paper is twofold: a lack of appropriate educational content for the EU’s high-school students and a lack of teachers with the knowledge and an understanding of cybersecurity topics, aligned content and skills. The paper addresses both problems by presenting the teaching materials and the delivery methods developed in line with the modern pedagogical approach. The demands for new learning methods are now more pressing than ever, especially regarding the need to introduce and upgrade the educational methodology at the high-school level. Several studies have found (Angafor et al., 2022) that even the education and training at the universities in several areas, including ICT, dedicated to prepare the high-school teachers are still very technocratic, and the educational approach is not promoted in line with the requirements for sustainable education and modern teaching (Burns et al., 2023; Cybersec4Europe, 2022).

The major contribution of this paper is: (a) the definition of the educational content for the high-school teaching in two areas: cybersafety and cybersecurity accompanied by available sources composed from video platforms, quizzes, YouTube presentations, educational games and hands-on sources for practical exercises, (b) demonstration of the applicability of the developed educational model and content for both audiences, the high-school teachers and students from nine European schools during the 2022–2023 school year and (c) an evaluation of the learning outcomes through interviews, surveys and by testing the students’ knowledge with the Capture the Flag game contest.

The paper is organized as follows. Section 2 provides a short overview of related previous work on the subject and by summarizing the results of an on-line survey among the European high-school students and teachers about the missing content in cybersecurity education, the desired educational methods for material delivery, and the methods supporting the class preparation. Section 3 describes the developed methodology and the curriculum design. Section 4 presents the implementation of the educational and the achieved outcomes. The organisation of the contest of Capture the Flag and the results are briefly described. The Sect. 5 discusses the approach applied and the experiences collected. Some recommendations and concluding remarks are provided in the conclusion section.

2 Related works

2.1 Identifying the problem and the suggested findings

To clarify the subject and the understanding the meaning of the both fields: Cybersafety and Cybersecurity the following definitions are provided: Cybersafety is understood as the means of protecting users from harmful online content. It concerns the emotional and psychological impact of what is seen, read and heard online. Being cyber safe means meeting appropriate standards of behaviour in the content the users put on the internet, knowing how to avoid harmful interactions online, and being equipped to seek help in the case that an unexpected result happens. Examples of cybersafety incidents are: cyberbullying, sexual harassment, exposure to hate speech/ inappropriate content (violent, sexual, etc.), leakage of sensitive information, personal abuse, etc. Cybersecurity is understood as the means of protecting data, systems and assets on the internet. It refers to the physical and software operation of the networks and computers over which the internet is delivered. Cybersecurity is described as processes, states, or activities in which communication systems and information are protected from modification, unauthorized access or exploitation (ENISA, 2021).

Examples of cybersecurity incidents

entry of viruses into the system, denial of service by the connected systems, hacked and abused accounts/profiles, man-in-the-middle attack (an attacker gets access to the communication channel and can see and modify the information exchange from the system in use), misuse of the social engineering (where a person tries to acquire the user or company credentials by asking simple questions), performing phishing (getting access to a user’s or company’s credentials in a financial institution and stealing money), encrypting data and requesting payment to unencrypt data and other important information, ransomware.

The first cybersecurity-strategy policy in the EU was adopted in 2013 and was later discussed and expanded by European experts, but education in terms of cybersecurity skills was mentioned very briefly. However, in December 2019, the development of cyber skills in the EU described by the ENISA report (ENISA, 2021) pointed to the growing lack of such skills among the general population and stressed the lack of cybersecurity professionals in most of the EU Member States. More recent works on the subject were published by Ackerman (2019), Ruiz (2019), Conklin et al., 2014), as well by Catota and collaborators (Catota et al., 2019).

The issue received additional attention when the problem of a labour force lacking cybersecurity skills was detected almost everywhere. This was estimated to be about 3 million workers globally, according to cybersecurity workforce studies carried in 2018 and 2019 (Jerman Blažič, 2022). High schools were not part of these studies and proposed programmes as the focus was given to the higher level educational institutions. Cybersecurity encompasses a broad range of specialty areas and working roles, and this is the reason why a no single educational programme can cover all the sector-specific knowledge wanted by each audience in the digital society. The subject topics should be selected according to the audiences in different sectors. However, most of the experts agree that there are certain knowledge sets and skills that are essential for most of the population in an individual’s everyday life, in participating in the process of studies or in performing a critical working role they need to adopt (Malan et al., 2018; Karagiannis & Magkos, 2021). Considering the broad range of specialty areas in the field, it is not surprising that cybersecurity education has been addressed differently by various countries that build cybersecurity strategies with different focuses. The educational part of these strategies is mostly formulated as strategies for improving the general state of cybersecurity skills in the country, which also includes the updating in the educational system, but the problem of high-school level education is still not sufficiently addressed. Several researchers that worked in the area have stressed the need for an improved educational approach by adding more technical tools and accessories like quizzes and games in the teaching. Coentrad et al. (2020) investigated the value of the digital-game-based learning in cybersecurity education. They stressed the cybersecurity game’s potential in enabling students to learn within simulated reality and test their skills at solving cybersecurity problems. Evidence from the high schools studies clearly suggests that the cognition among students is much greater if they are given the opportunity to actually engage in the classroom activities by supporting the development of critical thinking and problem solving, which is enabled by gamification (Jin et al., 2018; Karagiannis & Magkos, 2021). Gamification has been introduced in the last two decades as a method to motivate students to use a variety of techniques found in popular games and they were added to the educational modules. Training solutions including games in the area of cybersecurity have proven their capacity to raise awareness, defence and affective strategies based on an exhaustive analysis of more than 100 cybersecurity games available on the market (Coentrad et al., 2020, Jerman Blažič, 2021).

2.2 Cybersecurity competence centre contribution in designing the EU cybersecurity educational

These findings triggered the members of the Concordia Competence Centre (Concordia, 2021) to dedicate part of the project work to remedy the situation of missing skills among the EU population by preparing a modular cybersecurity body of knowledge, firstly for the workers in different industry sectors, composed from number of courses offered to the where the interested industrial organization to shape an appropriate education process for their employees according to the needs (. They were enabled to select modules from the offered set and to provide an on-line learning to their employees. Later the same Concordia team working on the cybersecurity curriculum’s development started to explore the situation in the cybersecurity education of the young EU population. For that purpose the team launched a study and large survey in the highs schools of the EU countries to identify the Key Areas (KAs) in cybersecurity that could become part of the EU high-schools’ regular education. The collection of KAs was launched recently by the UK National Cybersecurity Centre but they were dedicated to training for postgraduate degrees in higher level education accompanied by an initiative addressing the cybersecurity skills (Attwood & Williams, 2023). The knowledge base of KAs is located at the CyBok platform where the KAs in all the relevant topics of cybersecurity are identified with some additions covering the aligned areas from the Human, Organizational and Regulatory Aspects of cybersecurity. The base is licensed under the Open Government License v3.0. However, the KAs for the high school education level are not yet provided by the CyBok database (CyBok, 2023).

The intention of the Concordia team to develop a cybersecurity curriculum and training programme with modern delivery methods and principles in a high-school environment was faced with a lack of information about the current state in the EU high schools regarding cybersecurity education and how much the students know about cybersecurity when they are online. A few research questions required answers before the work dedicated to develop an appropriate curriculum could start. They were as follows. RQ1: To what extent are cybersecurity topics included in the EU’s high-school education programmes? RQ2: What are the missing topics in school programmes that provide information for enhancing the cybersecurity skills and ensure safety when online? RQ3: How much do high-school students know about cybersecurity and self-protection when using the internet? RQ4: What kind of educational delivery methods are most desired by high-school students? RQ5: What are the most relevant methods for testing the appropriateness of the developed curriculum and the teaching methodology.

The answers to the research questions were expected to be provided by a survey launched in 2021 all over the EU. A large pool of stakeholders, namely teachers, students and their parents, and the schools’ managers were addressed. Answers to the research questions were accompanied by the collected information from the interviews and the talks with students’ parents, teachers and school managers. The survey was prepared on the EU Survey platform in English and launched online with questions translated into several official EU languages, such as German, Spanish, French, Italian and Greek. The invitation to take part in the survey was disseminated on social media and promoted in specialized high-school networks. 100% of the participants provided information about their country of origin, 97.6% of them declared that they are based in an EU country. 100% of the participants also provided information about which group they belong to: high-school teacher, student, school manager or parent. All questions in the survey were answered by the participants by selecting one of the possible answers. The participants in the survey were anonymous.

A total of 366 participants provided answers by the end of November 2021, more than half of which were high-school students (63% of the sample, the rest was provided by the participating teachers and parents). Nine EU Member States provided a considerable number of answers from several involved schools in the survey to all the survey’s questions, from the approached audiences, the students, the teachers and the parents. These countries were Romania, Slovenia, Greece, Cyprus, Italy, Spain, Germany, Netherlands, and Poland. The teachers and the school managers were invited to the interviews for collecting their views about the participation in the education content application in participating schools. The number of answers received from the other five countries’ high schools were not numerous and were not included in the evaluation of the survey data. The demographic data revealed that almost equal participation of female and male students were took part in the survey. The addressed audiences were asked to provide the following information (Concordia report survey, 2021). The following information were requested from the survey participants:

  • Demographic data (anonymized, the country).

  • Digital services used by high-school students.

  • Digital services used by high-school students in the school environment.

  • Devices used by high-school students in general.

  • Devices used by high-school students in the school environment.

  • Degree of confidence of high-school students during specific online activities.

  • Degree of awareness of high-school students regarding online risks.

  • Incidents experienced by high-school students related to the online risks.

  • Possible subjects that could be presented and discussed within a relevant cybersecurity course for high-school students.

  • Type of methods/instruments to be used while teaching cybersecurity at the high-school level and Cybersecurity subjects in already-existing courses.

The answers of the participants about if cybersecurity topics are taught in their school (RQ1) are summarized on Fig. 1.

Fig. 1
figure 1

Cumulative responses to the questions if cybersecurity topics are taught in the student`s school

Full size image

The answers to RQ2 about the topics students would like to be presented and discussed are provided on Table 1.

Table 1 Cybersecurity topics to be introduced in the educational programmes
Full size table

The answers to question RQ3 about the feeling how much confident the students when are online are the presented in Fig. 2.

Fig. 2
figure 2

Collected answers to the question how much the students are confident when they use online activities

Full size image

The students’ answers to question QR 4 asking about their preferred method of content delivery are provided in Table 2.

Table 2 Preferred delivery methods for teaching cybersecurity
Full size table

The answer to RQ 5 was decided to be provided by the Concordia team within the envisaged cybersecurity and cybersafety educational model. The teachers that took part in the interviews during the survey pointed out to their limited knowledge about the cybersecurity topics and the use of advanced educational tools, so a need to upgrade their capabilities with the envisaged education was apparent. As an answer to this finding the curriculum and the design of the modules, more advanced knowledge and e-sources were envisaged to be developed to help the upgrading of the teachers’ knowledge and competence. Additional pre-session before the start of education was envisaged to be organized to help the teachers to understand the methodology and to learn how to use the available tools and the content of the e-sources. It was agreed as well by the team members that the developed educational content should focus on practical side of the education by enabling cyber skills, cyber hygiene, device and data protection to be enhanced in both target groups. The program was envisaged as well to generate ideas about the students’ career prospects through visits and discussion with experts from institutions caring about cybersecurity.

As the survey has shown, the students are familiar with games and seek fun when learning and working, so gamification of the educational content seemed to be obvious (Kim et al., 2023). Games offer meaningful and relevant context, which allows learners to understand the situation and the subject more effectively and in a more convenient manner replicating the reality (Omar et al., 2021). Games, compared with hands-on experiments, are more cost-effective, as they have low risk, provide an individually tailored pace of learning, standardized assessment of the results and immediate feedback for the learners and the teachers (Findley, 2011).

The design of cybersecurity education is a demanding task, requires an extra work by the teachers to understand the whole content of the applied KAs, but in the same time the involved work does not guarantee immediate benefits and payback for the time spent and the work. Most benefits from the involved amount of work should go to the students’ education and the expected outcomes. The team working on the education and content design was composed from cybersecurity experts and university teachers, some of them being contributors or editors of the developed methodology (Dynamic teaching model Concordia, 2021), the others were expert in e-platforms, other e-sources like Capture the Flag game and organization of contest. Playing the Capture the Flag game and taking part in a contest are recognized as very helpful tools in helping the teachers to build cybersecurity skills due to effectiveness and popularity of the game (Demmese et al., 2020; Švabensky et al., 2021).

3 The applied methodology and the curriculum design

3.1 Selecting the KA content and basic instructions for the teachers

The work on the curriculum started with inspection of the potential e-sources that address education in cybersecurity for students from 14 to 18 years for their potential inclusion in the program. Most of found are focused on spreading an awareness about cybersafety, while cybersecurity topics that contribute to the adoption of cybersecurity skills are less addressed. Similar findings were reported recently by researchers describing different initiatives for enhancing the safety behaviour of young people when using on-line services (Pencheva et al., 2020). Some studies addressing the relevance of cybersecurity education in high schools, have identified just a few topics that should become part of the high-school education for cybersecurity (Passy, 2017; Chindrus and Caruntu, 2023). Among the most listed topics are the cyberattacks and vulnerabilities like backdoors, eavesdropping, phishing, and social engineering. Another study ˙by Nygard et al. (2018) proposed a slightly different selection of cybersecurity to become part of the curriculum in computer science like data breaches in connection with web-browser configurations, generation of secure passwords, and fake accounts. The most complete overview of cybersecurity topics can be found in the UK CyBok data base (CyBok 2023), organized as a database with 19 KAs, each of them supported by two sub-areas. The application of the KAs for building skills was explored by Attwood and Williams (2023) but no educational model was provided. An additional finding of the carried research in 2022 was the abundance of cybersafety platforms that mainly provide advice on how to behave when using online services, explaining the danger of certain activities on the internet and providing information to adults about how to support children and young people in cases that might lead to abuse or when abuse happens. Most of the provided material on the internet is only available in written electronic form (brochures, posters or presentations, typically in pdf format). No recommendations were found on the internet platforms about how to apply the available educational tools like are the serious games and videos in cybersecurity educational program for the high-school level.

The game-based learning tools available on the internet cover very different topics that include spam, malware, cyberattacks, SQL injection XSS scripting, but they are intended to be used by very large and mixed audiences. Curriculum and fiches development require tremendous teamwork in analysing the existing offers and selecting the best solutions that match the training content for the target audience together with the selected e-sources. To extend the applicability of the program content to the level of knowledge in a particular school, three difficulty levels were prepared for each topic, i.e., beginner, intermediate and advanced. It was decided that the developed curriculum be made up of two major fields, i.e., cybersafety and cybersecurity, as these areas have different approaches for delivering content and the associated learning process.

The methodology and the KAs content (Concordia Roadmap and Methodology, 2020) were developed in parallel with the selection of the e-sources, which were evaluated for their appropriateness for provision of an active learning in the classes (Cole, 2022; Mirkovic & Peterson, 2016). The collaboration between the high-school teachers, the participating experts and the university professors engaged in teaching cybersecurity was continuous and as a result a set of 12 fiches were prepared for each of the two areas, i.e., cybersafety and cybersecurity, all of them equipped with information about the difficulty level and the requested students’ prerequisite knowledge. The desired topics identified by the students in the survey were included in the selection of the on-line tools, platforms, games and videos. The flexibility in the application of the education process was assured with the defined Dynamic model. The idea was that the education would follow each school’s selection of the module`s content as the survey results revealed that the participating schools differ very much in their programmes and applied attitudes for teaching. The flexibility of the approach is designed to enable different needs to be met by a particular school participating in the education exercise and the accommodation of the teaching content to be in line with the found student`s knowledge level in that school or class. That is why each KA fiche was prepared in three levels of difficulties. Teachers who are teaching informatics in the participating school, having a medium level of competence in informatics and computer science were invited to become familiar with the prepared program. The advanced knowledge modules were prepared to enable the teachers to upgrade their knowledge and, in cases when interest exists among the students, they could elaborate that content with them by offering additional training hours. Class preparation and the selection of the appropriate difficulty level of the content were considered as teacher’s task. The students enrolled for cybersecurity education were expected to have basic digital skills, but for the cybersafety modules no pre-course knowledge was required.

The significance of different tools in cybersecurity education was pointed already in a few other studies (Amankwa, 2021; Akinrotimi, 2023), where the researchers showed that students learn only 20% of what they hear and read, but can learn 90% of what they have practiced with the applied educational tools. The tools included in the modules address various learning objectives, but more attention in the program was paid to the most valuable ones that help the teachers to adapt the content and the teaching by practicing with the exercises on the online testing infrastructures and platforms with facilities that enable hands-on learning and experimenting by solving different tasks. Tools like are quizzes and games help the students’ knowledge to be evaluated in advance during the learning.

The quizzes encourage the teacher to test the students what they have learned. Usually a quiz requires 15–30 min to complete the task, if only one task needs to be fulfilled. Multiple choices are offered with questions to be answered. In the program it was envisaged the teachers to select quizzes that provide immediate feedback on the online system for the less demanding topics. The learning in cases when the taught topic is more complex the learning is enabled by playing cybersecurity games. They are much more common in the set with cybersecurity topics then in the cybersafety set. Playing games was envisaged to be applied after some progress is achieved in the learning of the presented content and when the teacher is confident that the level of understanding of the delivered content by the student is satisfactory. To check this, short tests are envisaged to be carried at the end of some classes. The selection of games to be included in the program (Kulkarni & Li, 2016) was not an easy task as the number of appropriate serious games with proven learnability (Jerman Blažič et al., 2017) for the high-school level education of cybersecurity is not numerous (Jerman Blažič, 2021; Miguel et al., 2020). It seems that the cybersecurity area for a less-demanding level of knowledge is somehow neglected by the serious game designers. The reason for that comes from the games’ origins, as most of the cybersecurity games were developed within military institutions and are not suitable for a non-military environment, for general education like the one to be implemented in most high schools. However, the games’ inclusion was based on an analysis from previous studies (Coentrad et al., 2020).

3.2 Organization of the educational content

The KAs of the educational content are organized in two modules each consisting from 12 fiches. A summary of the fiches collection is provided in Table 3. Selection of two fiches from each module is presented in the Annexes A1 and B1. Other fiches are available in the Conorcdia report where both authors actively worked for the study and contributed to the report writing (Teaching the Teachers report 2022).

Table 3 Overview of the 12 fiches in each of the addressed knowledge area: cybersafety and cybersecurity
Full size table

Each fiche listed in Table 3 is equipped with the following information that helps the teachers to prepare the class:

  • The level of knowledge addressed by the module.

  • The level of knowledge required by the students before they attend the class with that module (prerequisite);

  • The learning objectives (LOs) of each fiche.

  • The type of knowledge to be conveyed to the students.

  • Examples of questions to be discussed with the students during the class.

  • Practical examples to be used by the teacher, like games, hands-on exercises and quizzes.

  • Questions to be asked during the class.

  • List of links to e-resources to be used when preparing the class and later offered to the students during the class, like games, quizzes and videos.

Each fiche is accompanied by a list of URLs for the proposed practical examples and suggested e-resources. A total of 434 URLs are included in the teaching material, 195 of them address cybersafety and 239 cybersecurity. They are provided at the end of each fiche. Although all of them were tested before their inclusion in the program, regular updating is recommended to the teacher and possible replacement with new ones.

The list of serious games and quizzes in the safety module is modest and is located in fiche A11. The list of included games includes: “Identify and Challenge the information” (topic: Fake News) developed by the University of Portland, “IN ocular science and Harmony Square”. Example of the recommended quiz is “The all new NewsWise headline”. Examples from the game list in the cybersecurity modules are: “The Matching Games”, “Map OSI and TCP/IP Layers”, “Cisco learning – Binary Game”, “Print Matching game”, “Education Arcade”, “Cybersecurity NOVA Labs”, “PBS and Band Runner”, “Solve the Cryptoquote”, “RSA Encryption and Decryption”, “NCSC – the Cyber Sprinter”, “Capture the Flag.” Most of them are available without charge.

Capture the Flag is offered as a specific module composed of three fiches, Foundation, Intermediate and Advanced. It is recommended to students and teachers interested in the advanced learning of cybersecurity topics. Appendix B. presents the CTF fiche with an intermediate difficulty level.

4 Implementing the cybersecurity teaching model and the outcomes

4.1 Class preparation and delivery

Demonstration of the applicability of the developed educational model for both audiences, the high-school teachers and students in 9 European schools, was implemented in the 2022–2023 school year. The call for participation was sent to the high school’s management that participated in the survey but from 14 schools 9 accepted the invitation. The endorsement of the school’s participation after they had previously expressed a willingness to participate in the educational process according to the offered educational model required several conditions to be fulfilled by them. The school management agreed to the teaching of the selected curriculum topics to become part of the regular ICT classes. A background knowledge in ICT was requested from the high-school teachers that applied to take the preliminary courses dedicated to upgrade their knowledge in cybersecurity and to implemented the selected KAs in their schools. The enrolment of the students in the cybersafety and cybersecurity classes was voluntary and they were allocated by the school management in groups according to the presented knowledge in the tests, performed before the beginning of the classes. The tests were planned to be anonymous, as their purpose was to offer information about the students’ knowledge that helps the selection the fiches with the difficulty level that best matches the students’ knowledge. The test was recommended to last a maximum of 30 min. The tests was considered as a prerequisite for each class portfolio developed from the cybersecurity fiches.

The involved teachers were considered to be the person who builds, organizes, provides the content and cares for the application of the tools in the class. How the proposed methodology works and how it should be applied in the classes was presented during organized, on-line courses, before the education in the schools started in September 2022. The engaged instructors that led the courses for the teachers were university professors that participated in the curriculum’s development. Within the course, teachers were familiarised with the fiches’ content, with the application of the educational material for the classes and with the fiches dedicated to upgrade their knowledge of cybersecurity. Dynamic Teaching Model(2021), a method known as active learning approach was introduced as well. Due to the volatility of some e-resources and their rapid changes, the teachers were asked to check them before the class and in case they have changed to ask for appropriate replacement. A month was given to them to prepare the first classes before starting with the education. For students that showed an advanced knowledge of cybersecurity in the tests, the Capture the Flag module was prepared to be delivered to them, together with additional training hours. The students interested to take part in the contest at the end of the educational process attended the additional classes.

Open and lively discussion with the students was expected to be promoted by the teachers enabling to know how much the students are aware of threats and the taught subject. This method is more suitable for cybersafety topics as the students are, in general, more familiar with them in everyday activities on the internet. To increase the focus on one subject and enable, after the materials’ presentation, a lively discussion, was recommended all related activity to be carried out in one class interval. Among these activities the test before the class starts and the story line introduced by the teacher were suggested to be mandatory.

For most of the lessons the fiches provide practical examples. In presenting them it was advised they to be complemented with some local news relevant to the class topic or with inspirational talks from experts. Such examples are expected to engage the students in the learning process among them Bring-your-own-example (BYOE) tactic was considered to help the interchange of opinions between the students and the teacher, and to enable more lively discussions in the class. The fiches provide information about the learning objectives of each of the KAs, and when the teacher feels that they have been achieved their completion, a final test may be carried out by end of the class in the last 10 min. Questions for this type of test are also provided in the fiches, with exception of the fiches B12, B14 and B15 (Capture the Flag fiches).

The classes were held in laboratories equipped with computers connected to the internet. The classes started with an introduction about “what a career in cybersecurity means”. Some schools managed to invite experts from the industry that delivered a short speech about that topic by presenting cybersecurity role models and their career experiences, which was concluded with an invitation to the students to consider a career in cybersecurity. In two cases the speakers were school parents working in the cyber domain, like police, cybersecurity centres and CERTect.

The impression from the first classes were discussed with the teachers. The purpose of the discussion was to stress the topic’s priority within the curriculum. Story telling prepared by the teacher at the beginning of the class was as well approach intended to launch the discussion for an easier understanding of the game scenarios to be played during and after the class. The presented approach enables better flexibility in the teaching and rise the student interest. The last 10 teaching minutes before the class ends were suggested to be spent for asking the students if they have understood the lesson, either by talking or with a short test to be applied just before the end of the class that could reveal the adoption of the taught content. The order of presenting different topics was given to be selected by the individual teacher but alternating the cybersafety topics with the cybersecurity topics was recommended as a method, based on the findings that cybersafety topics positively impact on the teaching due to the more emotional/psychological content addressed in these topics. The alternating made the learning more relaxed. The topics selected by the teachers were different; however, the most frequently selected fiches for teaching were found to be “Data protection (Foundation, Intermediate level)”, “Essentials of cybersecurity (Foundation, Intermediate and Advanced level)”, “Attacks, Threats and Vulnerability (Intermediate level)”, “Defence against Cyber threats (Foundation level)” and “Capture the Flag (Intermediate and Advanced level)”.

The classes in all the participating high schools were very well attended. For example, in Slovenia, 365 students from five different high schools attended the classes that were provided by five teachers of computer science. In addition, training hours were applied for the five teams that decided to take part in the Capture the Flag contest. The class portfolios in these school were rich and contained both the foundation fiches as well the fiches with advanced topics, for example, “Fake news” was one of the most frequently taught topics. The additional hours for Capture the Flag game were well attended as they enabled the students to work at their own speed.

The selected modules taught in different schools were not the same as the selection was made by the teachers. However, the most frequently selected fiches were Protection of data (Foundation, Intermediate level), Essentials of cybersecurity (Foundation, Intermediate and Advanced level), Attacks, Threats and Vulnerability (Intermediate level), Défense against Cyber threats (Foundation level) and Capture the Flag (Advanced level).

4.2 Learning outcomes and the acceptance of the educational program

After the classes ended at the beginning of March 2023, a survey was carried out in the majority of participating schools to evaluate the adopted knowledge and the satisfaction with the learning. Questions were designed following the ACM education recommendation that defines three types of learning outcomes to be assessed in their Computer Science Curricula (Leune & Petrilli, 2017). They are the following: Familiarity, meaning that a student has understood the concept at the theoretical level, achieved mainly through lectures. Usage: a student understands the concept and can apply it correctly when the situation requires it. Assessment: a student understands the concept and can recognize the given concept in practice, (Redcliffe & Karcher, 2021). Seven questions were prepared that address the suggested learning outcomes and 6 questions addressed the satisfaction with the applied education. They are presented in Tables 4 and 5.

A total of 68% of the student` participants provided answers to the survey’s questions that measured the satisfaction with the teaching. There was no major difference in the answers regarding the gender. The same applies to the teams that took part in the Capture the Flag contest as both genders were present almost equally. Tables 4 and 5 contain the answers from the Slovenian high schools students about the satisfaction with the education and how much they have learnt. The answers provided by the students are presented in Table 4.

Table 4 Answers from students to questions asked after the Cybersecurity courses about their satisfaction with the education
Full size table
Table 5 Answers from the students on questions addressing the learning outcomes
Full size table

The interviews with the teachers provided some additional insights. The total number of teachers that took part in the conversation about their experience with the teaching was 14. They were carried out in the schools of the three countries that applied to participate in the Capture the Flag contest. Most of the teachers were glad to be part of the educational experiment, but most benefit they have found came from the clearly presented and structured fiches content of the fiches and the possibility to learn and prepare classes with material provided on the shelves. The prepared questions to be asked during the classes and the those included in the tests were found to be useful and contributed to the discussion with the students. The provided e-sources helped them to upgrade their knowledge and contributed less time to be spent for the class preparation. In the same time they gave useful guidance in testing the students pre-knowledge before the class started. The prepared recommendation for how to use the provided material was assessed to be very useful and beneficial. The involved videos, and game playing introduced more amusement and enjoyment in the classes and made the communications between the students and the teacher easier. easier learning. The teachers extra work, in attending the education prepared for them, and the obvious success of the implemented classes, were recognized and awarded.

4.3 Capture the Flag contest and the students’ results

To verify the results of the applied methodology in more real circumstances an international contest was organized with the game Capture the Flag, which is a game designed to test the cybersecurity ability of the participants. Evidence from other Capture the Flag contests with beginners shows that the students that had typically never worked with a Linux OS and with some security utilities before with the Capture the Flag concepts were capable of solving the Capture the Flag contest’s challenges (Švabensky et al., 2018).

In March 2023, the classes ended and an evaluation of the acquired knowledge by the students that attended the Capture the Flag classes with the three difficulty levels and the took the additional training hours. It was prepared as a participation in the international Capture the Flag contest that was set for March 23th 2023. The idea for the development of the Capture the Flag game comes from battle strategy. The participating teams in the Capture the Flag “battle” are supposed to protect their “base” and their “flag”. The other objective of the participating team(s) is to capture the flag from the other competing team (s) without being caught (Akinrotimi, 2023). The adaptation of the battle strategy to cybersecurity field allows different scenarios to be prepared, and as a consequence the results of the contests are organized with different tasks and they are not the same. This is enabled by the game’s nature, which allows it to be played in many different forms (Cole, 2022). The most frequently played form is the “defensive” scheme, where each team attacks an established network and the connected servers. Another frequently applied form is known as the “defensive/ offensive” scheme, in which a team either defends its network or attacks the other one. In the so-called “mixed” scheme and the teams are defending and attacking the designed network at the same time. The flag`s format is specific for each competition or contest, and usually the contest moderator is defining the format, the network and machine where the scores are collected and registered. The organiser of the contest appoints the moderator, who define the tasks, the scoring, the presentation of the network and provides an explanation of the game’s solutions after the game’s completion (Redcliffe, 2021). Tools like Wireshark that help to analyse the network are usually given to the competitors for their use. The Capture the Flag game was created by the NSA GentCyber program with the purpose to bring cybersecurity education into the curriculum of students before they get to college (Švabensky et al., 2018). Capture the Flag is a very popular game used in many contests that organized from 2012 by USA universities within the camps for pre-college cybersecurity education (Švabensky et al., 2021). The game was further developed after its early use and the number of camps and contests according to some Capture the Flag archives is now more than 100.

Due to the international dimension of the action the Capture the Flag contest was run remotely on the internet, which is usually the more frequently applied model for cybersecurity knowledge evaluation. The challenges for the contest were prepared according to the Jeopardy Challenges model, as a sequence of challenges and tasks (questions). This model is easier to be organized as the tasks are usually prepared as challenges with different levels of difficulty, enabling participation of teams with different knowledge and capability to solve them. To each of the tasks a score value was appointed by the moderator. In the simplest form the “flags” usually are simple text files placed in a specific location that should be discovered by the players. Each team worked in its own computer at their school. The participation in the contest was limited to a maximum of five high-school students in each of the participating teams, but the final number of team members was set to 4 members. The playing time was defined by the moderator to be 3 h, this timing is considered as the best time for beginners in a Capture the Flag contest. After the game completion the moderator reported the results and presented the correct solutions to the challenges. The time to solve the challenges is limited and the team that collects all the flags or gets the highest score after the playing time is over is the winner of the game. The challenges that were given in the contest required discovering of the server’s vulnerabilities (flags) in the established network, exploitation of the vulnerability, (e.g., enabling remote connecting), identification of the IP number of the attacker, and discovering whether a remote shell of the server is working correctly. The score engine was continuously receiving the solved challenges and was confirming the correctness of the solutions during the allocated playing time. The competing 19 teams from the three participating countries (Croatia, Romania and Slovenia) did not solve all the presented challenges, so the winner was the team that collected the highest number of scores. The points acquired during the contest ranged from 5 points (the team with the lowest scores) up to 87 points from a possible 100, obtained by the best team, coming from Romania. The ranking of the 19 teams that participated in the contest, each with 4 members, are presented in Table 6 with their names and the country code.

Table 6 Teams, country code and results from the CTF contest
Full size table

5 Discussion and assessment of the approach

The students that attended the additional training hours were asked after the Capture the Flag classes to report if they have understood the delivered content, the rules of the game and whether they felt capable of solving tasks that simulate real cases. All the participating students gave very positive answers and showed self-confidence regarding their absorbed knowledge. The Capture the Flag game is the most frequently used game for acquiring cybersecurity skills and for assessments of whether the students have understood the cybersecurity concepts that make them capable of acting in reality when they are faced with cyber-attacks (Švabensky et al., 2021, McDaniel et al., 2016), which gives an insight into the level of acquired skills and knowledge. The results of the contest in March 2023 gave positive answers to the questions. Half of the participating teams in the contest received more than 50% of the possible points, and this is considered as a good result and evidence that the applied education was useful. However, it should be noted that this contest was the first one where the high-school students participated.

The feelings of the involved students about the Capture the Flag exercise were very positive, although they complained about the lack of experiences with Capture the Flag. Students reported as well that they have enjoyed the teaching and the additional spare time spent on training the Capture the Flag game. A major part of the satisfaction and pleasure was due the applied interactive teaching and the gaming that gave them a sense of reality. It is worth mentioning that the students attending the Capture the Flag modules showed higher scores in the tests from the regular classes compared to the scores of the other students that did not attend the Capture the Flag teaching module. This confirmed the effectiveness of the Capture the Flag game in acquiring cybersecurity knowledge, as reported by other Capture the Flag events (McDaniel et al., 2016,; Kulkarni & Li, 2016). Other students assessed the classes positively after they experienced simulated reality provided by the games.

By considering the results the applied methodology was shown to be applicable and useful. The originality of the approach, that has its origin in up-to-date educational methods and material, made the classes attractive. The efforts put into the promotion of the active students’ participation improved the teaching and the students’ learning. The use of some competitive computer games, videos from involved e-platforms, storylines and other tools are elements that made the classes enjoyable with an interactive participation of the students. The advanced testing of the students’ knowledge before the lessons for accommodating the difficulty level of the delivered material, the provision of the teacher`s active involvement in presenting the content, selected according to the students’ interest and needs, and the additional components applied in the education process, made the proposed methodology applicable and attractive. One of the main components that contribute to that was the offered guidance to teachers for preparing the classes with ready-to-use content, use of additional learning tools and available e-sources. The methodology was designed to encourage the teachers to apply specific additional measures for making the classes attractive for the participating students and ensuring their participation. The introduction of the BYOE tactic in the area of cybersecurity is another component that makes the proposed methodology innovative and acceptable for all involved.

The novelty of the developed methodology and its applicability was confirmed by the answers collected during the surveys and the expressed personal interviews from the teachers, the school managements that scouted the participants needs in terms of teaching cybersecurity and cybersafety subjects at a level mostly adopted for the high-school students. However, the main innovative component of the applied methodology is in its flexibility that proved a best match between the different knowledge levels of the high-school students, with their interest in specific cybersecurity topics. The cybersafety content was offered as a supportive and theoretical component of the taught subjects, enabling good protection from the current threats that high-school students are facing every day. On the other hand, the cybersecurity content was focused on building more technical skills that help the students to know more about the protection of devices and the internet users. An additional contribution of the exercise was the information given to the students about jobs and career paths in cybersecurity that they might decide to follow in the future. Having the ability to practice potentially dangerous techniques in a controlled environment with the games and platforms, contributed to increasing the students’ confidence and their ability to recognize and defend themselves against malicious acts on the internet, which is the main objective of the cybersecurity education among the youth of Europe.

6 Concluding remarks

Cybersecurity education at all levels is still an area that needs further support as well as the application of innovative methods and flexible teaching approaches. Cybersecurity is an area with very fast and dynamic development that needs to be followed. Cybersecurity knowledge due to its important role in the digital society needs more dedicated attention for building cybersecurity skills among all European citizens. The need for the adoption of cybersecurity skills was stressed by most of the experts working on economic development issues, originating from the finding that the digital economy’s winners and losers will be determined by those that have the cybersecurity skills. The new and modern use of tools, videos, e-platforms prepared for teaching cybersecurity skills at the high-school level enable the transfer of the necessary knowledge to start early in the educational process as the youngsters are faced with internet threats every day in their private and business lives. The integration of cybersecurity and cybersafety in secondary-school education will have multiple benefits, among them it will enable more young people to pursue a professional cybersecurity career, which will reduce the cyber-skills shortage and the lack of skilled labour. However, it should also be recognized that the introduction of the cybersecurity programmes in high schools in the EU will still require significant time and effort by all concerned.