1 Introduction

Software-defined networking (SDN) has revolutionized the way of managing networks. It is a new paradigm in networking that has gained significant consideration from researchers and industry experts. SDN provides a centralized approach to managing the network’s infrastructure. Using SDN, network administrators can manage and monitor the network more efficiently. Centralized control of the network enables them to configure the network through software interface without physical intervention to network devices [1]. Despite providing many promising features in network management, SDN has also embraced several challenges, particularly regarding security. SDN controllers, the centralized management entities of SDN, are highly vulnerable to attacks that can lead to network disruptions or total network control. In recent research, several solutions have been proposed to address these security issues. However, conventional security solutions cannot fulfill the security requirements of unique SDN architecture. Recent research has also considered machine learning (ML) and artificial intelligence-based security models for SDN security. However, the main limitations of these models are the training complexity, the unavailability of datasets for training, and the testing accuracy. Among many different approaches proposed to enhance SDN security. Blockchain seems a promising solution to overcome SDN security because of its temper-proof and decentralized nature [2,3,4,5,6,7,8,9,10]. However, it is important to resolve the research gaps identified in these papers to accept blockchain-based solutions for SDN security [11].

1.1 Software Defined Networks (SDN)

SDN is a modern network paradigm separating the data and control planes. The centralized SDN controller does the management and configuration of the network. The data plane forwards the network packets based on the rules provided by the controller [9]. SDN provides a flexible and programmable approach to managing the networks, which allows network administrators to configure the networks through software interfaces without physical interventions [12]. SDN is an efficient way of managing the network as it provides a centralized view of the network through a software interface for network administrators. SDN seems to be a promising approach to fulfill the requirements of future networks [13]. This centralized control also allows network administrators to quickly respond to changes in network demand, such as traffic spikes, and adjust network resources accordingly. Figure 1 shows the architecture of the SDN.

Fig. 1
figure 1

SDN architecture

SDN architecture provides flexibility and agility not only on the management side but also on the security side. Separating the data and control planes allows network administrators to easily implement security policies and protocols, such as firewalls, intrusion detection systems, and access controls, to protect the network. Furthermore, Provides an improved and efficient way of network monitoring so that It is easy to monitor the malicious activities in the network and respond to security threats in real-time [13]. It also facilitates the new network and application requirements of organizations by implementing them through SDN controllers [14,15,16]. Figure 2 presents the main advantages of SDN technology to different communication networks.

Fig. 2
figure 2

Advantages of SDN

There are three main layers in SDN architecture: the application layer, the control layer, and the infrastructure layer. Each layer has its own interfaces and application programming interfaces (APIs). Figure 3 presents the structure of the main layers of the SDN network [17].

Fig. 3
figure 3

Layers of SDN

  1. A.

    Application Layer:

    The application layer in SDN controls the network’s behavior according to requirements. It defines the attributes and rules and makes decisions based on network changes. This layer uses the northbound APIs to communicate with the SDN controller about the implications of required changes.

    • Application layer interface (ALI): This layer provides the interface for interaction between SDN controllers and applications.

    • Northbound API: This sub-layer allows communication between network applications and SDN controllers.

  1. B.

    Control Layer:

    In SDN, the control layer is the most powerful and significant layer, and it possesses all the authority of the network. The control layer is connected to the lower and upper layers; the lower layer devices are via southbound APIs, and the upper layers are connected via northbound APIs. The controller directs the behavior of network devices based on the demands of the network.

    • Control layer interface (CLI): This sub-layer provides an interface for the SDN controller to interact with the underlying network devices.

    • Southbound API: Through this layer, the SDN controller communicates with the lower-layer devices and gives network-related instructions.

  1. C.

    Infrastructure Layer:

    All physical components in SDN lie in this layer. The infrastructure layer comprises all physical components, such as routers and switches. Southbound API is used to communicate with upper-layer devices.

    • Infrastructure layer interface (ILI): This sub-layer provides an interface for the network devices to interact with the SDN controller.

    • Data plane: This sub-layer forwards data packets between network devices.

    More flexibility and scalability are made possible by the modular approach to SDN architecture offered by these layers and sub-layers. Additionally, centralized network management and control are made possible by separating the control and data planes, enhancing network security and performance [17].

2 Related Work

SDN is a relatively new technology, and with respect to security, it has both advantages and disadvantages. On the one hand, SDN can be utilized to reduce or eliminate certain risks and vulnerabilities that are commonly exploited in traditional networks, such as lack of network segmentation, limited visibility and control, lack of fine-grained access control, Inflexible network provisioning, limited adaptability to emerging threats. However, the unique architecture of SDN can overcome the risk and vulnerabilities of traditional networks by providing programmability, flexibility, and more control over the network, but it also introduces new vulnerabilities and threat vectors that attackers can exploit [18] such as controller vulnerabilities, authentication issues of Northbound and Southbound APIs, and application-related issues. This is due to the separation of control and data planes, as well as the centralized nature of network controllers, which creates a single point of failure that can compromise an entire SDN network [19]. Each SDN layer has different security issues, illustrated in Fig. 4 and briefly discussed in Sect. 6.

This section details common attacks and malicious behaviors targeting various layers of the SDN architecture. Unlike traditional networks, each SDN network component, layer, or protocol may be subject to intentional or inadvertent misuse, leading to the exposure of system vulnerabilities or abnormal behavior [18, 20, 21].

2.1 Application Layer Attacks

In SDN, the application layer is responsible for providing network services to users and applications [21]. Security breaches on the application layer can lead to the exploration of sensitive information, and attacks can gain unauthorized access to sensitive information [22]. On the top of the list is a denial of service (DoS) attack, which overloads the application with bogus traffic so that application resources get busy in responding to the fake traffic and remain unavailable for legitimate users [23]. Another way attackers attack the application layer is by installing a malicious application. Once a malicious application is installed on the top layer, the SDN controller can be manipulated during the packet handling and forwarding rule implementation [24]. Attacks on the Northbound API (NBI) are also used to implement misconfiguration on this layer to target the communication between the controller and application [18, 25].

2.2 Control Layer Attacks

The control layer is SDN’s most important and sensitive layer because it manages the major network operations. Every attacker wishes to attack control because the whole network is compromised once the control layer is bypassed [18]. Various attacks are observed on the control layer, such as controller hijacking and intrusion of fake controllers. Controller hijacking means the attacker bypasses the controller, leading to unauthorized access to network devices and forged rules. Meanwhile, the intrusion of a fake controller means a fake system enters the SDN network and pretends to be a controller [26]. This illegal access can lead to manipulation of the network rules and unauthorized access to sensitive data [27, 28]. Controller poisoning through infected applications is also a type of attack on the control layer, which further negatively impacts the data plane by sending tempered packets and rules [19, 22, 29,30,31].

Fig. 4
figure 4

Layer-wise security issues of SDN

2.3 Infrastructure Layer Attacks

The infrastructure layer is considered the backbone of the SDN network as it consists of the physical and software infrastructure of the network [32]. One common attack on the infrastructure layer is a DoS attack. This attack floods the network with a large traffic volume, causing it to become overwhelmed and unresponsive. This can lead to network downtime and loss of service for legitimate users [23, 33, 34]. Network devices can be isolated through DoS attacks. Switches can be installed with tempered information to modify the flow rules, malicious applications or bypassed controllers can be sources to launch these attacks [29]. Using flow-rule flooding on the infrastructure layer is important to consider where an infected or malicious switch forces the controller for a new rule installation, which can affect the performance of network switches [12, 35].

3 Research Methodology

This literature review is organized to provide extensive knowledge in the field of SDN security. This literature review aims to provide valuable information for researchers to identify the gaps in SDN security. This section contains the research methodology used to conduct this work. The methodology contains research questions, a search string to find the related research papers, a list of scientific repositories to access the research articles, and selection criteria (inclusion and exclusion) of research papers.

3.1 Research Questions

This literature survey reviewed the security solutions used for SDN security. This survey mainly categorized SDN security solutions into three domains: conventional security, AI/ML-based security solutions, and blockchain-based security solutions. Based on these categories, the following research questions are formulated (see Table 1).

Table 1 Research questions formulated for this survey

3.2 Search String

To retrieve the SDN security-related studies, we used the following search strings:

SDN Security, blockchain-based security solutions for SDN, AI-based SDN Security solutions, and traditional/ conventional security solutions for /applied to SDN.

3.3 Search Repositories

The scientific digital libraries introduced in Table 2 are selected to obtain the results on developed search strings. However, search engines and studies indexers are not mentioned in this table.

Table 2 Considered repositories

3.4 Data Origin

In literature, many solutions have been proposed to address the SDN security issues. Researchers highlighted the sensitivity of SDN layers and proposed solutions to ensure the security of each layer. Moreover, different techniques are also considered in the domain of SDN security. Different search repositories mentioned in Table 2 were accessed to find these studies. Figure 5 shows the frequency (%) of research studies obtained from search repositories, and Fig. 6 shows the number of research studies obtained from each repository.

Fig. 5
figure 5

Percentages of research studies obtained from search repositories

Fig. 6
figure 6

Number of research studies obtained from search repositories

3.5 Publications Span (in Years)

Figures 7 and 8 mention the included research studies mentioned above in Figs. 5 and 6 based on inclusion and exclusion criteria. Figure 7 shows the percentage of studies selected between 2013 and 2024, and Fig. 8 shows the number of studies between these years.

Fig. 7
figure 7

Percentage of research studies year-wise

Fig. 8
figure 8

The number of research studies year-wise

3.6 Inclusion vs. Exclusion Criteria

Table 3 shows the inclusion vs. exclusion criteria of selected studies. The research strings extracted hundreds of studies from the selected repositories, and based on the criteria in Table 3, studies were included and excluded to prepare this work.

Table 3 Inclusion and exclusion criteria during the research process

4 Analysis

This literature review aims to synthesize and explore industry and research practices related to SDN security and identify the key areas to focus on. By analyzing a diverse range of scholarly articles, technical reports, and industry publications, this review will provide valuable insights into the latest advancements, trends, and best practices in SDN security, ultimately guiding the development of effective security solutions in SDN environments. The systematic literature review conducted for this paper sheds light on the wide range of security measures developed to protect SDN. The findings of the systematic literature review revealed that the SDN security mechanisms can be broadly categorized into three distinct domains: conventional security mechanisms, AI/ML-based security solutions, and blockchain-based security solutions, as illustrated in Fig. 9.

Fig. 9
figure 9

The considered domains of SDN security for this work

4.1 Conventional Network’s Security Mechanisms Applied to SDN

Table 4 comprises a detailed literature review of conventional security solutions applied to SDN architecture. Due to its increasing complexity and dynamic nature, traditional security solutions have shortcomings in securing SDN architecture.

Table 4 A critical review of conventional security solutions applied to SDN

In Table 4, a literature review of conventional security solutions applied to SDN has been conducted. This table concludes by summarizing the key findings of the literature review and highlighting the reasons why relying solely on conventional security solutions is ineffective for the security demands of SDN. Traditional networks are configured and managed manually. Additionally, traditional networks rely on vendor-specific firmware and hardware, which limits the flexibility and customizability of the network. In contrast, SDN introduces a change in the network landscape by separating the data plane from the control plane. SDN facilitates organizations in building more dynamic and adaptable networks according to their specific needs and subsequently requires different security solutions [47]. Conventional security solutions such as Firewall and Intrusion Detection / Prevention Systems (IDS/IPS), Virtual Private Networks (VPN), Access Control Lists (ACLs), Encryption and SSL/TLS, Role-Base Access Control (RBAC), and authentication authorization mechanisms, which are commonly used in traditional networks, lack when applied to SDN architecture. For example, firewalls and IDS/IPS mechanisms struggle to respond to the frequent changes and updates in the SDN environment. Moreover, maintaining comprehensive and consistent security policies across all layers of SDN is challenging. Similarly, traditional networks’ authentication and authorization mechanisms are not scalable enough to manage the increased number of devices and nodes in SDN networks. RBAC and ACL are rigid in nature and purely designed according to traditional networks and are not suitable for flexible and dynamic SDN architecture. Furthermore, the encryption and SSL/TLS are insufficient to provide sophisticated cyberattacks to target SDN controllers and communication channels [48, 49].

4.2 AI/ML-Based Security Solutions Applied to SDN

Table 5 investigates AI/ML-based solutions applied to SDN security. In this section, an extensive literature review uncovers AI/ML-empowered solutions employed to tackle real-time threat detection and mitigation in SDN. The subsequent section performs a detailed analysis of AI/ML techniques and sheds light on their advantages and limitations in the context of SDN.

Table 5 Critical review of AI/ML security solutions applied to SDN

Table 5 provides the exploration of AI/ML security solutions applied to SDN. During the literature review, renowned algorithms such as support vector machine (SVM), decision trees (DTs), k-nearest neighbors (KNN), convolutional neural networks (CNN), random forest (RF), and XGBoost have been evaluated for their performance in real-time threat detection and mitigation. The literature review highlighted the limitations of AI/ML-based solutions when applied to SDN. One of the main challenges faced by these solutions lies in training complexity and the requirement of real-time datasets to achieve accuracy in prediction, which causes considerable overhead and resource demand on SDN controllers. Additionally, the dynamic and rapidly changing nature of SDN networks badly impacts the performance of the pre-trained model [58]. Moreover, AI/ML algorithms demand high computational requirements, which introduces latency in key network operations and makes it difficult to provide real-time processing. Furthermore, AI/ML-based solutions are susceptible to adversarial attacks, where malicious actors intentionally manipulate network data to deceive the AI/ML models, leading to false positives or negatives in security alerts [58,59,60,61,62]. Considering these limitations, it can be concluded that the AI/ML solutions are unable to provide comprehensive security solutions to address the security issues in the SDN domain [63,64,65].

4.3 Blockchain-Based Security Solutions for SDN

Section 4.1 and 4.2 comprehensively reviewed the security solutions applied for SDN security. Moreover, the literature revealed that the SDN demands new and improved security mechanisms due to its unique architecture. Limitations of both traditional and AI/ML-based security solutions are also discussed in the above sections. This section examines the blockchain-based security solutions, their contributions, and limitations [11].

Table 6 Critical review of blockchain-based SDN security literature

Table 6 examines the blockchain-based security solutions applied to SDN security and found that blockchain technology has significant potential to revolutionize the SDN security approach by inheriting blockchain properties such as decentralization, immutability, and transparency. However, blockchain-based security has several challenges, such as scalability and a growing ledger with increasing transactions which add delays in network operations [76, 77]. Additionally, security automation is a crucial aspect of the SDN security domain because manual interventions decrease performance. Furthermore, lightweight Blockchain based solutions are important to reduce the computational overheads and resource requirements. To develop an effective and more agile SDN security solution based on Blockchain technology, it is important to address the mentioned issues [76, 78,79,80,81]. However, the research gap analysis on identified parameters from the literature can be observed in Table 7.

Table 7 Research gap analysis based on blockchain-solutions

5 Future Directions for SDN in B5G and 6G Networks

SDN represents a promising paradigm for Beyond fifth-generation (B5G) and sixth-generation (6G) networks. Table 8 provides the main applications of SDN for these upcoming networks [90, 91]. It also provides the potential research directions for each application. Despite its benefits, SDN poses new security risks, which will only get worse in the environment of 5G and 6G networks. Table 9 provides the main security issues with the SDN for B5G and 6G networks and the potential research directions to solve each security issue.

Table 8 Applications of SDN in B5G and 6G networks and main research directions [90,91,92]
Table 9 Security issues with SDN in B5G and 6G networks and main research directions [93,94,95]

6 Conclusion

The use of SDN has revolutionized network management, providing centralization and automation that improves efficiency and agility. However, security threats to SDN controllers continue to grow, necessitating the exploration of innovative security mechanisms that can address the security challenges faced by SDN networks. Blockchain technology offers a promising solution, providing secure and transparent data storage and transfer capabilities. However, current Blockchain-based security solutions have limitations, including scalability, high computational cost, and interoperability among different devices and layers [70, 73, 81, 85,86,87,88].

This work has conducted a comprehensive analysis, providing valuable insight into the landscape of security solutions applied to SDN. The review was categorized into three main sections: conventional security solutions, AI/ML-based security solutions, and Blockchain-based security solutions for SDN. A detailed literature review of conventional security solutions employed for SDN security was discussed in Sect. 4.1. It became evident that these traditional methods have their limitations when applied to SDN’s dynamic and programmable nature. These limitations include a lack of agility, inability to adapt to real-time threats, and manual configuration requirements, making them less effective in providing robust security for SDN networks, as summarized in Sect. 4.1. Similarly, Sect. 4.2 comprised an in-depth literature analysis of AI/ML-based security solutions considered for SDN security and identified that AI/ML solutions also face challenges such as the need for extensive training data, high computational overheads, and the risk of false positives and negatives as discussed in Sect. 4.2. The literature review on Blockchain-based security solutions for SDN conducted in Sect. 4.3 highlighted the key benefits of Blockchain technology to enhance SDN security [89]. However, the challenges mentioned in Sect. 4.3 need to be addressed to effectively implement Blockchain-based solutions in SDN environments.

In the future, SDN security studies should concentrate on creating and executing more robust security architectures, protecting communication protocols inside the SDN data plane, developing advanced SDN-specific intrusion detection and prevention mechanisms, utilizing AI to strengthen SDN stability, and creating frameworks that support interoperability and compliance to standard security procedures. SDN is a crucial component of contemporary networks, and its security requires more investigation. Researchers may strengthen SDN’s assault resistance and shield networks from various vulnerabilities by exploring these and other potential areas of research.