Traditional firewall systems use static rule sets, making them unsuitable for growing cyber threats and network circumstances. In this research, we automate firewall rule development and optimization using reinforcement learning (RL) to increase network security and reduce human setup. This paper introduces FireRL, a system that uses reinforcement learning to help make smart decisions about firewall rules by treating it like a Markov Decision Process, enabling an RL agent to learn good firewall rules from simulated network traffic. This proposed method utilizes the Deep Q-learning algorithm to balance throughput, latency, and threat mitigation via repeated improvement. Experiments are performed in benign and dangerous traffic situations. Firewalls outperform static firewalls because they quickly react to new threats and reduce false positives. Resistance to new attack vectors demonstrates the system's flexibility and resilience. This research concludes with a self-optimizing firewall approach that greatly lowers expert-led settings. FireRL's proactive and scalable RL-based defense is ideal for current cybersecurity.