Backup and Restore with pfSense 2.4 - pfSense Hangout August 2017
1 like5,203 views
The document discusses various methods for backing up and restoring the pfSense configuration file (config.xml) including: 1) Backing up config.xml from the GUI by downloading it as an XML file, which can optionally be encrypted. 2) Restoring config.xml from the GUI by uploading a backup file. Interface assignments may need to be adjusted during restore. 3) The configuration history in the GUI automatically backs up config.xml on changes and allows restoring previous versions. 4) Manually editing config.xml requires care to avoid breaking the XML format. Removing package information can prevent reinstalling unwanted packages.
Backup and Restore with pfSense 2.4 - pfSense Hangout August 2017
- 1. Backup and Restore with 2.4 August 2017 Hangout Jim Pingle
- 2. About this Hangout ● Project News ● pfSense Configuration File ● When Reinstalling for 2.4 may be necessary ● Backup from the GUI ● Restore from the GUI ● How Restore Works ● Restoring to Different Hardware ● Using the Configuration History ● Using the 2.4 installer to recover or restore ● Manually Editing config.xml ● AutoConfigBackup ● Other Backup Techniques
- 3. Project News ● 2.4.0-RC is out! – http://www-dev.netgate.com/blog/pfsense-2-4-0-rc-now-available.html – https://doc.pfsense.org/index.php/2.4_New_Features_and_Changes – 2.4.0-RELEASE will be here soon, speed depends on what RC testing uncovers – No 32-bit x86, no NanoBSD – FreeBSD 11 base – 2.4.1 will follow shortly after with a FreeBSD 11.1 base – If tracking snapshots, make sure firewall is set to Stable on Update Settings, otherwise it won’t reach the RC or RELEASE ● pfSense Firewalls are now available on AWS GovCloud (US) – Government agencies and customers, sensitive unclassified workloads, specific regulatory and compliance requirements (e.g. ITAR) – https://www.netgate.com/press-releases/netgate-announces-pfsense-firewalls-on-aws-govcloud-us.html ● A free 30 day trial is now available for the pfSense 2.3.4 AMI on the AWS Marketplace ● Some more info on what will become pfSense 3.0 – https://www.reddit.com/r/networking/comments/6upchy/can_a_bsd_system_replicate_the_performance_of/dlvdq2e/ ● FRR Package is now available on 2.4 for routing protocols (BGP, OSPF, OSPF6), replacement for Quagga and OpenBGPD ● 2.4 translation effort still ongoing – Up to 13 languages total included in 2.4, more getting added as they cross the 75% barrier – Over 300 translators signed up ● New “pfSense Supplementals 1” advanced topics course – https://www.netgate.com/training/pfsense-supplementals-1.html
- 4. pfSense Configuration File ● All configuration data for pfSense and packages is held in one file, /cf/conf/config.xml ● The file contains an XML representation of the configuration data ● The file is plain text and can be read in many text editors, browsers, XML viewers, etc ● Using this file alone, a firewall can be restored back to a fully operational state – No need for full disk or other filesystem backups ● There are several ways to backup this file and restore it, as we will discuss today ● This file contains sensitive data! – Certificates, passwords, and other important information are contained in the config.xml file – These must be stored clear or in an easily reversible format (e.g. base64) because the firewall needs to use the passwords in various ways ● Placing them into configuration files for use by daemons or other parts of pfSense software ● Using them to make outbound connections (e.g. PPP WANs or VPN clients) – These items are not obscured to better drive home the point that the file contains sensitive data and needs protecting ● Secure config.xml backups appropriately according to your organization’s policies and security best practices ● Keep copies of the backup in safe places (on site, off site, offline, etc) ● Periodically test the backups – Few things are worse than an unusable backup! ● Does not backup any unsupported manual changes made outside of pfSense (e.g. edited files)
- 5. Reinstalling for 2.4 ● Reinstalling 2.4 may be needed, or desired, for a variety of reasons – Upgrading to 64-bit hardware – Moving from 32-bit to 64-bit install on 64-bit hardware – Moving from NanoBSD to a full install – Switching from UFS to ZFS – Upgrading to hardware with AES-NI to prepare for pfSense 2.5 – Getting a fresh start with a clean installation base or a new disk ● As a part of reinstalling, backing up and restoring the existing configuration is key in getting back up and running quickly
- 6. Backing Up from the GUI ● To take a backup using the GUI, visit Diagnostics > Backup & Restore ● In most cases, clicking Download Configuration as XML is sufficient to get a good, usable backup of config.xml ● To back up only part of a configuration, choose that in the Backup Area – Partial backups can be useful for transferring sections to another firewall or for restoring just part of the configuration later – Partial backups CANNOT be restored to different versions of pfSense! ● Choosing to Skip Packages will omit packages and their settings from the backup – This is useful for clearing out old/broken package data from a configuration, but it’s not a good way to migrate to a new setup because it loses the package settings not just the installation records – It’s better to uninstall the packages and then take a backup, or manually edit out the <package> tags (see later slides) ● By default the Skip RRD data box is checked, meaning that the RRD data is not included in the backup – RRD data can consume large amounts of space, it will increase the size of the backup by several MB – RRD data is backed up in an architecture-independent way, so backing up on 32-bit and restoring to 64-bit is possible ● Enabling Encryption for the configuration uses a Password to encrypt config.xml before downloading with AES-256 – Example: openssl enc -e -aes-256-cbc -in config.xml -out config.xml.enc -k mypassword ● Click Download Configuration as XML after all of the desired options are set
- 7. Restore from the GUI ● Make sure you have working console access before starting, just in case ● To restore using the GUI, visit Diagnostics > Backup & Restore ● When restoring a complete configuration backup it can be from the same version or any older pfSense version – For example, a configuration from pfSense 2.3 can be restored to 2.4 – Restoring from an older version requires a complete configuration, partial configurations cannot be restored on different versions – Best practice is to make sure there are no installed packages in the backup when crossing versions ● When restoring a partial configuration, pick the appropriate Restore area – A partial configuration is the XML section for the area wrapped in <pfsense></pfsense> ● Use the Browse… button to locate the config.xml backup – Exact method varies by OS ● The Encryption option uses the supplied Password to decrypt an encrypted backup taken using the pfSense GUI with the Encryption option – Example: openssl enc -d -aes-256-cbc -in config.xml.enc -out config.xml -k mypassword
- 8. How Restore Works ● Supplied config.xml is put into place as /cf/conf/config.xml and it replaces the running configuration ● If the interfaces do not match, an assignment prompt is shown to reassign them as needed – Navigate to the VLAN, PPP, and other tabs under Interfaces > Assignments to ensure the proper physical interfaces are picked – On the Interface Assignments tab, Select appropriate interface assignments and click SAVE – The interface settings are updated at that time, including IP address assignments, so the local client may need adjusted to reach the GUI again – If contact with the firewall is lost, reboot it from the console or if all else fails, power cycle the firewall – After saving, click Apply Changes from the GUI to complete the interface assignment process and reboot ● Restoring RRD (GUI Restore only) – The RRD data in the restored config.xml is read back into a native binary format and then the data is removed from the running config.xml – If the graphs under Status > Monitoring contain no data and/or do not update after 15-20 minutes or so, reset the RRD data ● Console changes – Primary console, serial console speed, and password protected console are setup and will be active once the firewall reboots ● Packages – After the reboot, packages referenced in the restored config.xml will be fetched and reinstalled ● Internet connectivity is required post-restore, even without installed packages, or else the firewall will be delayed waiting for queries to the package server to timeout
- 9. Restoring to Different Hardware ● Check console settings to be sure they are correct – Some hardware will force the serial console to be on and primary, but a specific speed is not forced! ● Best practice is to remove packages before taking a backup, though if the same version of pfSense is installed on both the old and new hardware it should be OK ● Unless the new hardware is similar to the old hardware, disable any extra hardware-specific options such as cryptographic hardware, temperature monitoring, and so on ● Reassign the interfaces after restoring, be careful to adjust VLANs, PPPs, etc ● Watch for after effects of changing hardware after swapping it in – New hardware means new NICs, which means new MAC addresses (unless spoofed) – ARP caches in upstream/downstream gear may need cleared or devices rebooted – Cable modems on WAN or other CPEs will likely need powered off/back on
- 10. Using the Configuration History ● From the GUI, Diagnostics > Backup & Restore, Config History tab ● Last 30 backups are kept by default, but that can be adjusted by clicking + and setting a different number – Or enter 0 to disable the backups ● List of backups shows the date, configuration revision, backup size, change description, and actions ● Convenient, but hosted on the firewall so not a great source if the hardware has a problem ● To see the difference between two configurations: – Select the older configuration using the left column of radio buttons – Select the newer configuration using the right column of radio buttons – Click Diff to view the difference with lines removed in red and added in green ● The three buttons in the Action column let you perform the following: – Refresh arrow: Restores the older version of the configuration on this row ● This does NOT reboot the firewall, so you must save/apply any affected area(s) or manually reboot – Download button: Lets you download a copy of the previous configuration on this row – Trash Can: Permanently removes the configuration file on this row ● Use to remove known-bad configurations ● Configuration History is also available on the console! Option 15, then 1 to list backups, 2 to pick one to restore
- 11. Other Restore Methods ● “Recover config.xml” in 2.4 installer – Used for recovering configurations from a previous installation of pfSense on the same disk – Lists existing UFS and ZFS partitions on the firewall – Attempts to repair (if needed) and mount the paritition, then looks for an existing config.xml – If a config.xml is found it is copied into the installation after the drive is wiped ● Automatically restore from USB during install (FKA “PFI”) – Requires a USB thumb drive that is DOS formatted (FAT16 or FAT32) separate from the installation media – Configuration file must be named config.xml and it must be placed in the conf directory, e.g. J:confconfig.xml – USB thumb drive must remain connected throughout the entire installation process – At the end of the installation, remove both the installation media and this thumb drive ● Caveats – Console setting changes will take one additional reboot to fully apply – Neither of these methods are currently capable of restoring RRD data or encrypted configuration files – Interface mismatches must be handled manually at an assignment prompt on the console after the firewall reboots ● Not usually a concern for Recover since it’s run on the same hardware
- 12. Manually Editing config.xml ● Best to avoid it if possible, but a few tasks are made easier by hand editing – Interface assignment changes, especially those with numerous VLANs ● Example: Change from emX to igbX – Trimming out package information ● <package>...</package> blocks denote that a package is installed, removing those blocks will prevent a restore from reinstalling the packages referenced by those <package> blocks ● Look for other related tags from dangling package installations, such as <menu>, <service>, <tab>, and <plugin> ● File is in XML format, can be edited with many text editors (but not Notepad) – Use an advanced/programming style editor such as Notepad++, UltraEdit, Kate, vi/vim/emacs, etc. ● Be very careful not to break XML format or leave mismatched tags – Can run through xmllint to check for errors, if available
- 13. Manually Editing config.xml ● Do not perform an automatic search and replace! – Carefully check each instance and approve each change manually ● Example edits of VLAN and PPP tags to change from emX NICs to igbX
- 14. AutoConfigBackup ● Available to all Gold Subscribers ● Automatically uploads an encrypted copy of config.xml to our servers on each change ● Backups are keyed off of the firewall hostname, so ensure firewalls have unique names ● Retains last 100 configurations per hostname, number of hostnames depends on subscription status ● No “bare metal” restore method at this time, must be online and able to access the AutoConfigBackup package
- 15. AutoConfigBackup ● Install the package from System > Package Manager if it is not already present ● Access via Diagnostics > AutoConfigBackup ● Enable AutoConfigBackup ● Fill in your Gold Subscription username (not e-mail address) and password ● Enter a strong Encryption Password, but one you can remember or recover! – Without this Encryption Password, the backups cannot be decrypted! – Use a password manager/vault/etc – Keep a local unencrypted backup in a secure place, that backup will contain the Encryption Password
- 16. AutoConfigBackup ● Restore Tab – Hostname drop-down lists all hosts in your account with backups ● A host in your account can read backups for all other hosts on the same account, which can help if you have a firewall down and no handy backup! – All backups for the selected host are listed – Buttons work similar to those on the Configuration History GUI – Download button goes to an information page which includes both the encrypted and decrypted config.xml copies – Copy/paste the decrypted contents to a local file to save ● Backup Now tab – Enter a custom description and force a backup entry – Useful for significant milestones, for example “Before 2.4 upgrade” ● Stats Tab – Shows a list of all hostnames on the account and how many backups have been stored for each – Magnifying glass icon leads to the Restore tab for that hostname – Trash can icon will remove all backups for that host ● Useful for removing defunct hosts so they do not count against your account host limit
- 17. Other Backup Methods ● Fetch using wget, cURL, or similar – Requires special handling to submit the login and request with CSRF tokens – https://doc.pfsense.org/index.php/Remote_Config_Backup ● Push or pull via scp – Copy from /cf/conf/config.xml – Use SSH keys to automate – Avoid using keys without a passphrase, use ssh-agent where possible
- 18. Conclusion ● Questions? ● Ideas for hangout topics? – Post on forum, comment on the blog posts, Reddit, etc