Understanding SQL Server 2016 Always Encrypted
0 likes1,057 views
The document discusses the evolution of encryption in SQL Server, highlighting the importance of data security and introducing the 'Always Encrypted' feature in SQL Server 2016. Always Encrypted improves encryption practices by allowing data to remain encrypted during transport and while at rest, solving issues around key management and visibility in code. However, it also has limitations, such as performance impacts and specific operational restrictions, necessitating best practices for its effective deployment.
Understanding SQL Server 2016 Always Encrypted
- 1. UNDERSTANDING SQL SERVER 2016 ALWAYS ENCRYPTED Ed Leighton-Dick
- 2. ABOUT ME Email: [email protected] Blog: edleightondick.com Twitter: @eleightondick
- 3. AGENDA • A bit of background • Always Encrypted • Best Practices
- 4. THE CURRENT STATE OF SQL SERVER ENCRYPTION
- 5. SECURITY HAS BECOME CRUCIAL • More risk to companies’ systems than ever before • Security is all about mitigating risk • Encryption is an important fail-safe in a security strategy • Variety of existing methods
- 6. ENCRYPTION IN THE APPLICATION • Extremely flexible • Extremely customizable • Wide variety of algorithms • Data may be encrypted from application to storage
- 7. COLUMN-LEVEL ENCRYPTION • Introduced in SQL Server 2005 • Highly configurable • Major types of encryption provided • Simple to use • Developer has to know little about encryption to use • Can be made secure using SQL Server permissions
- 8. TRANSPARENT DATA ENCRYPTION • Introduced in SQL Server 2008 • All data in database protected while on disk • Can be used with any application
- 9. ALL OF THESE HAVE SERIOUS SHORTCOMINGS… The bottom line: None of these are sufficient anymore! BY APPLICATION • Developers control the keys • No separation of duties • Not consistently implemented • Time-consuming IN-COLUMN • DBAs control the keys • Not consistently implemented • Time-consuming TDE • DBAs control the keys • All data viewable • Data decrypted in memory
- 10. WE NEED TO DO BETTER • Hackers are becoming more sophisticated • Data breaches are becoming more frequent and more expensive • Regulations (legal and contractual) are becoming more strict • Public tolerance is declining
- 11. AND YET… • Many companies have not implemented a comprehensive security strategy! • “Too expensive” • “Too difficult” • “Too time-consuming” • “It won’t/can’t happen to us” • “No one wants what we have” • “That’s someone else’s job” • “I don’t get it”
- 12. ALWAYS ENCRYPTED The New Black?
- 13. ALWAYS ENCRYPTED: A STEP IN THE RIGHT DIRECTION • Encryption occurs in the data driver • ADO.NET, ODBC, JDBC drivers currently available • Data is only unencrypted between application and driver • SQL Server cannot decrypt data on its own • Transparent to applications • Searchable while encrypted • Works on both on-premises and cloud databases • Available in SQL 2016, Azure SQL DB v12
- 14. ALWAYS ENCRYPTED ARCHITECTURE Image credit: MSDN (http://bit.ly/1Se0sm0)
- 15. ALWAYS ENCRYPTED SOLVES SOME BIG PROBLEMS • Strong encryption • Separation of duties • Consistent, simple implementation • Encryption methods are not visible in code or database
- 16. DEMO
- 17. TYPES OF ENCRYPTION • Algorithm: AEAD_AES_256_CBC_HMAC_SHA_256 • Deterministic • Searchable • Allows grouping, indexing, joining • Requires *_BIN2 collation • Randomized • No searches, grouping, indexing, joining
- 18. DATATYPE CONVERSIONS • Conversions happen behind the scenes • Ciphertext stored as VARBINARY • Length • Most data types: 65 bytes • Decimal/Numeric/UID: 81 bytes • Variable length: Variable (formula) • Reference: bit.ly/1S9ieNu
- 19. TYPES OF KEYS • Column master key (CMK) • Encrypts CEK • Stored externally in key stores • Column encryption key (CEK) • Encrypted by up to 2 CMKs to allow rotation • Stored internally
- 20. KEY STORES • Local certificate store • Hardware Security Modules (HSM) • aka Extensible Key Management (EKM) • Azure Key Vault • Custom
- 21. METADATA • Column Master Keys • sys.column_master_keys • Column Encryption Keys • sys.column_encryption_keys • sys.column_encryption_key_values • Column encryption • sys.columns • Parameters • sys.sp_describe_parameter_encryption
- 22. DEPLOYMENT • Can be scripted using T-SQL or PowerShell • CMK via PowerShell
- 23. AND NOW, THE BAD NEWS… • Always Encrypted is not a silver bullet • V1 product = Significant limitations
- 24. LIMITATIONS • Comparisons on AE columns • Deterministic – Equality • Randomized – None • Other operations disallowed • LIKE, range lookups • Functions, etc. • Data must be passed as typed parameters • ADO.NET: SqlParameter • ODBC: SQLBindParam
- 25. LIMITATIONS: DATATYPES/FEATURES xml timestamp/rowversion image ntext/text sql_variant hierarchyid geography geometry alias/sysname UDTs FILESTREAM ROWGUIDCOL String w/o BIN2 Full-text search Computed Ref by computed Sparse column set Ref by statistics Partitioning column DEFAULT, CHECK IDENTITY CDC Temporal table Masked PolyBase Hekaton Table variables FOR XML FOR JSON PATH Replication Linked servers SSDT *Key/index *Stretch *Triggers … and more
- 26. LIMITATIONS: PERFORMANCE (CTP) • Significant impact on both speed and space...? Image credit: Aaron Bertrand, SQLPerformance.com (http://bit.ly/1quelGS)
- 27. LIMITATIONS: PERFORMANCE (RTM) ©2016 Kingfisher Technologies 33 38 87 44 - 10 20 30 40 50 60 70 80 90 100 Insert 100K Rows Read Random Rows Impact of Always Encrypted - 1-Core Client, 1-Core Server Unencrypted Always Encrypted
- 28. LIMITATIONS: PERFORMANCE (RTM) ©2016 Kingfisher Technologies 33 38 87 44 400 191 - 50 100 150 200 250 300 350 400 450 Insert 100K Rows Read Random Rows Impact of Always Encrypted – 1-Core Client, 1-Core Server Unencrypted Always Encrypted Legacy Encryption
- 29. LIMITATIONS: PERFORMANCE (RTM) ©2016 Kingfisher Technologies 92 106109 118 594 520 - 100 200 300 400 500 600 700 Insert 100K Rows Read Random Rows Impact of Always Encrypted - 4-Core Client, 16-Core Server Unencrypted Always Encrypted Legacy Encryption
- 30. LIMITATIONS: PERFORMANCE (RTM) 5.164 17.031 15.282 0 2 4 6 8 10 12 14 16 18 Total Space Space Used (MB) Unencrypted Always Encrypted Legacy Encryption ©2016 Kingfisher Technologies
- 31. LIMITATIONS: SECURITY • Key management still necessary • CMK must be accessible to each client • CMK can be compromised in key store if not properly secured • CEK still stored in database • Compromise less likely but theoretically possible • Deterministic encryption values can be guessed • Driver replaced by malware
- 32. BEST PRACTICES
- 33. MIGRATING PROTECTED DATA • CREATE/ALTER USER … ALLOW_ENCRYPTED_VALUE_MODIFICATIONS • Skips metadata checks on server during bulk copy • Be extremely careful… Data corruption is possible • Best Practices • Use separate account or disable option when not in use • When possible, use client driver with capability to suppress cryptographic metadata checks for a single session (i.e., ADO.Net) • Reference: bit.ly/1SmXEmv
- 34. CONTROLLING THE PERFORMANCE IMPACT • Only enable when necessary • When most queries access encrypted columns… • Enable in connection string • Disable on individual queries • When most queries do not access encrypted columns... • Disable in connection string • Enable on individual queries • Reference: bit.ly/1SmXPhI
- 35. KEY MANAGEMENT • Lost keys = Lost data • Backup all CEKs regularly • Protect your keys • EKM > Local • Regular backups + strong passwords + strict controls • Rotate column master keys regularly • Reference: bit.ly/1SmXXh8
- 36. WRAP-UP
- 37. REVIEW • New challenges require new techniques • Always Encrypted is a good fit… for some workloads • Not a silver bullet • Whatever encryption you use, take care of it!
- 38. RESOURCES • MSDN has complete documentation • More links on edleightondick.com (coming soon)
- 39. QUESTIONS? Email: [email protected] Blog: edleightondick.com Twitter: @eleightondick
- 40. THANK YOU FOR ATTENDING! ~ Please remember to fill out your comment cards ~
Editor's Notes
- #11: Code is time-consuming to build, maintain Code that implements encryption should be locked down, but rarely is Writing own encryption code assumes expertise that may not exist – Some developers even like to create their own “encryption” routines!
- #15: All editions of Azure SQL DB v12 – No word on SQL 2016 editions yet .Net 4.6, ODBC v13, JDBC v6
- #17: It’s not perfect, though… we’ll talk about that later.
- #19: BIN2 - binary code point comparison sort Main collation: Latin1_General_BIN2 (131 total - most languages covered) – Latin1_General_100_BIN2 handles Unicode better
- #22: Azure Key Vault/EKM/HSM not available with ODBC (yet) Azure Key Vault – Installed separately