Http - All you need to know
Download as PPTX, PDF5 likes772 views
This document provides an overview of HTTP including: - HTTP is a stateless protocol that does not require servers to retain user information across requests. - Popular HTTP proxy tools like Fiddler and Burp Suite can be used to inspect and debug HTTP traffic. - Key parts of HTTP include requests methods, response codes, headers for accepting content types, encoding, authentication, and more. - Common players that interact with HTTP include web servers, load balancers, caching servers, CDNs, and security tools.
![Methods
Method Used for
GET Retrieve a resource
POST Create / Update a resource [Not Idempotent]
PUT Create / Update a resource [Idempotent]
DELETE Delete a resource
HEAD Retrieve a resource except the body](https://image.slidesharecdn.com/http-allyouneedtoknow-161125101227/85/Http-All-you-need-to-know-13-320.jpg)
Http - All you need to know
- 1. HTTP All you need to know
- 2. Who are you?
- 3. Gökhan Şengün R&D Business Dev., New Product & Solutions Manager www.gokhansengun.com @gokhansengun
- 4. Aim • Better understand HTTP basics to debug problems better • Know HTTP players to see the big picture • Know useful tools to do things faster
- 5. HTTP • HTTP is a stateless protocol. • How is being stateless like? • A stateless protocol does not require the server to retain information or status about each user for the duration of multiple requests.
- 6. Http Proxy
- 7. Popular Http Proxies • Fiddler • Burp Suite • Browser Developer Tools (Embedded Proxy)
- 8. Demo – Bare Metal - Using Telnet
- 9. Demo – Browser Developer Tool
- 10. Demo – Fiddler
- 11. Demo – Burp Suite
- 12. Http Protocol – Important Parts
- 13. Methods Method Used for GET Retrieve a resource POST Create / Update a resource [Not Idempotent] PUT Create / Update a resource [Idempotent] DELETE Delete a resource HEAD Retrieve a resource except the body
- 14. Response Codes Code Meaning 1xx Informative 2xx Success 3xx Requires Additional Action 4xx Client Error (It is your fault) 5xx Server Error (It is my fault)
- 15. Accept (Req) MIME used for media-type. Client gives hint about the types that it understands well and preference. Syntax: • Accept: <MIME_type>/<MIME_subtype> Examples: • Accept: application/json, text/xml;q=0.9, */*;q=0.8
- 16. Content-Type (Req / Resp) MIME used for media-type Examples: • Content-Type: text/html; charset=utf-8 • Content-Type: application/json • Content-Type: text/xml
- 17. Demo – Accept and Content-Type
- 18. Host (Req) • Hints the web server about the domain name requested • Optionally includes port, default • HTTP: 80 • HTTPS: 443 Examples: • Host: www.gokhansengun.com • Host: localhost:8090
- 19. Connection (Req / Resp) • Hint from both client and the web server about TCP connection • close: if either party for some reason wants to close • keep-alive: if either party want to keep open for further requests • Persistent connection (default in HTTP/1.1 • RFC 2616 limits 2 connection per host, browsers have 6 now. Examples: • Connection: close • Connection: keep-alive
- 20. BTW: Http Pipelining • Only Idempotent requests allowed (GET, HEAD) • Guess why? • Has benefit only on high latency setups.
- 21. Accept-Languge (Req) • Hint from client about its language preference Examples: • Accept-Language: en-US,en;q=0.8 • Accept-Language: tr-TR, tr;q=0.9, en;q=0.8, *;q=0.5
- 23. Accept-Encoding (Req) • Hint from client about its encoding preference Examples: • Accept-Encoding: Accept-Encoding: gzip, deflate, sdch • Omit for non-encoding
- 25. Referer (Req) • Hint from client about the last page user navigated from. • Allows analytics, caching, logging Examples: • Referer: http://ads.xyz.com
- 26. User-Agent (Req) • Hint from client about the type of client Examples: • User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.71 Safari/537.36
- 27. Cache-Control (Req / Resp) • Hint from server to all over the world about resource’s cache eligibility. • Cache-Control: no-cache • Cache-Control: public • Cache-Control: private • Cache-Control: no-store • Cache-Control: max-age=300 • Cache-Control: public, max-age=31536000
- 28. Post / Redirect / Get Pattern (1) • Problem (Multiple Post requests)
- 29. Post / Redirect / Get Pattern (2)
- 30. Post / Redirect / Get Pattern (3) • Solution
- 31. Demo – Mix
- 32. HTTP Players • Web Servers • Load Balancers • DDoS Protection and WAF Systems • Cache Server • CDN (Content Delivery Networks) • Cloudflare
- 33. Web Servers • Nginx • Apache • IIS
- 34. Load Balancers • Balance HTTP load between servers • Balance statefully (needs your SSL private key) • Cache responses • Alters requests and responses • Blocks, rate-limits requests • Does SSL-offloading (needs your SSL private key and beneficial only if you have HW LB)
- 35. DDoS Protection Systems and WAF • Observes traffic (needs your SSL private key) • Detects malicious activity – several attacks • Blocks IP, IP Range • Redirects to No CAPTCHA or reCAPTCHA • Rate-limits requests
- 36. Cache Servers • Caches any type of HTTP responses from origion • Could be static file or reference data • Like very very simple KV store • Powerful if scripting allowed Examples: • Varnish • Nginx
- 37. CDN (Content Delivery Network) • Caches the content on the edges • Request does not enter your data center • Very very efficient
- 39. Cloudflare • CDN • Load Balancing (Cloud – Region Based through DNS) • DDoS • WAF • Rate Limiting • Website Optimization • Cache Header Optimization • AutoMinify • Aggressive Gzip • Automatic Content Caching
- 40. Cookies • Helps stateless HTTP protocol statefulness when necessary, • Has restrictions in EU. Types: • Session Cookies • Persistent Cookies
- 41. Authentication and Tokens • Basic Authentication • Forms Authentication • Token Authentication
- 42. Session Cookie vs Token Auth
- 43. HTTP Security • Use SSL/TLS for transport layer security (HTTPS everything) • Why? • Set Cookies with HttpOnly • Avoid Cross Site Scripting • Set Cookies with Secure • Avoid sending cookies in HTTP requests • Use HSTS (HTTP Strict Transport Security) header • Instruct browser to comm only with HTTPS for a period of time • Avoid SSL-stripping attacks
- 44. HTTP Performance Measurement • Use Apache ab • Use Apache JMeter (blogs from www.gokhansengun.com) • http://loader.io/ • https://www.blazemeter.com/ • Use APM (Application Performance Monitoring) tools • NewRelic, Dynatrace, Riverbed, App
- 45. Scaling HTTP • Use Cache Server • Use CDN • Cache Aggressively • Use DNS load balancing • Use SPA (Single Page Application) Technique • Minify and bundle JS / CSS
- 46. Questions?
Editor's Notes
- #9: Demo using Telnet
- #11: Show Text wizard Repeat Requests Compose
- #14: GET carries parameters in the URL whereas POST carries in the request body COPY paste the URL easily
- #15: GET carries parameters in the URL whereas POST carries in the request body COPY paste the URL easily
- #18: Using Github API https://api.github.com/users/gokhansengun
- #23: www.facebook.com using Fiddler’s Composer
- #25: www.gokhansengun.com using Fiddler’s Composer
- #32: Visit www.milliyet.com.tr Revisit Visit www.gokhansengun.com Revisit Visit localhost/app, login Post Redirect Get Tell what 301, 302, 304 are.
- #45: Apache Ab ab -k -c 1000 -n 10000 http://172.16.41.197/ist