Customising Your Own Web Framework In Go

Customising Your Own Web
Framework in Go
20 January 2015
Jonathan Gomez
Engineer, Zumata

This Talk
Overview
- Intro to serving requests with http/net
- Customising Handlers
- Writing Middleware
- Ecosystem
Key takeaways
- Compared with Ruby/Node.js, mainly using the standard library is considered normal
- Interfaces and first-class functions make it easy to extend functionality
- Ecosystem of libraries that work alongside http/netis growing

Intro to Serving Requests with http/net

Serving Requests via Standard Lib (1/4)
packagemain
import"net/http"
funchandlerFn(whttp.ResponseWriter,r*http.Request){
w.Write([]byte(`Helloworld!`))
}
funcmain(){
http.HandleFunc("/",handlerFn)
http.ListenAndServe("localhost:4000",nil)
}
ListenAndServe - creates server that will listen for requests
Each request spawns a go routine: goc.serve()

ServeMux matches incoming request against a list of patterns (method/host/url)
ServeMux is a special kind of Handlerwhich calls another Handler
Handler interface
typeHandlerinterface{
ServeHTTP(ResponseWriter,*Request)
}

Request handling logic in ordinary function func(ResponseWriter,*Request)
funcfinal(whttp.ResponseWriter,r*http.Request){
w.Write([]byte("OK"))
}
Register the function as a Handler on DefaultServeMux
http.Handle("/",http.HandlerFunc(final))
Also can:
http.HandleFunc("/",final)

func(ResponseWriter,*Request)
ResponseWriter interface
typeResponseWriterinterface{
Header()Header
Write([]byte)(int,error)
WriteHeader(int)
}
Request struct
typeRequeststruct{
Methodstring
URL*url.URL
HeaderHeader
Bodyio.ReadCloser
ContentLengthint64
Hoststring
RemoteAddrstring
...
}

Demo: Customising Handlers - DRY Response Handling (1/3)
typeappHandlerstruct{
hfunc(http.ResponseWriter,*http.Request)(error)
}
func(ahappHandler)ServeHTTP(whttp.ResponseWriter,r*http.Request){
err:=ah.h(w,r)
iferr!=nil{
switcherr:=err.(type){
caseErrorDetails:
ErrorJSON(w,err)
default:
ErrorJSON(w,ErrorDetails{"InternalServerError","",500})
}
}
}
In app code we might extend this further:
Add error types and respond differently.
e.g. warn vs error-level log, send alerts, increment error metrics

typeErrorDetailsstruct{
Messagestring`json:"error"`
Detailsstring`json:"details,omitempty"`
Statusint`json:"-"`
}
func(eErrorDetails)Error()string{
returnfmt.Sprintf("Error:%s,Details:%s",e.Message,e.Details)
}
funcErrorJSON(whttp.ResponseWriter,detailsErrorDetails){
jsonB,err:=json.Marshal(details)
iferr!=nil{
http.Error(w,err.Error(),500)
return
}
w.Header().Set("Content-Type","application/json")
w.WriteHeader(details.Status)
w.Write(jsonB)
}

Use of special struct and special handler function to satisfy Handlerinterface
http.Handle("/",appHandler{unstableEndpoint})
Reduce repetition, extend functionality.
funcunstableEndpoint(whttp.ResponseWriter,r*http.Request)(error){
ifrand.Intn(100)>60{
returnErrorDetails{"Strangerequest","Pleasetryagain.",422}
}
ifrand.Intn(100)>80{
returnErrorDetails{"Seriousfailure","Weareinvestigating.",500}
}
w.Write([]byte(`{"ok":true}`))
returnnil
} Run

Demo: Customising Handlers - Avoiding Globals
Allows injecting dependencies rather than relying on global variables.
typeApistruct{
importantThingstring
//db*gorp.DbMap
//redis*redis.Pool
//logger...
}
typeappHandlerstruct{
*Api
hfunc(*Api,http.ResponseWriter,*http.Request)
}
func(ahappHandler)ServeHTTP(whttp.ResponseWriter,r*http.Request){
ah.h(ah.Api,w,r)
}
funcmyHandler(a*Api,whttp.ResponseWriter,r*http.Request){
w.Write([]byte("2015:Yearofthe"+a.importantThing))
} Run

Middleware: Why?
Abstract common functionality across a set of handlers
Bare minimum in Go:
func(nexthttp.Handler)http.Handler
Typical uses of middleware across languages/frameworks:
- logging
- authentication
- handling panic / exceptions
- gzipping
- request parsing

Demo: Middleware Example (Panic Recovery)
funcrecoveryMiddleware(nexthttp.Handler)http.Handler{
returnhttp.HandlerFunc(func(whttp.ResponseWriter,r*http.Request){
deferfunc(){
iferr:=recover();err!=nil{
log.Println("Recoverfromerror:",err)
http.Error(w,http.StatusText(500),500)
}
}()
log.Println("ExecutingrecoveryMiddleware")
next.ServeHTTP(w,r)
})
}
funcfinal(whttp.ResponseWriter,r*http.Request){
log.Println("ExecutingfinalHandler")
panic("walau!")
w.Write([]byte("OK"))
} Run

Demo: Chaining Middleware
funcmiddlewareOne(nexthttp.Handler)http.Handler{
log.Println("->ExecutingmiddlewareOne")
next.ServeHTTP(w,r)
log.Println("->ExecutingmiddlewareOneagain")
})
}
Calling chain of middleware
http.Handle("/",middlewareOne(middlewareTwo(http.HandlerFunc(final))))
funcmiddlewareTwo(nexthttp.Handler)http.Handler{
log.Println("--->ExecutingmiddlewareTwo")
next.ServeHTTP(w,r)
log.Println("--->ExecutingmiddlewareTwoagain")
})
} Run

Chaining Middleware - Alternate Syntax
3rd Party Library: Alice
Manages middleware with the standard function signature
Nice syntax for setting up chains used in different endpoints
chain:=alice.New(middlewareOne,middlewareTwo)
http.Handle("/",chain.Then(finalHandler))
Our example
noAuthChain:=alice.New(contextMiddleware,loggerMiddleware)
authChain:=alice.New(contextMiddleware,loggerMiddleware,apiKeyAuthMiddleware)
adminChain:=alice.New(contextMiddleware,loggerMiddleware,adminAuthMiddleware)

Demo: Creating Configurable Middleware
e.g. Pass the dependency on *AppLogger
varlogger*AppLogger=NewLogger()
loggerMiddleware:=simpleLoggerMiddlewareWrapper(logger)
http.Handle("/",loggerMiddleware(http.HandlerFunc(final)))
funcsimpleLoggerMiddlewareWrapper(logger*AppLogger)func(http.Handler)http.Handler{
returnfunc(nexthttp.Handler)http.Handler{
startTime:=time.Now()
next.ServeHTTP(w,r)
endTime:=time.Since(startTime)
logger.Info(r.Method+""+r.URL.String()+""+endTime.String())
})
}
} Run

Demo: Customising ResponseWriter (1/3)
typeResponseWriterinterface{
Header()http.Header
Write([]byte)(int,error)
WriteHeader(int)
}
ResponseWriter as an interface allows us to extend functionality easily
Example:
Step 1: Create a struct that wraps ResponseWriter
typeresponseWriterLoggerstruct{
w http.ResponseWriter
datastruct{
statusint
size int
}
}
Record data that would be otherwise be untracked.

Step 2: Define methods required for implicit satisfaction
func(l*responseWriterLogger)Header()http.Header{
returnl.w.Header()
}
func(l*responseWriterLogger)Write(b[]byte)(int,error){
//scenariowhereWriteHeaderhasnotbeencalled
ifl.data.status==0{
l.data.status=http.StatusOK
}
size,err:=l.w.Write(b)
l.data.size+=size
returnsize,err
}
func(l*responseWriterLogger)WriteHeader(codeint){
l.w.WriteHeader(code)
l.data.status=code
}

funcspecialLoggerMiddlewareWrapper(logger*AppLogger)func(http.Handler)http.Handler{
returnfunc(nexthttp.Handler)http.Handler{
startTime:=time.Now()
w2:=&responseWriterLogger{w:w}
next.ServeHTTP(w2,r)
logger.Info(r.Method+""+r.URL.String()+""+
time.Since(startTime).String()+
"status:"+strconv.Itoa(w2.data.status)+
"size:"+strconv.Itoa(w2.data.size))
})
}
} Run

Growing Middleware Ecosystem
Excerpt from Negroni Github page
graceful:(https://github.com/stretchr/graceful)graceful HTTP Shutdown
oauth2:(https://github.com/goincremental/negroni-oauth2)oAuth2 middleware
binding:(https://github.com/mholt/binding)data binding from HTTP requests into structs
xrequestid:(https://github.com/pilu/xrequestid)Assign a random X-Request-Id: header to each request
gorelic:(https://github.com/jingweno/negroni-gorelic)New Relic agent for Go runtime
Mailgun's Oxy
stream:(http://godoc.org/github.com/mailgun/oxy/stream)retries and buffers requests and responses
connlimit:(http://godoc.org/github.com/mailgun/oxy/connlimit)Simultaneous connections limiter
ratelimit:(http://godoc.org/github.com/mailgun/oxy/ratelimit)Rate limiter

Other Web Framework Components
Routing & Extracting URL Params
- standard library can be inflexible
- regex for extracting url params can feel too low level
- plenty of third party routers, e.g. Gorilla mux
funcShowWidget(whttp.ResponseWriter,r*http.Request){
vars:=mux.Vars(r)
teamIdStr:=vars["team_id"]
widgetIdStr:=vars["widget_id"]
...
}
Request-specific context
- sharing data between items in middleware chain and final handler
- solutions involve either global map, or per-request map/structs using custom
handlers/middleware

Web frameworks vs Build on top of standard library?
Time/expertise to build what you need? Too much re-inventing?
Your optimisation vs framework optimisation?
Performance? Does performance order of magnitude matter?
How much magic do you want?
Compatibility with net/http/ ecosystem? Framework interchangeability?
Martini -- 6.1k(https://github.com/go-martini/martini)
Revel -- 4.7k(https://github.com/revel/revel)
beego -- 3.7k(https://github.com/astaxie/beego)
goji -- 1.9k(https://github.com/zenazn/goji)
gin -- 1.9k(https://github.com/gin-gonic/gin)
negroni -- 1.8k(https://github.com/codegangsta/negroni)
go-json-rest -- 1.1k(https://github.com/ant0ine/go-json-rest)
Gorilla/mux -- 1.1k(https://github.com/gorilla/mux)
Tiger Tonic -- 0.8k(https://github.com/rcrowley/go-tigertonic)
Gocraft/web -- 0.6k(https://github.com/gocraft/web)

Thank you
Jonathan Gomez
Engineer, Zumata
jonathanbgomez@gmail.com(mailto:jonathanbgomez@gmail.com)
@jonog(http://twitter.com/jonog)

Customising Your Own Web Framework In Go

More Related Content

What's hot (20)

Viewers also liked (7)

Similar to Customising Your Own Web Framework In Go (20)

Recently uploaded (20)

Customising Your Own Web Framework In Go