Scala(finagle)@SmartNews_English
8 likes•3,048 views
This document summarizes how SmartNews built a high performance ad server using Finagle to handle requests asynchronously. Key points include: - The ad server was built in about 2 months using Finagle, an RPC framework from Twitter that uses asynchronous handling with Futures. - Finagle allows building the server as a function that takes a request and returns a response asynchronously. It provides high performance backed by Netty and metrics via Tracer. - The ad server implementation uses JSON parsing, data loading from a database, campaign filtering, and response generation in an asynchronous chained Future pattern for high throughput. - Tips are provided for async handling, caching data, and focusing on IO-bound rather than CPU
![Server as a function
• Server : Request => Response
!
class Service[-Req, +Rep] extends (Req => Future[Rep])](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-15-320.jpg)
![Ad Service
• Launches ListeningServer(backed by Netty)
on `:port`
!
class AdService extends Service[Request, Response] {
override def apply(request: Request): Future[Response] = {
// hogehoge
}
}
val server = Http.serve(":port", new AdService())
Await.result(server)
* caution: This code needs transformation of Request to HttpRequest
(also Response)](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-16-320.jpg)
![JSON Parse
• Jackson Streaming API
• AdRequest is defined in .thrift file
!
trait JsonParser {
def read(request: Request):Option[AdRequest]
}](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-17-320.jpg)
![Data Load
• Future type indicates the function causes
heavy IO task
!
trait ContextLoader {
def load(): Future[CampaignContext]
}](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-18-320.jpg)
![Data Load
• Campaign Master Info is stored in MySQL,
and be cached on JVM memories.
• Use AtomicReference to ConcurrentHashMap
as cache
!
trait ContextLoader {
def load(): Future[CampaignContext]
}](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-19-320.jpg)
![Campaign Filtering
• Filtering campaigns based on Context and
Request
!
trait CampaignFilter {
def filter(
request: AdRequest,
ctx: CampaignContext
): Option[AdResponse]
}](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-20-320.jpg)
: Future[B]](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-21-320.jpg)
![Future Chain (flatMap)
!
def apply(request: Request): Future[Response] = {
val adReqOpt: Option[AdRequest] = JSONParser.read(request)
val fAdResOpt: Future[Option[AdResponse]] = adReqOpt.map { adReq =>
val fCtx: Future[CampaignContext] = ContextLoader.load()
fCtx.map { ctx =>
CampaignFilter.filter(adReq, ctx)
}
}.getOrElse {
Future.value(None)
}
!](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-22-320.jpg)
![Future Chain (optionT)
• optionT[A]: Future[Option[A]] => OptionT[Future, A]
• need to make Future as instance of Monad (gist)
!
def apply(request: Request): Future[Response] = {
(for {
adReq <- optionT(JSONParser.read(request).point[Future])
ctx <- LocalCache.load().liftM[OptionT]
adRes <- optionT(CampaignFilter.filter(adReq, ctx).point[Future])
sessionKey <- optionT(RedisService.getKey())
} yield mkResponse(adRes, sessionKey)).run.map {
_.getOrElse(Response(NO_CONTENT))
}
}
!](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-24-320.jpg)
,
DefaultTimer.twitter){
}
!
Http.serve(":port", new HttpTimeoutFilter(100) andThen AdService)
!](https://image.slidesharecdn.com/scalafinaglesmartnewsen-150305093412-conversion-gate01/85/Scala-finagle-SmartNews_English-25-320.jpg)
Scala(finagle)@SmartNews_English
- 1. SCALA @ SMARTNEWS High Performance Ad Server with Finagle @taketon_
- 2. Who? • @taketon_ • Ph.D in Psychology(Vision) • TypeSafe and Beer Lover
- 3. SmartNews • Mobile News App, which uses machine learning algorithms to discover and aggregate news • 10 million DL globally
- 4. Chikyukun
- 5. Ad • Started Ad delivery(only in Japan) from 2014/Dec • Policy: Ads as Content
- 6. Team members
- 7. Ad Server • Implemented within about 2 months
- 9. Finagle • OSS RPC Framework by Twitter, Inc. • Using Future like Akka • Wrapper of Netty • "Server as a function"
- 10. Finagle • We use Finagle for Movie Ad Server • avg. 20ms latency / Max 2000qps
- 11. Pros • easy to build server • High Performance (backed by Netty) • Async Handling by Future type • shared model by thrift IDL • provides metrics via twitter-server and Tracer
- 12. Cons • slightly slow to update (e.g. upgrade to Scala 2.11) • (waiting for finagle-redis update) • Different Behavior between twitter.util.Future and scala.concurrent.Future (pop-quiz) • limited documentation • not a good fit with NewRelic (instead, use Tracer)
- 13. How to implement Ad Server
- 14. Handling Async • Type as a document • Future indicates IO call • Procedures in AdServer • Request -> JSON parse -> Data load -> Campaign Filtering -> Response
- 15. Server as a function • Server : Request => Response ! class Service[-Req, +Rep] extends (Req => Future[Rep])
- 16. Ad Service • Launches ListeningServer(backed by Netty) on `:port` ! class AdService extends Service[Request, Response] { override def apply(request: Request): Future[Response] = { // hogehoge } } val server = Http.serve(":port", new AdService()) Await.result(server) * caution: This code needs transformation of Request to HttpRequest (also Response)
- 17. JSON Parse • Jackson Streaming API • AdRequest is defined in .thrift file ! trait JsonParser { def read(request: Request):Option[AdRequest] }
- 18. Data Load • Future type indicates the function causes heavy IO task ! trait ContextLoader { def load(): Future[CampaignContext] }
- 19. Data Load • Campaign Master Info is stored in MySQL, and be cached on JVM memories. • Use AtomicReference to ConcurrentHashMap as cache ! trait ContextLoader { def load(): Future[CampaignContext] }
- 20. Campaign Filtering • Filtering campaigns based on Context and Request ! trait CampaignFilter { def filter( request: AdRequest, ctx: CampaignContext ): Option[AdResponse] }
- 21. Future Chain • add callback to the function with flatMap • callback will receive the result of former Future function, then returns new Future ! def flatMap[B](f: A => Future[B]): Future[B]
- 22. Future Chain (flatMap) ! def apply(request: Request): Future[Response] = { val adReqOpt: Option[AdRequest] = JSONParser.read(request) val fAdResOpt: Future[Option[AdResponse]] = adReqOpt.map { adReq => val fCtx: Future[CampaignContext] = ContextLoader.load() fCtx.map { ctx => CampaignFilter.filter(adReq, ctx) } }.getOrElse { Future.value(None) } !
- 23. Future Chain (flatMap) ! fAdResOpt.flatMap { adResOpt => adResOpt.map { adRes => val fSessionKeyOpt = RedisService.getKey() fSessionKeyOpt.flatMap { sessionKeyOpt => sessionKeyOpt.map { sessionKey => Future.value(mkResponse(adRes, sessionKey)) }.getOrElse { Future.value(Response(NO_CONTENT)) } } }.getOrElse { Future.value(Response(NO_CONTENT)) } } }
- 24. Future Chain (optionT) • optionT[A]: Future[Option[A]] => OptionT[Future, A] • need to make Future as instance of Monad (gist) ! def apply(request: Request): Future[Response] = { (for { adReq <- optionT(JSONParser.read(request).point[Future]) ctx <- LocalCache.load().liftM[OptionT] adRes <- optionT(CampaignFilter.filter(adReq, ctx).point[Future]) sessionKey <- optionT(RedisService.getKey()) } yield mkResponse(adRes, sessionKey)).run.map { _.getOrElse(Response(NO_CONTENT)) } } !
- 25. Filter • Set filter for application-agnostic behavior, like timeout class HttpTimeoutFilter ( val timeOutMs: Int ) extends TimeoutFilter[Request, Response]( timeOutMs.milliseconds, new GlobalRequestTimeoutException(timeOutMs.milliseconds), DefaultTimer.twitter){ } ! Http.serve(":port", new HttpTimeoutFilter(100) andThen AdService) !
- 26. Routing val muxer = new HttpMuxer() .withHandler("/ad", new AdService()) ! val server = Http.server(":port", muxer) Await.result(server)
- 27. Tips for Performance • Basically the server will not be CPU-bound • Wrap IO-bound process by Future
- 28. Tips for Performance • The cost for creating instances for Future or Option can be ignored • For constructing List, use mutable data structure (e.g. ArrayBuffer) and finally call toList • For transforming List, use Iterator and finally call toList • Practice for Performance tuning in Scala (Japanese)
- 29. Summary • Finagle will be a good fit for Ad Server • We used mutable for data construction • However, no need for sensitive for tuning • (́-`).｡oO(Hope Finagle will become more popular)