SlideShare i18n
Moving off Smartling
What’s all this about?
SlideShare was using a third party to serve localized pages at runtime
This was identified as a big security loophole by internal security team
Why security had a problem?
What’s the fix?
Move to internal localization stack.
Simple
Isn’t It?
Scale of problem
75 Million active users
~700 templates
LinkedIn and SlideShare stack differences
as SlideShare is built on Ruby
SlideShare web app repo is not a Multiproduct
Xmessage format is only supported format by LI i18n infra
Ruby doesn’t supports .properties files natively
SlideShare’s old version of ruby isn’t very encoding friendly
No native unicode escape support
No use of languages packs before
……….
So How to do it?
Initial Plan
Create some magic solution to crawl all code, find strings, generate
properties and send to i18n team
What would it lead to?
Confused Engineers
Missed strings
Lower quality extraction and translation
missing all goodies like pluralizations,
names and dates formatters and more.
So What did we do?
Started a POC on an [In]Day to verify the cost of creating Xmessage in Ruby
Agreed on manual extraction to ensure high quality
Decided to write custom properties file parser and custom i18n backend to use
them
From
To
Main Focus points
No resistance should be added in development process
Minimum or no new method calls
Keep response time unaffected
No significant delays in shipping code to production.
And remove Smartling ASAP as it’s a security issue
Main Focus points
No resistance should be added in development process
Minimum or no new method calls
Keep response time unaffected
No significant delays in shipping code to production.
And remove Smartling ASAP as it’s a security issue
No resistance in dev process
Created New repo for storing translations but tied it up with Main repo using Git
Hooks.
Pre-rebase and post-merge hooks ensure your translations are latest on dev
without any need to rebase
Main Focus points
No resistance should be added in development process
Minimum or no new method calls
Keep response time unaffected
No significant delays in shipping code to production.
And remove Smartling ASAP as it’s a security issue
No New method calls added
Everything happens behind I18n.t
Fetch from properties, Xmessage interpolation, Name, Date formatting all
behind the same method call
Main Focus points
No resistance should be added in development process
Minimum or no new method calls
Keep response time unaffected
No significant delays in shipping code to production.
And remove Smartling ASAP as it’s a security issue
Xmessage Benchmarking
Generally a page has ~10
strings with placeholders
Main Focus points
No resistance should be added in development process
Minimum or no new method calls
Keep response time unaffected
No significant delays in shipping code to production.
And remove Smartling ASAP as it’s a security issue
Need to create two RB’s
Need to push properties files separately but no deploy step for translations repo
keeps the process smooth
How 700 templates were localized?
Created detailed documentation for team
Conducted a Hands-on/demo to teach devs new workflow
Created ~700 tickets using JIRA API
Did two string extraction bashes and rewarded whoever localized the most
Most of the issues created by L10n team were resolved within two days
Created detailed documentation
The team was new to this stack
Never created properties files before
Never used Xmessage placeholders before
New flow for pushing translations
Best Practices
Involved L10n team at every step
In Documentation Reviews
L10n engineering team reviewed our documentation and suggested needed edits
before we released it to team
Code Reviews by L10n team
Did our code reviews - Masha’s review rate was much higher than mine
Code Reviews by L10n team helped us catch errors much early and iterate
faster
Every code review taught us best practices and raised the quality bar amongst
the team
Decided on complex patterns much earlier
user.message={:type,map,MEMBER#Hello, {:value,name,full}|COMPANY#Hello,
{:value}|DEFAULT_TEXT#Hello, {:user_login}}
Avoid XSS
At every step ensure that you don’t insert broken HTML and take special care of
quotes
Do it before security finds it
Write tons of tests
80 UITF tests for EN
~400 Tests for Xmessage
40 tests for new i18n backend
Added 80 for each of the languages which SlideShare supports
Be Available
Most of the queries resolved on slack
Most of the discussions done on slack
Resolve issues as soon as they come
Most of the issues resolved within 1-2 days to maintain the pace
The Timeline
Questions

i18n tech talk

  • 1.
  • 2.
    What’s all thisabout? SlideShare was using a third party to serve localized pages at runtime This was identified as a big security loophole by internal security team
  • 3.
    Why security hada problem?
  • 4.
    What’s the fix? Moveto internal localization stack. Simple Isn’t It?
  • 5.
    Scale of problem 75Million active users ~700 templates LinkedIn and SlideShare stack differences as SlideShare is built on Ruby
  • 6.
    SlideShare web apprepo is not a Multiproduct Xmessage format is only supported format by LI i18n infra Ruby doesn’t supports .properties files natively SlideShare’s old version of ruby isn’t very encoding friendly No native unicode escape support No use of languages packs before ……….
  • 7.
    So How todo it? Initial Plan Create some magic solution to crawl all code, find strings, generate properties and send to i18n team
  • 8.
    What would itlead to? Confused Engineers Missed strings Lower quality extraction and translation missing all goodies like pluralizations, names and dates formatters and more.
  • 9.
    So What didwe do? Started a POC on an [In]Day to verify the cost of creating Xmessage in Ruby Agreed on manual extraction to ensure high quality Decided to write custom properties file parser and custom i18n backend to use them
  • 10.
  • 11.
    Main Focus points Noresistance should be added in development process Minimum or no new method calls Keep response time unaffected No significant delays in shipping code to production. And remove Smartling ASAP as it’s a security issue
  • 12.
    Main Focus points Noresistance should be added in development process Minimum or no new method calls Keep response time unaffected No significant delays in shipping code to production. And remove Smartling ASAP as it’s a security issue
  • 13.
    No resistance indev process Created New repo for storing translations but tied it up with Main repo using Git Hooks. Pre-rebase and post-merge hooks ensure your translations are latest on dev without any need to rebase
  • 14.
    Main Focus points Noresistance should be added in development process Minimum or no new method calls Keep response time unaffected No significant delays in shipping code to production. And remove Smartling ASAP as it’s a security issue
  • 15.
    No New methodcalls added Everything happens behind I18n.t Fetch from properties, Xmessage interpolation, Name, Date formatting all behind the same method call
  • 16.
    Main Focus points Noresistance should be added in development process Minimum or no new method calls Keep response time unaffected No significant delays in shipping code to production. And remove Smartling ASAP as it’s a security issue
  • 17.
    Xmessage Benchmarking Generally apage has ~10 strings with placeholders
  • 18.
    Main Focus points Noresistance should be added in development process Minimum or no new method calls Keep response time unaffected No significant delays in shipping code to production. And remove Smartling ASAP as it’s a security issue
  • 19.
    Need to createtwo RB’s Need to push properties files separately but no deploy step for translations repo keeps the process smooth
  • 20.
    How 700 templateswere localized? Created detailed documentation for team Conducted a Hands-on/demo to teach devs new workflow Created ~700 tickets using JIRA API Did two string extraction bashes and rewarded whoever localized the most Most of the issues created by L10n team were resolved within two days
  • 21.
    Created detailed documentation Theteam was new to this stack Never created properties files before Never used Xmessage placeholders before New flow for pushing translations
  • 22.
  • 23.
    Involved L10n teamat every step
  • 24.
    In Documentation Reviews L10nengineering team reviewed our documentation and suggested needed edits before we released it to team
  • 25.
    Code Reviews byL10n team Did our code reviews - Masha’s review rate was much higher than mine Code Reviews by L10n team helped us catch errors much early and iterate faster Every code review taught us best practices and raised the quality bar amongst the team
  • 26.
    Decided on complexpatterns much earlier user.message={:type,map,MEMBER#Hello, {:value,name,full}|COMPANY#Hello, {:value}|DEFAULT_TEXT#Hello, {:user_login}}
  • 27.
    Avoid XSS At everystep ensure that you don’t insert broken HTML and take special care of quotes Do it before security finds it
  • 28.
    Write tons oftests 80 UITF tests for EN ~400 Tests for Xmessage 40 tests for new i18n backend Added 80 for each of the languages which SlideShare supports
  • 29.
    Be Available Most ofthe queries resolved on slack Most of the discussions done on slack
  • 30.
    Resolve issues assoon as they come Most of the issues resolved within 1-2 days to maintain the pace
  • 31.
  • 32.