Stanford 2014 Tech Entrepreneurship Slides - Talent, Opportunity, and Data @ LinkedIn
2 likes•317 views
Mario Rodriguez gave a presentation at Stanford on LinkedIn's talent recommendation products. He discussed how LinkedIn uses data on members, jobs, skills and companies to match talent to opportunities through products like TalentMatch and JYMBII. He also explained how LinkedIn predicts job transitions and migration patterns to improve recommendations. Finally, he briefly described how the economic graph maps the global economy using LinkedIn data.
Stanford 2014 Tech Entrepreneurship Slides - Talent, Opportunity, and Data @ LinkedIn
- 1. Talent, Opportunity, and Data @ LinkedIn Mario Rodriguez Senior Data Scientist and Team Lead Reputation – Recommendations – Search LinkedIn Confidential ©2014 All Rights Reserved E145 – Technology Entrepreneurship Stanford February 13, 2014
- 3. LinkedIn Confidential ©2013 All Rights Reserved 3 Members Worldwide 2 new Members Per Second 100M+Monthly Unique Visitors 225 M+ 2M+ Company Pages ...
- 4. LinkedIn Confidential ©2013 All Rights Reserved 4 Members Worldwide 2 new Members Per Second 100M+Monthly Unique Visitors 225 M+ 2M+ Company Pages ...
- 5. Some LinkedIn products (Agenda) JYMBII – Recommend opportunities to talent TalentMatch – Recommend talent to opportunities Job Search – Let talent find opportunities People Search – Let opportunities find talent Bonus: Economic Graph LinkedIn Confidential ©2013 All Rights Reserved 5
- 6. LinkedIn Confidential ©2013 All Rights Reserved 6 Members Worldwide 2 new Members Per Second 100M+Monthly Unique Visitors 225 M+ 2M+ Company Pages ...
- 7. Some LinkedIn products (Agenda) JYMBII – Recommend opportunities to talent TalentMatch – Recommend talent to opportunities Job Search – Let talent find opportunities People Search – Let opportunities find talent Bonus: Economic Graph LinkedIn Confidential ©2013 All Rights Reserved 7
- 8. Job Posting
- 10. 10 Corpus StatsJob User Base Filtered title geo company industry description functional area … Candidate General expertise specialties education headline geo experience Current Position title summary tenure length industry functional area … Similarity (candidate expertise, job description) 0.56 Similarity (candidate specialties, job description) 0.2 Transition probability (candidate industry, job industry) 0.43 Title Similarity 0.8 Similarity (headline, title) 0.7 Transition probabilities Connectivity yrs of experience to reach title education needed for this title … Interaction Features Binary Exact matches: geo, industry, … Soft transition probabilities, similarity, … Text Source/Target Features - Job Popularity - Job Seeking Propensity Find, Engage & Hire Jobs You May be Interested In
- 11. Find, Engage & Hire Jobs You May be Interested In
- 12. Open to relocation ? - Region similarity based on profiles or network - Region transition probability - predict individuals propensity to migrate and most likely migration target Impact on job recommendations Lift in views/viewers/applications/applicants Find, Engage & Hire Jobs You May be Interested In
- 13. 13 What would you transition to .. and when ? Months since graduation Probabilityofswitch Find, Engage & Hire Jobs You May be Interested In
- 14. Where are you likely to stay ? 14 Find, Engage & Hire Jobs You May be Interested In
- 15. Power of aggregation.. Before employees worked at Yahoo! (247) Google (139) Microsoft (105) Oracle (93) eBay (68) Before employees worked at Microsoft (1379) IBM (939) Yahoo! (608) Oracle (558) 15 Find, Engage & Hire Jobs You May be Interested In
- 16. Some LinkedIn products (Agenda) JYMBII – Recommend opportunities to talent TalentMatch – Recommend talent to opportunities Job Search – Let talent find opportunities People Search – Let opportunities find talent Bonus: Economic Graph LinkedIn Confidential ©2013 All Rights Reserved 16
- 17. 17 Real Time Talent Match
- 18. Job Seeker Intent Model • Propensity Score o Indicates receptiveness to new opportunities o p(switch jobs in next time period) • Model o Survival Analysis of Positions o Accelerated failure time (AFT) model log Ti = Σkβkxik+σεi ACTIVE PASSIVE NON-JOB- SEEKER
- 19. Connecting Talent to Opportunity Job Seeking Propensity Review of Survival Analysis is the time of death/event/purchase is the survival time Probability density distribution of event Survival function Hazards function
- 21. Tenure-based recommendations 1 million job applications over 5 years as training data – Different job transitions happen at different times – Transition to a higher position usually takes longer time – Various factors affect transition time – It matches with our motivation Connecting Talent to Opportunity Job Seeking Propensity time time
- 22. Job-Seeking Intent: actives & passives 16x reply rate on career-related mail Reply Rate Increase TalentMatch Utility fn(booking rate, email rate, reply rate) Connecting Talent to Opportunity Multi-Objective Optimization
- 23. Talent Match ranking Match Score 1, Item X, 0.98, Non-Seeker 2, Item Y, 0.91, Non-Seeker --------------------------------------- 3, Item Z, 0.89, Active Perturbed ranking Match Score, Perturbed Score 1, Item X, 0.98, 0.98, Non-Seeker 2, Item Z, 0.89, 0.93, Active ------------------------------------------------ 3, Item Y, 0.91, 0.91, Non-Seeker Perturbation Function f() Divergence Function Δ() Divergence score Objective Function g() Objective score Match Score Distributions Connecting Talent to Opportunity MOO How: Controlled Perturbation
- 24. Perturbation Function Divergence Function Objective Function Connecting Talent to Opportunity MOO
- 25. Loss Function Objective and divergence depend on a sort/rank, so gradient-based optimization not directly applicable Lambda value? Connecting Talent to Opportunity MOO
- 26. Pareto Optimization Connecting Talent to Opportunity MOO
- 27. 0 27 54 100 Match Score Histogram Divergence Connecting Talent to Opportunity MOO
- 28. Experiments A/B Test – Treatment 1: 1.15 boost (8/12) – Treatment 2: 1.07 boost (6/12) – Control: 1.0 boost (4/12) Expectations – 50% increase in reply rate for 1.07 boost – 100% increase in reply rate for 1.15 boost – Expected booking rate and email rate to remain unchanged or minimally affected Connecting Talent to Opportunity MOO
- 29. Booking rate α = β = 1.07 0% α = β = 1.15 -0.4% Email rate α = β = 1.07 31% α = β = 1.15 25% Reply rate α = β = 1.07 22% α = β = 1.15 42% Connecting Talent to Opportunity MOO
- 30. Some LinkedIn products (Agenda) JYMBII – Recommend opportunities to talent TalentMatch – Recommend talent to opportunities Job Search – Let talent find opportunities People Search – Let opportunities find talent Bonus: Economic Graph LinkedIn Confidential ©2013 All Rights Reserved 30
- 32. ©2013 LinkedIn Corporation. All Rights Reserved. 32 LinkedIn Data ? Human Evaluation Crowd LinkedIn’s Recruiters Machine Learning Rankings for every Skill … Product Usage Professional Reputation - Search
- 34. Some LinkedIn products (Agenda) JYMBII – Recommend opportunities to talent TalentMatch – Recommend talent to opportunities Job Search – Let talent find opportunities People Search – Let opportunities find talent Bonus: Economic Graph LinkedIn Confidential ©2013 All Rights Reserved 34
- 35. Economic graph Digitally map the global economy:people, jobs, skills, companies, and schools. Spot in real-time the trends pointing to economic opportunities. LinkedIn Confidential ©2013 All Rights Reserved 35
- 36. Economic graph LinkedIn Confidential ©2013 All Rights Reserved 36
- 37. LinkedIn Confidential ©2013 All Rights Reserved 37
- 39. Credits Engineering : Anmol Bhasin, Abhishek Gupta, Adam Smyczek, Adil Aijaz, Alan Li, Baoshi Yan, Bee-Chung Chen, Deepak Agarwal, Ethan Zhang, Haishan Liu, Igor Perisic, Jonathan Traupman, Liang Zhang, Lokesh Bajaj, Mario Rodriguez, Mitul Tiwari, Mohammad Amin, Monica Rogati, Parul Jain, Paul Ogilvie, Sam Shah, Sanjay Dubey, Tarun Kumar, Trevor Walker, Utku Irmak, Andrew Hill, Christian Posse, Gyanda Sachdeva, Mike Grishaver, Parker Barrile, Sachit Kamat, and many more… Alphabetically sorted