Understanding Feature Space in Machine Learning - Data Science Pop-up Seattle
23 likes6,931 views
Alice Zheng discusses the significance of feature space in machine learning, emphasizing that features are numeric representations of raw data that enable predictive modeling. The presentation covers techniques for representing natural text and images, highlights the importance of feature engineering, and introduces the concept of tf-idf as an improvement over the bag-of-words model. Additionally, Zheng touches on various machine learning topics, including model classification, clustering, and regression.
Understanding Feature Space in Machine Learning - Data Science Pop-up Seattle
- 1. #datapopupseattle Understanding Feature Space in Machine Learning Alice Zheng Director of Data Science, Dato RainyData Datoinc
- 2. #datapopupseattle UNSTRUCTURED Data Science POP-UP in Seattle www.dominodatalab.com D Produced by Domino Data Lab Domino’s enterprise data science platform is used by leading analytical organizations to increase productivity, enable collaboration, and publish models into production faster.
- 3. Understanding Feature Space in Machine Learning Alice Zheng, Dato October, 2015 3
- 4. ‹#› My journey so far Applied+machine+learning+ (Data+science) Build+ML+tools Shortage+of+experts+ and+good+tools.
- 5. ‹#› Why machine learning? Model data. Make predictions. Build intelligent applications.
- 6. ‹#› The machine learning pipeline I+fell+in+love+the+instant+I+laid+ my+eyes+on+that+puppy.+His+ big+eyes+and+playful+tail,+his+ soft+furry+paws,+… Raw+data Features Models Predictions Deploy+in+ production
- 7. ‹#› Three things to know about ML • Feature = numeric representation of raw data • Model = mathematical “summary” of features • Making something that works = choose the right model and features, given data and task
- 8. Feature = Numeric representation of raw data
- 10. ‹#› Representing natural text It#is#a#puppy#and#it#is# extremely#cute. Classify:++ puppy+or+not? Raw+Text Bag+of+Words it 2 they 0 I 0 am 0 how 0 puppy 1 and 1 cat 0 aardvark 0 cute 1 extremely 1 … … Sparse+vector+ representation
- 13. ‹#› Feature space in machine learning • Raw data ! high dimensional vectors • Collection of data points ! point cloud in feature space • Feature engineering = creating features of the appropriate granularity for the task
- 15. Crudely speaking, mathematicians fall into two categories: the algebraists, who find it easiest to reduce all problems to sets of numbers and variables, and the geometers, who understand the world through shapes. -- Masha Gessen, “Perfect Rigor”
- 16. ‹#› Visualizing bag-of-words puppy cute 1 1 I+have+a+puppy+and+ it+is+extremely+cute I#have#a#puppy#and# it#is#extremely#cute it 1 they 0 I 1 am 0 how 0 puppy 1 and 1 cat 0 aardvark 0 zebra 0 cute 1 extremely 1 … …
- 19. Model = Mathematical “summary” of features
- 20. ‹#› What is a summary? • Data ! point cloud in feature space • Model = a geometric shape that best “fits” the point cloud
- 25. ‹#› When does bag-of-words fail? puppy cat 2 1 1 have I+have+a+puppy I+have+a+cat I+have+a+kitten Task:+find+a+surface+that+separates++ documents+about+dogs+vs.+cats Problem:+the+word+“have”+adds+fluff++ instead+of+information I+have+a+dog+ and+I+have+a+pen 1
- 26. ‹#› Improving on bag-of-words • Idea: “normalize” word counts so that popular words are discounted • Term frequency (tf) = Number of times a terms appears in a document • Inverse document frequency of word (idf) = • N = total number of documents • Tf-idf count = tf x idf
- 27. ‹#› From BOW to tf-idf puppy cat 2 1 1 have I+have+a+puppy I+have+a+cat I+have+a+kitten idf(puppy)+=+log+4+ idf(cat)+=+log+4+ idf(have)+=+log+1+=+0 I+have+a+dog+ and+I+have+a+pen 1
- 28. ‹#› From BOW to tf-idf puppy cat1 have tfidf(puppy)+=+log+4+ tfidf(cat)+=+log+4+ tfidf(have)+=+0 I+have+a+dog+ and+I+have+a+pen,+ I+have+a+kitten 1 log+4 log+4 I+have+a+cat I+have+a+puppy Decision+surface Tf[idf+flattens+ uninformative+ dimensions+in+the+ BOW+point+cloud
- 29. ‹#› That’s not all, folks! • Geometry is the key to understanding feature space and machine learning • Many other fun topics: - Feature normalization - Feature transformations - Model regularization • Dato is hiring! [email protected] @RainyData,+@DatoInc
- 31. #datapopupseattle Thank You To Our Sponsors