Argument extraction from news, blogs and social media.
Download as PPTX, PDF3 likes458 views
The document discusses the challenges and methods of argument extraction from social media text, emphasizing its informal nature and lack of prior research in the area. It outlines a proposed two-step process for automatic argument extraction involving the identification of argumentative sentences and the subsequent extraction of claims and premises through various classifiers and conditional random fields. The study is based on a corpus of documents collected from social media focusing on renewable energy sources, and presents evaluation results demonstrating the effectiveness of the methodology.
![Existing methods● Palau et al. [4,7]
○ Classification at the sentence level by trying to identify possible argumentative sentences. Using NB, SVM, maximum
entropy.
○ Identify groups of sentences that refer to the same argument, using semantic distance based on the relatedness of
words contained
○ Detect clauses of sentences through a parsing tool, which are classified as argumentative or not with a maximum
entropy classifier
○ Argumentative clauses are classified into premises and claims through support vector machines
○ Araucaria corpus and ECHR corpus [11], achieving an accuracy of 73% and 80%
● A rule based system - Input an argumentation scheme and an ontology concerning an object, for example, a camera and its
characteristic features. Argumentation schemes are populated with discourse indicators, domain specific features and rules
are constructed.](https://image.slidesharecdn.com/md74w8mqhujrcsrvp7oj-signature-4adc640531daa19a55cb9e8840d0c3664dccd3a433a003015260f21da507ac44-poli-170127210703/85/Argument-extraction-from-news-blogs-and-social-media-5-320.jpg)
![Feature Selection for Corpus (contd.)
New Domain Specific Features:
Adjective number : Number of adjectives in a sentence .Usually in argumentation opinions are expressed towards an entity/claim,
through adjectives.
Entities in previous sentences: Number of entities in the nth previous sentence. History of n = 5 sentences, obtain five features,
Correlates to the probability that the current sentence contains an argument element.
Cumulative number of entities in previous sentences: Total number of entities from the previous n sentences. Considering a history
of n = 5 we obtain four features.
Ratio of distributions: Two Language models created from sentences that contain argument elements and from sentences that do
not contain an argument element. The ratio between these two distributions based on unigrams, bigrams and trigrams of
words. Can be described as :
Distributions over unigrams, bigrams, trigrams of part of speech tags (POS tags): Identical to [4] with the exception that unigrams,
bigrams and trigrams are extracted from the part of speech tags instead of words.](https://image.slidesharecdn.com/md74w8mqhujrcsrvp7oj-signature-4adc640531daa19a55cb9e8840d0c3664dccd3a433a003015260f21da507ac44-poli-170127210703/85/Argument-extraction-from-news-blogs-and-social-media-8-320.jpg)
Argument extraction from news, blogs and social media.
- 1. Argument Extraction from News, Blogs,and Social Media. Theodosis Goudas, Christos Louizos, Georgios Petasis, Vangelis Karkaletsis. Presented by : Sharath T.S Shubhangi Tandon
- 2. What is Argument Extraction? An argument can be usually decomposed into a claim and one or more premises justifying it. Task of identifying arguments along with their components in text Difficult even for humans to distinguish whether a part of a sentence contains an argument element or not
- 3. Why social media? ● Most widely used and accessible platform available to seek advice or express opinion ● Is a storehouse of both meaningful and meaningless information on social media about recent trends and topics. ● Almost no prior research in this field; only one publication related to product reviews on Amazon.
- 4. Why is this difficult ? ● Almost no prior research in this field; only one publication related to product reviews on Amazon! ● Text from social media may not always contain arguments ● Expressed in an informal form, and they do not follow any formal guidelines or specific rules ● Absence of widely used corpora in order to comparably evaluate approaches for argument extraction. ● Traditional research in the area, concentrates mainly on law documents and scientific publications.
- 5. Existing methods● Palau et al. [4,7] ○ Classification at the sentence level by trying to identify possible argumentative sentences. Using NB, SVM, maximum entropy. ○ Identify groups of sentences that refer to the same argument, using semantic distance based on the relatedness of words contained ○ Detect clauses of sentences through a parsing tool, which are classified as argumentative or not with a maximum entropy classifier ○ Argumentative clauses are classified into premises and claims through support vector machines ○ Araucaria corpus and ECHR corpus [11], achieving an accuracy of 73% and 80% ● A rule based system - Input an argumentation scheme and an ontology concerning an object, for example, a camera and its characteristic features. Argumentation schemes are populated with discourse indicators, domain specific features and rules are constructed.
- 6. Proposed Method The proposed Automatic Argument Extraction is a two step process : Step A : Identification of Argumentative Sentences (Supervised Classification using standard classifiers : Logistic regression, Random Forest, Support Vector Machines, Naive Bayes) Step B :Extraction of Claims and Premises (From output of Step A , using Conditional Random Fields)
- 7. Feature Selection for Corpus State of the art features: Position Comma Token Number Connective Number Verb Number Word Number Cue words # verbs in passive voice Domain Entities Number Adverb Number Word Mean Length
- 8. Feature Selection for Corpus (contd.) New Domain Specific Features: Adjective number : Number of adjectives in a sentence .Usually in argumentation opinions are expressed towards an entity/claim, through adjectives. Entities in previous sentences: Number of entities in the nth previous sentence. History of n = 5 sentences, obtain five features, Correlates to the probability that the current sentence contains an argument element. Cumulative number of entities in previous sentences: Total number of entities from the previous n sentences. Considering a history of n = 5 we obtain four features. Ratio of distributions: Two Language models created from sentences that contain argument elements and from sentences that do not contain an argument element. The ratio between these two distributions based on unigrams, bigrams and trigrams of words. Can be described as : Distributions over unigrams, bigrams, trigrams of part of speech tags (POS tags): Identical to [4] with the exception that unigrams, bigrams and trigrams are extracted from the part of speech tags instead of words.
- 9. Step B: Argument Extraction with CRFWhy Conditional Random Fields ? Structured prediction algorithm Can take local context into consideration ( help maintain linguistic aspects such as the word ordering in the sentence) Features: The words in these sentences Gazetteer lists of known entities for the thematic domain related to the arguments we want to extract, Gazetteer lists of cue words and indicator phrases Lexica of verbs and adjectives automatically acquired using Term Frequency - Inverse Document Frequency (TF-IDF) between two “documents” ( With and without argumentative text from Step A )
- 10. Corpus Preparation ● 204 documents (in Greek) collected from the social media ● Thematic domain of Renewable Energy Sources ● Selected documents were manually annotated with domain entities and text segments that correspond to argument premises. ● Claims are not represented into documents as segments, but implied by the author as positive or negative views 760 sentences: Annotated as containing arguments 16000 sentences from 204 documents Final Output Ellogon Step A Step B
- 11. Evaluation : Base Case Simple base case classifier: 1. Manually annotated segments (argument components) used to form a gazetteer. 2. Applied on the corpus in order to detect all exact matches of all these segments. a. All segments identified are marked as argumentative segments b. All sentences that contain at least one argumentative segment identified by the gazetteer, are characterised as an argumentative sentence. 3. Argumentative segments/sentences are compared to “gold” counterparts, manually annotated by humans. a. Sentences that contain these recognized fragments are marked as argumentative for the first step base case. b. Segments marked as argumentative are evaluated for the second step base case. 4. Results are taken through 10-fold cross validation on the whole corpus (all 16k sentences)
- 12. Evaluation : Step A Each sentence represented as a fixed-size vector using features described (including class - Supervised learning ) Tested against classifiers such as : Support Vector Machines, Naive Bayes, Random Forest and Logistic Regression. Initial Data set is heavily skewed towards non-argumentative documents . Therefore, Data Sampling and Testing was done in two different ways : Use Precision , Recall , F-1 Measure and Accuracy for Evaluation Logistic Regression and Naive Bayes performed the best Way #1 Way #2 Sampling Randomly ignore negative examples. Result set contains equal number of instances from both classes Split Initial Data set in the ratio 70:30 for testing and training Evaluation 10-fold cross validation, achieved high accuracy Achieved 49% accuracy , Discarded
- 13. Evaluation : Step B To use CRF, need BIO tagging for sentences: B for starting a text segment (premise), I for a token in a premise other than the first, and O for all other tokens (outside of the premise segment) Example for “Wind turbines generate noise in the summer” Final Result after CRF Baseline Results
- 14. What did we think ?
- 16. Appendix
- 18. Evaluation Results :Step A (contd) Go back
Editor's Notes
- #11: The corpus was constructed by manually filtering a larger corpus, automatically collected by performing queries on popular search engines (such as Bing2 ), Google Plus 3 , Twitter 4 , and by crawling sites from a list of sources relevant to the domain of renewable energy.