Classification of Code-Mixed Bilingual Phonetic Text Using Sentiment Analysis

DOI: 10.4018/IJSWIS.2021040104
International Journal on Semantic Web and Information Systems
Volume 17 • Issue 2 • April-June 2021

Copyright © 2021, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.

59
Classification of Code-Mixed Bilingual
Phonetic Text Using Sentiment Analysis
Shailendra Kumar Singh, Computer Science and Engineering, Sant Longowal Institute of Engineering and Technology,
India
https://orcid.org/0000-0001-9658-1441
Manoj Kumar Sachan, Computer Science and Engineering, Sant Longowal Institute of Engineering and Technology, India
ABSTRACT
The rapid growth of internet facilities has increased the comments, posts, blogs, feedback, etc., on a
large scale on social networking sites. These social media data are available in an unstructured form,
which includes images, text, and videos. The processing of these data is difficult, but some sentiment
analysis, information retrieval, and recommender systems are used to process these unstructured data.
To extract the opinion and sentiment of internet users from their written social media text, a sentiment
analysis system is required to develop, which can work on both monolingual and bilingual phonetic text.
Therefore, a sentiment analysis (SA) system is developed, which performs well on different domain
datasets. The system performance is tested on four different datasets and achieved better accuracy of
3% on social media datasets, 1.5% on movie reviews, 1.35% on Amazon product reviews, and 4.56%
on large Amazon product reviews than the state-of-art techniques. Also, the stemmer (StemVerb) for
verbs of the English language is proposed, which improves the SA system’s performance.
Keywords
Code-Mixed Phonetic Text, Opinion Verb, Sentiment Analysis, Sentiment Score, Social Media, Stemmer
1. INTRODUCTION
With the enhancement of internet services and facilities, social networking sites such as YouTube,
Google Plus, LinkedIn, Twitter, and Facebook have increased rapidly (Press Trust of India, 2013).
These social networking sites provide facilities to share the users’ feelings, emotions, comments,
feedbacks, and reviews over the internet. Thus, the size of such content over social media (SM)
increases exponentially day by day. Most of the SM text contents are written using more than one
language and called code-mixed language. The text of languages other than English is written using
Roman script’s alphabets called phonetic text. The phonetic text mixed with English language text, but
there is no fixed format for these SM texts (Dutta et al., 2015). These contents are used as input text to
extract information, opinion, text summarization, etc., using various linguistic computations, natural
language processing, text mining, and information retrieval systems (S. K. Singh & Sachan, 2019b).
Opinion mining or sentiment analysis (SA) is a sub-field of text mining and is one of the most
recent research topics of interest (Pang & Lee, 2008). The SA is related to predicting and analyzing
hidden information, emotion, and feelings from the written text. The SA is widely used to analyze
feedbacks on government regulation and policy proposed, to analyze the customers’ likes/dislikes,
to know the product demand, brand reputation, real-world event monitoring and analyzing of

60
political party demand, competitors products’ merit or demerit analyzes, and subtask component of
recommender system (Bonadiman et al., 2017; D’Andrea et al., 2015). In April 2013, 90% of consumers
decided to purchase things or services based on online reviews (Peng et al., 2014).
The user-written texts are classified using SA into two or three classes based on different formats
such as positive/negative/neutral, like/dislike, and good/bad (Hopken et al., 2017; S. K. Singh &
Sachan, 2019b). Mostly two approaches are used to classify the text using SA, such as (i) feature-
based and (ii) bag-of-words. Machine learning techniques are based on features, while lexicon-based
techniques use bag-of-words approaches. In SA of products and services, machine learning is widely
used, but the bag-of-words are used for social issues (Karamibekr & Ghorbani, 2012). Machine learning
systems are trained on the labeled dataset(s) and classify the testing dataset(s) based on the trained
system. The lexicon-based techniques are categorized as two approaches such as corpus-based and
dictionary-based (S. K. Singh & Sachan, 2019b). The text classification using the dictionary-based
approach depends upon the opinion word’s sentiment score. This dictionary consists of opinion words
and their sentiment scores (Alharbi & Alhalabi, 2020). The opinion words are nouns, verbs, adverbs,
and adjectives, which act as features in dictionary-based approach as discussed in the articles (Hopken
et al., 2017; Shamsudin et al., 2016; P. K. Singh et al., 2015; R. K. Singh et al., 2020; S. K. Singh &
Sachan, 2019b). These opinion words are used to develop opinion dictionaries such as SentiWordNet
3.0 (Baccianella et al., 2010), the latest SenticNet 4 (Cambria et al., 2016), etc.
The SM text is available in monolingual, bilingual, and multilingual. The processing of
multilingual texts is a difficult task as compared to bilingual and monolingual text. Monolingual
text can be easily processed, but bilingual text up to some extent. Taboada et al. (2011) developed
“Semantic Orientation CALculator (SO-CAL)” using a dictionary, which includes opinion words
(nouns, adverb, verbs, and adjectives) along with their polarity and strength value (Taboada et al.,
2011) and achieved the best performance in term of the accuracy 70.10% on movie dataset among
other datasets. In 2012, Karamibekr & Ghorbani developed a sentiment classification system in which
verbs are considered the core element of the system and created an opinion verb dictionary of 440
verbs and 1726 terms for the term opinion dictionary. The value of polarity is assigned to each word
in the dictionary ranging from +2 to -2, and their system was tested on a dataset related to social
issues with an accuracy of 65% (Karamibekr & Ghorbani, 2012). P. K. Singh et al., (2015) used
negation handling rules and a dictionary-based approach to classify the social issues related dataset
into positive or negative class and achieved an accuracy of 79.16% for negative sentences. Iqbal et
al., (2015) proposed a Bias-aware thresholding (BAT) method with the combination of AFINN and
SentiStrength to reduce the bias in the lexicon-based method for SA and obtained 69% of accuracy
using Naïve Bayes classifier.
In 2016, Bhargava et al. developed a system of SA for code-mixed sentences of the English
language with four Indian languages (Hindi, Telugu, Bengali, and Tamil) using the count-based
approach and language-specific SentiWordNets to determine the word’s polarity value. Their system
classified sentences based on the number of negative and positive words present within the sentence
and obtained 54.4% of F score on English_Hindi mixed dataset. In the same year, a lexical-based
method was proposed by (Shamsudin et al., 2016) to classify the Facebook comments written in
Malay language using the verb, adverb, and negation. The Verb + Negation combination achieved
the highest accuracy of 52.12%.
In 2017, a SA system for a specific domain text was proposed by (Cruz et al., 2017), using a
dictionary-based approach. The system was tested on different datasets such as kitchen, book, movie
reviews, and agriculture; and achieved the best performance on movie reviews (F score of 66%). A
multi-task tri-training model based on deep learning was proposed by (Ruder & Plank, 2018). They
have trained their model with three model-specific outputs jointly and achieved 79.15% accuracy
on Amazon product reviews. Later, in 2018 Han et al. proposed a method to reduce the bias in the
lexicon-based approach. They used SentiWordNet and tried to improve accuracy but achieved 69.52%
accuracy and 69.65% F score (Han et al., 2018). Baoxin Wang (2018) proposed a deep learning-based

61
Disconnected Recurrent Neural Networks model, and Yu & Liu (2018) proposed Sliced Recurrent
Neural Networks model. They have tested their models on large Amazon product reviews, but their
system achieved less than 65.00% accuracy.
SentiVerb system was developed by (S. K. Singh & Sachan, 2019b) and tested on GST dataset
and movie reviews. The system considered verb as a feature, and obtained 82.50% accuracy on the
GST dataset and 71.30% accuracy on movie reviews even with small dataset availability. In 2020,
an event named SemEval-2020 Task 9 was conducted on SA of code-mixed tweets, and the highest
performance was reported as 75.00% F score by one participant on bilingual (English_Hindi) text
using XLM-R method (Patwa et al., 2020). In 2020, a method to automatically label the document
with polarity or sentiment class was proposed and reduce human intervention, process time, and cost
(Kansal et al., 2020). Their polarity detection task, along with the sentiment classification system,
achieved 81.98% accuracy on Amazon product reviews using a logistic regression approach.
It is observed that most of the research work has been conducted on SA/opinion mining for
monolingual language text instead of bilingual and multilingual text for resource-scarce languages
(Lo et al., 2017) as per Table 1 and recent survey articles (Drus & Khalid, 2019; Guellil & Boukhalfa,
2015; Hussein, 2018; Medhat et al., 2014; Ravi & Ravi, 2015; Serrano-Guerrero et al., 2015; R. K.
Singh et al., 2020; Yue et al., 2019). The highest accuracy on SM dataset is 82.5%; 71.3% on movie
reviews (S. K. Singh & Sachan, 2019b); 81.98% on Amazon product reviews (Kansal et al., 2020)
and 64.43% on large Amazon product reviews. Most of the studies other than English language are
in Chinese, Japanese, German, Spanish, French, Italian, Swedish, Arabic, and Romanian (Lo et al.,
2017). Therefore, a smaller number of studies and experiments have been conducted for bilingual and
multilingual text and have not shown adequate system performance. There is no work found related to
the sentiment classification of code-mixed bilingual (English and Punjabi) phonetic text. The machine
learning techniques require a large amount of labeled dataset(s) for training purposes, and code-mixed
phonetic text dataset(s) is not available sufficiently. Hence, there is a need to design and develop a SA
system, which can give better performance for both monolingual and bilingual code-mixed phonetic
text. With this perspective, a system is designed and developed for SA of bilingual (English and
Punjabi) code-mixed phonetic text using a rule-based classifier and dictionary-based approach. The
first time the verb opinion dictionary (VOD) for Punjabi words is developed with the motivation of
the opinion verb dictionary (OVD) of the English language (S. K. Singh & Sachan, 2019b).
The significant contributions are: First time bilingual (English & Punjabi) code-mixed phonetic
texts are considered for SA. In this article, a sentiment analysis system is proposed to classify code-
mixed bilingual (English_Punjabi) phonetic text using handcrafted rules, rule-based classifiers,
and dictionaries. The bilingual code-mixed testing datasets (86,400 reviews) are generated from
monolingual text. The spell checker, VOD, negation words, positive words, negative prefixes, negation
handling rules, stop words, and StemVerb system is developed and used to extract the writer’s sentiment
from their written text. The VOD of the Punjabi and English language is developed, which includes
653 words of Punjabi and 677 English words. The root verb list of 3582 words and an irregular verb
list of 222 words is developed. The StemVerb system (Sub-system of SA system) is proposed to extract
the root verb from the inflected form of the English language verb and obtain 22.83% better accuracy
than the existing snowball stemmer system. The proposed SA system achieved better performance
than state-of-art approaches on all four benchmark datasets, for monolingual text (better accuracy of
3% on SM dataset, 1.5% on movie reviews, 1.35% on Amazon product reviews, and 4.56% on large
Amazon product reviews).
This article is organized as follows: Section 1 describes the introduction of SA and the idea about
previous research works on SA. Section 2 provides detailed information about the proposed framework
of the system and an explanation of the different phases for the implementation of the system. Section
3 is related to the performance evaluation of the proposed system on the different dataset(s) and
result analysis along with the comparison of performance with state-of-the-art techniques. Finally,
a brief detailed conclusion of this article is provided along with the future scope in the last section.

62
2. PROPOSED SYSTEM FRAMEWORK AND IMPLEMENTATION
The proposed system classifies the code-mixed phonetic text into positive or negative using a
dictionary-based approach. This system considered verb words as a feature. This system has been
Table 1. Analysis of existing works related to SA
Authors Method
Accuracy
(%)
F Score
(%)
Limitations
(Taboada et al., 2011) Lexicon-based 70.10
● Small data size (5100)
● Only for English language text
(Karamibekr &
Ghorbani, 2012)
Dictionary-
based
65.00
● Small data size (1016)
● Only tested on social issue dataset
(Iqbal et al., 2015)
BAT with
Lexicon-based
69.00
● System designed only for English language
text
(P. K. Singh et al.,
2015)
Dictionary-
based
79.00
● Very Small data size (48)
● Considered only negative comments for
testing.
(Shamsudin et al.,
2016)
Dictionary-
based
52.12
● Very Small data size (450 Facebook
comments)
● Limited to Malay language text
● Not sufficient accuracy
(Bhargava et al., 2016) Sentiwordnet 54.40
● Small dataset (637)
● Not considered Punjabi and English code-
mixed text
● Performance not sufficient
(Cruz et al., 2017)
Dictionary-
based
66.00
(Ruder & Plank, 2018)
Multi-task tri-
training
79.15
(Han et al., 2018)
Dictionary-
based
69.52 69.60
● Only Amazon product reviews
(Baoxin Wang, 2018)
Disconnected
Recurrent
Neural
Networks
64.43
(Yu & Liu, 2018)
Sliced
Recurrent
Neural
Networks
61.65
(S. K. Singh & Sachan,
2019b)
Dictionary-
based
82.50 87.04
(S. K. Singh & Sachan,
2019b)
Dictionary-
based
71.30 71.39
(Patwa et al., 2020) XLM-R 75.00
● Considered only Hindi and English code-
mixed text
(Kansal et al., 2020)
Logistic
regression
81.98

63
designed which can handle bilingual phonetic text and constituted by (a) lower case conversion
(b) tokenization (c) spell checker (d) part-of-speech (POS) tagging (e) stop words elimination (f)
stemming(g) sentiment calculation (h) document classification as shown in Figure 1.
2.1. Testing Datasets Generation
The existing four datasets in the English language are used to generate bilingual (English_Punjabi)
code-mixed phonetic text datasets (shown in Table 2) with the help of a bilingual dictionary of
English and Punjabi words. The Punjabi words are written in phonetic using Roman script letters.
The transliteration of Punjabi words into phonetic words is done using the GRT system (S. K. Singh
& Sachan, 2019a). The GST implementation Facebook comments are taken from an article (S. K.
Singh & Sachan, 2019b), movie reviews (Pang & Lee, 2004), Amazon product reviews (Book, DVD,
Electronics, Kitchen) (Blitzer et al., 2007) and large Amazon product reviews (He & McAuley,
2016). The size of the dataset for testing is 172,800 comments/reviews (86,400 for each English and
English_Punjabi language).
2.2. Conversion in Lower Case and Tokenization
The proposed system uses various dictionaries in which data are stored in lower case, so the testing
dataset must be converted into lower case. The comments are split into sentences, and sentences into
words because the proposed system works at the word level. The importance of tokenization (splitting)
is discussed in the article (P. K. Singh et al., 2015).
2.3. Spell Checker of Bilingual Text
The testing dataset is generated by inserting the Punjabi (Phonetic) words within the English language
sentence, and only correct Punjabi words without spelling mistakes are used. So, there is no need to
check the spelling of Punjabi words. The misspelled word is detected and corrected using the existing
system for English words (S. K. Singh & Sachan, 2019b).
Figure 1. Proposed system framework for code-mixed bilingual phonetic text

64
2.4. POS Tagging
Python’s NLTK tool is used to tag Part-of-speech (POS) along with each word. The POS tagging
makes it easy to identify the verb, adverb, adjective (Turney, 2002) from the sentence.
2.5. Elimination of Stop Words from the Bilingual Text
The stop words are those words that are useless for some natural language processing tasks. The uses
of stop words depend upon tasks where these words will be used. The significance of stop words
removal and various methods to eliminate such words are explained in the article (Saif et al., 2014).
After removing these words, the remaining words are further processed. There are 217 stop words in
the dictionary, which include 134 English stop words from the article (S. K. Singh & Sachan, 2019b)
and 83 selected Punjabi stop words from the article (Kaur & Saini, 2016).
2.6. Stemming
Stemming is the process of finding the root word from its inflectional form. During the stemming,
generally the suffixes and prefixes are removed from its inflectional form to get the root word (Jivani,
2011). For example- keeps, kept, keeping are the inflectional (grammatical) form of the ‘keep’ word.
Here, the root word ‘keep’ is found using the removal of suffixes from keeps and keeping words.
The size of the database becomes bulky when all forms of a word are stored. Hence, the database
stores only root words. Therefore, a verb stemmer system is developed for English language text,
as shown in Figures 2 and 3, in algorithms 1 and 2. The stemming is done only for verbs of English
language words because VOD contains only opinion verbs. StemVerb algorithm 1 is used to extract
the root verb word from a different form of a verb. The POS taggers are attached with each word, and
identified all verbs for stemming. There are three sub-stemming functions based on verb forms, such
as extraction from (a) ‘ing’, (b) “ies, es, s” and (c) past and past participle (ied, ed, en, d, n) verb form.
Algorithm 1: StemVerb
Input: Word along with POS tag
Output: Word (root/original word)
Step 1: If POS tag are VBP, VBG, VBZ, VBN and VBD, then
word is a verb
Step 2: (a) if word’s last letters are ‘ing’ then call
stemming function to find root word from ‘ing’ verb form (shown in
Figure 2)
(b) if word’s last letters are ‘ies’, ‘es’, ‘s’, then
call stemming function to find root word from ‘ies’, ‘es’, ‘s’
verb form (shown in Figure 3)
(c) if word’s last letters are ‘ied’, ‘ed’, ‘en’, ‘d’,
‘n’ or VBN/VBD POS tag then call stemming function to find root
word from past and past participle verb form (algorithm 2)
Table 2. Testing datasets
Sr. No. Datasets Size Language
Existing Generated
1 GST Implementation 400 comments English English and Punjabi
2 Movie Reviews 2000 reviews English English and Punjabi
3 Amazon Product Reviews 8000 reviews English English and Punjabi
4 Large Amazon Product Reviews 76,000 reviews English English and Punjabi

65
Step 3: return word (root word/original word)
Step 4: Otherwise, word in not verb
Step 5: Return word.
The process to extract root verb from ‘ing’ verb form is shown in Figure 2 and explained with
suitable examples.
Example-1: W=Word ‘keeping’ is the inflected form of ‘keep’. The W contains ‘ing’ characters at
the end, so remove the ‘ing’ characters from W and now check W (keep) exist in root verb list
(RVL), yes then returned W (keep) to calling algorithm 1. Example-2: W=Word ‘lying’ is the
inflected form of ‘lie’. The W contains ‘ing’ characters at the end, so remove the ‘ing’ characters
from W and now check W (ly) exist in RVL; no W does not exist in RVL. Then check ‘y’ is the
last character of W (ly), yes so replace ‘y’ with ‘ie’, and now W is ‘lie’, again check in RVL, W
exists in RVL. Finally, returned W (lie) to calling algorithm 1.
Example-3: W=Word ‘getting’ is the inflected form of ‘get’. The W contains ‘ing’ characters at the
end, so remove the ‘ing’ characters from W and now check W (gett) exist in RVL, no W does
not exist in RVL. Then check ‘y’ is the last character of W (gett), no so check last two characters
are same and not vowels, yes now remove one last character from W (gett). Finally, W is ‘get’
word, which is returned to calling algorithm 1.
Example-4: W=Word ‘choosing’ is the inflected form of ‘choose’. The W contains ‘ing’ characters
at the end, so remove the ‘ing’ characters from W and now check W (choos) exist in RVL; no
W does not exist in RVL. Then check ‘y’ is the last character of W (choos), no so check last two
characters are same and not vowels, no now add ‘e’ at the end of W (choos). Finally, W is ‘choose’,
check-in RVL, W exists in RVL. Therefore, returned W (choose) to the calling algorithm 1.
Figure 3 shows the process to extract the root verb word from “ies, es, s” verb form and explained
with suitable examples.
Example-5: W=word ‘applies’ is an inflected word of ‘apply’. The W contains ‘ies’ characters at
the end. Therefore, ‘ies’ characters are replaced with ‘y’, and W (apply) is checked in RVL, yes
W exist in RVL, so finally W (apply) is returned to calling algorithm 1.
Example 6: W=word ‘bashes’ is an inflected word of ‘bash’. The W contains ‘es’ characters at the
end. Then remove ‘es’ from the end of W (bashes) and checked W (bash) in RVL; yes W exist
in RVL. Therefore, finally returned W to calling algorithm 1.
Example 7: W=word ‘awakes’ is an inflected word of ‘awake’. The W contains ‘es’ characters at the
end. Then remove ‘es’ from the end of W (awakes) and checked W (awak) in RVL; no W does
not exist in RVL. Now remove‘s’ from W (awakes) and check W (awake) in RVL; yes W exist
in RVL. Finally, W (awake) is returned to calling algorithm 1.
Algorithm 2: Stemming verb from past and past participle verb form
Abbreviations Used: W=word, RVL= root verb list, WT
= target word
Input: Word along with POS tagger
Output: Word (Root/Original word)
Step 1: If word’s POS tag (VBN/VBD) and exist in an
irregular verb form list, then extract the root verb word from the
irregular verb list.
Step 2: If ‘ied’ last letter of W, then WT
=W minus ‘ied’
and add ‘y’ at end of WT
(a) If WT
exist in RVL, then return WT
(b) Otherwise return W
Step 3: If ‘ed’/ ‘en’ last letter of W, then WT
= W minus

66
Figure 2. Extraction of root verb from ‘ing’ verb form
Figure 3. Extraction of root verb from (ies, es, s) verb form

67
‘ed’/ ‘en’ and check WT
in RVL
(a) If WT
(b) If ‘d’/ ‘n’ last letter of W, then WT
= W minus ‘d’/ ‘n’
and check WT
in RVL
(i) If WT
(c) If last two letter of WT
is same, then
(i) If ‘en’ is last letter of W, then remove one last
letter from WT
(1) If WT
(2) Otherwise, add ‘e’ at end of WT
and return WT
(ii) Otherwise, remove one last letter from WT
(1) If WT
(2) Otherwise, return W
Step 4: Otherwise, return W
2.7. Sentiment Calculation
In this section, the sentiment score of each sentence is calculated. Some negation handlings rules
are proposed to determine the polarity of sentences or words, whenever negation words, negative
prefixes, and positive words occur within the sentence.
2.7.1 Negation Handling
The significance of negation words was first time discussed by (Polanyi & Zaenen, 2006). The six
negation handling rules are proposed and are used when the polarity of words changes due to the
presence of negation words1
or negative prefixes in the sentence (P. K. Singh et al., 2015). For
example- I do not like this movie. Here ‘not’ is a negation word due to which the polarity of the
sentence changes from positive to negative. Threshold negative ( )
Tn
value is fixed for all negation
words, which is equal to -0.25 discussed in the article (S. K. Singh & Sachan, 2019b). The negative
prefixes (dis, de, mis, ir, il, in, un) or suffixes are also acting as negation words, so the negation
handling rules (shown in Table 5) are used to calculate the sentiment score whenever these negation
words and negative prefixes or suffixes occur within a sentence (S. K. Singh & Sachan, 2019b). All
14 negation words, 7 negative prefixes of English language discussed in the article (S. K. Singh &
Sachan, 2019b) and 8 negation words (shown in Table 3), 4 negative prefixes (de, ka, da, a) of Punjabi
language are used for the implementation of the proposed system.
A sentence’s sentiment is expressed by the opinion words (positive words, negation words, and
verbs) present within the sentence. The positive words are the opinion words, but they are not opinion
verbs. For example- Nice photo. Here ‘nice’ word is not a verb, but still expresses the sentiment of
the sentence. Therefore, these kinds of opinion words are considered as positive words (S. K. Singh
& Sachan, 2019b). There are a total of 35 positive words, out of which 20 words of English are taken
from the article (S. K. Singh & Sachan, 2019b) and 15 words of Punjabi language (shown in Table
4). The sentiment score value for these positive words is denoted as threshold positive( )
Tp
, which
is equal to 0.125 value as discussed by (S. K. Singh & Sachan, 2019b).
2.7.2 Calculation of Sentiment Score
The VOD consists of opinion verbs of both languages (English and Punjabi) and the sentiment
score value. The value of the sentiment score is from -1 to 1. The negation and positive words are
not included in this VOD because they may be adverbs, adjectives, determiners, and prepositions.
There are 1330 opinion verbs, out of which 677 English words and 553 words of Punjabi language.
The sentiment score to these words is assigned manually based on their polarity and with the help of

68
existing English SentiWordNet 3.0 (Baccianella et al., 2010). This VOD is domain-dependent, but it
can be further used in other domains only after adding some words from that domain.
The opinion verbs are considered as a feature and are extracted from the subjective sentences;
and searched into the VOD, if the word is present in VOD, then extract the value of sentiment score
of the opinion verb which is denoted as Wordscore
. The value of Wordscore
is positive (Wordscore
> 0 )
then the word is positive, otherwise the value of Wordscore
is negative (Wordscore
< 0 ), then the word
Table 3. Negation words of Punjabi language
Sr. No. Negation Word Sr. No. Negation Word
1 naahi 5 bekaar
2 nahiin 6 galat
3 viruddh 7 gair
4 bura 8 nakaaraatamak
Table 4. Positive words of Punjabi language
Sr. No. Positive Word Sr. No. Positive Word Sr. No. Positive Word
1 haan 6 bharosa 11 piaar
2 thiik 7 changa 12 sahaaita
3 dhannavaad 8 shaanadaar 13 kadar
4 khair 9 vadhiia 14 vadhaaiiaan
5 mahaan 10 sahii 15 sakaaraatamak
Table 5. Rules for Negation Handling
S.No. Opinion verb Negative
Prefixes
Negation
Words
Positive
Word
Sentiment
Score
Positive/
negative
Sentiment score calculation
1 Positive Yes Yes No Positive Pos Pos Word
score score score
= +
2 Positive Yes No No Negative Neg Neg Word
score score score
= − ×
1
3 Positive No Yes No Negative Neg Neg Word
score score score
= − ×
1
4 No No Yes Yes Negative Neg Neg T
score score p
= − ×
1
5 Verb out of the
dictionary
No Yes No Negative
Neg Neg T
score score n
= +
6 Negative No Yes No Positive Pos Pos Word
score score score
= − ×
1

69
is negative. The positive sentiment score Posscore
( ) value for all positive opinion words (verb/term)
is calculated using Equation (1) and negative sentiment score ( Negscore
) value for all negative opinion
words (verb/term) using Equation (2) within a sentence. The sentence sentiment score ( )
Sentscore
is
calculated using Equation (3) in which the normalized value of Posscore
and Negscore
is added. To
normalize the Posscore
and Negscore
, Posscore
is divided by the total positive opinion words ( p ) and
Negscore
is divided by the total negative opinion words ( n ) respectively in Equation (3). The document/
comment sentiment score is calculated using Equation (4), by taking the summation of the sentiment
score value ( )
Sentscore
. of all the sentences, and the total number of sentences is denoted by ‘s’ in
Equation (4). The value of each sentence’s sentiment score is calculated using the ‘calculation of
sentiment score’ algorithm 3.
Pos Word if Word
score
i
p
score i score
=
=
∑
0
0
( ) (1)
Neg Word if Word
score
j
n
score j score
=
=
∑
0
0
( ) (2)
Sent
Pos
p
Neg
n
score
score score
= + (3)
Doc Sent
score
k
s
score k
=
=
∑
0
( ) (4)
Algorithm: 3 Calculation of sentiment score
Notations: W- Word, VOD - Verb Opinion Dictionary, NW- Negation
Words, NHR-Negation Handling Rules, NP- Negative Prefixes, PWL-
Positive Word List
Input: Sentence (text)
Output: Sentence’s sentiment score
Step1: Read word (W) by word from the sentence and repeat steps 2
to 5
Step2: If last W of a sentence is a NW, then add Tn
to Negscore
and add 1 to n, goto step 1
Step3: If W is found in VOD then extract Wordscore
from VOD
(a) if NW comes before W, then update PosScore
or Negscore
using NHR and add 1 to p/n, goto step 1
(b) otherwise, update PosScore
or Negscore
using Equation (1)
or (2) and add 1 to p/n, goto step 1
Step4: If W is found in PWL, then

70
(a) if NW comes before W, then update Negscore
using NHR
and add 1 to n, goto step 1
(b) otherwise, update PosScore
by adding Tp
and add 1 to p,
goto step 1
Step5: If W does not find in VOD and PWL, then
(a) if NW comes before W, then update Negscore
NHR and add
1 to n, goto step 1
(b) if NP exist in W and root word found in VOD, then
extract Wordscore
from VOD
(i) if NW comes before W, then update PosScore
using NHR and
add 1 to p, goto step 1
(ii) otherwise, update NegScore
using NHR and add 1 to n,
goto step 1
Step6: Sentscore
is updated using Equation (3)
Step7: Sentscore
is return
2.8. Document Classification
The document is classified into two classes (negative or positive) using binary class classification.
The document’s sentiment score ( Docscore
) is calculated using Equation (4), and the document is
classified using a rule-based classifier Equation (5). The document is classified as a positive class if
its’ Docscore
value is greater than or equal to zero; otherwise, a negative class using Equation (5).
Those documents whose Docscore
value is equal to zero is considered positive due to the presence of
positive words in most cases in place of an opinion verb (S. K. Singh Sachan, 2019b).
Document
Positive if Doc
Negative otherwise
score
=
≥









0
(5)
3. EXPERIMENTAL RESULTS AND DISCUSSION
The proposed SA system’s performance is evaluated for English and English_Punjabi (bilingual) text
on four different datasets (discussed in section 2.1). The proposed system used a dictionary-based
approach and various dictionaries such as stop words, negative prefixes, negation words, positive
words of both English and Punjabi language. The Tn
= −0 25
. is the threshold negative value and
Tp
= 0 125
. is the threshold positive value were used (S. K. Singh Sachan, 2019b). Also, the
performance of the proposed StemVerb system is discussed in this section.
The system performance is measured using performance metrics such as recall, precision,
accuracy, and F score (Equation 6 to 9). Its’ some related terms are defined as true positive (Tp
):
positive texts are predicted as positive; false positive ( Fp
): negative texts are predicted as positive;
true negative (Tn
): negative texts are predicted as negative and false negative ( Fn
): positive texts are
predicted as negative.

71
Recall
T
T F
p
p n
=
+
( )
(6)
Precision
T
T F
p
p p
=
+
( )
(7)
Accuracy
T T
T F F T
p n
p p n n
=
+
( )
+ + +
( )
(8)
F score
recall precision
recall precision
=
× ×
+
2
(9)
3.1 Performance of System Evaluation on Different Datasets
The performance of the proposed system is discussed in this section on four different dataset(s).
All dataset(s) size and its’ domain are different. The system’s performance is better on the GST
implementation dataset than other datasets (shown in Figures 4 and 5). The performance of the
proposed system is better in the English than the English_Punjabi language datasets.
The proposed system is designed for code-mixed bilingual phonetic text, but its performance is
evaluated on both monolingual (English) and bilingual (English_Punjabi) text. As per Table 6, the
system performance is better than the existing state-of-the-art methods on all datasets. The system
achieved better performance in terms of accuracy in the range of 3% to 33.38% on social media dataset,
1.5% to 3.8% on movie reviews, 1.35% to 13.81% on Amazon product reviews, 4.56% to 7.34% on large
Amazon product reviews for monolingual text. In the bilingual text, system performance is better than
other existing methods in terms of F score (13.22% on social media and 17.03% on movie reviews).
3.2 Performance Evaluation of StemVerb System
A StemVerb system is proposed to extract the root verb from the inflectional form of verbs. The
framework and working process of the StemVerb system (discussed in section 2.6). The root verb list
of 3582 words and irregular verb forms a list of 222 words is generated with different resources2, 3,
.
There are 3992 words used to test the StemVerb system and the existing two algorithms performance,
such as Snowball4
and Lancaster5
. The StemVerb system is the only proposed for the verb of the
English language. The performance of StemVerb is better than Snowball and Lancaster stemmer
(shown in Figure 6).
The performance of SA system evaluated in terms of accuracy for bilingual text using all three
stemmers, but SA system performed better using the StemVerb system than the other two existing
stemmers (shown in Figure 7). Also, the accuracy of the proposed SA system has been improved after
the stemming of verbs on bilingual datasets (shown in Figure 8). The VOD contains only root verbs
along with their sentiment scores. Therefore, the stemming of verb words improves the performance
of the proposed SA system.

72
4. CONCLUSION AND FUTURE SCOPE
The sentiment analysis system for bilingual code-mixed phonetic text is developed and tested on
four datasets, written in both monolingual and bilingual. The existing monolingual text in English
is converted into bilingual (English and Punjabi) code-mixed phonetic text. The spell checker,
VOD, negation words, positive words, negative prefixes, negation handling rules, stop words, and
StemVerb system is developed and used to extract the writer’s sentiment from their written text.
The developed VOD for English and Punjabi language, contains 1330 verb opinion words of both
Figure 4. Performance of system on the different dataset(s) (English)
Figure 5. Performance of system on the different datasets (English and Bilingual)

73
Table 6. Experimental results are compared with state-of-the-art
Authors Method POS Dataset Language
Accuracy
(%)
F
Score
(%)
(Taboada et al.,
2011)
Lexicon-based
Adjective,
adverb, verb,
noun
Movie reviews English 70.10
(Karamibekr
Ghorbani, 2012)
Dictionary-
based
Verb, adverb,
adjective
Social media English 65.00
(Iqbal et al., 2015)
BAT with
Lexicon-based
(P. K. Singh et al.,
2015)
Dictionary-
based
Verb Social media English 79.00
(Shamsudin et al.,
2016)
Dictionary-
based
Verb, adverb,
adjective
Social media Malay 52.12
(Bhargava et al.,
2016)
SentiWordNet Movie reviews English_Hindi 54.40
(Cruz et al., 2017)
Dictionary-
based
Adjective,
noun
(Ruder Plank,
2018)
Multi-task tri-
training
Amazon product
reviews
English 79.15
(Han et al., 2018)
Dictionary-
based
Amazon product
reviews
English 69.52 69.60
(Baoxin Wang,
2018)
Disconnected
Recurrent
Neural
Networks
Large Amazon
product reviews
English 64.43
(Yu Liu, 2018)
Sliced
Recurrent
Neural
Networks
Large Amazon
product reviews
English 61.65
(S. K. Singh
Sachan, 2019b)
Dictionary-
based
Verb Social media English 82.50 87.04
(S. K. Singh
Sachan, 2019b)
Dictionary-
based
Verb Movie reviews English 71.30 71.39
(Patwa et al., 2020) XLM-R Social media English_Hindi 75.00
(Kansal et al., 2020)
Logistic
regression
Amazon product
reviews
English 81.98
Proposed system
Dictionary-
based
Verb
Social media
English 85.50 89.18
English_Punjabi 84.25 88.22
Movie reviews
English 72.80 73.12
Amazon product
reviews
English 83.33 83.27
Large Amazon
product reviews
English 68.99 68.07

74
English and Punjabi language. The list of irregular verbs of 222 words and root verb list of 3582
words is developed to improve the SA system’s performance. The SA system classifies text into
negative or positive class using a rule-based classifier and sentiment score value extracted from the
VOD. The SA system obtained an accuracy of 85.5% and 84.25% on the GST dataset; 72.8% and
71% on the movie reviews; 83.33% and 80.87% on Amazon product reviews; 68.99% and 65.34%
on large Amazon product reviews for monolingual and bilingual text respectively. This system will
automatically classify customers’ feedback into positive and negative class, and will help customers
to decide before purchasing a product; manufacturers can enhance the quality of product and services
Figure 6. Performance of stemmers (Snowball, Lancaster, and StemVerb)
Figure 7. Sentiment analysis system’s accuracy using all three stemmers

75
based on it, and can keep an eye on their competitors; know the mood of the public before election
and government policies. The proposed SA system for bilingual text can be further extended to other
code-mixed bilingual texts by modifying some components easily. The developed VOD can also be
extended for other part-of-speech and to other languages.
Figure 8. Sentiment analysis system’s accuracy with and without stemming

76
REFERENCES
Alharbi, J. R., Alhalabi, W. S. (2020). Hybrid Approach for Sentiment Analysis of Twitter Posts Using a
Dictionary-based Approach and Fuzzy Logic Methods. International Journal on Semantic Web and Information
Systems, 16(1), 116–145. doi:10.4018/IJSWIS.2020010106
Baccianella, S., Esuli, A., Sebastiani, F. (2010). SENTIWORDNET 3.0: An enhanced lexical resource for
sentiment analysis and opinion mining. 7th International Conference on Language Resources and Evaluation,
2200–2204. http://nmis.isti.cnr.it/sebastiani/Publications/LREC10.pdf
Wang, B. (2018). Disconnected Recurrent Neural Networks for Text Categorization. 56th Annual Meeting of
the Association for Computational Linguistics, 2311–2320. doi:10.18653/v1/P18-1215
Bhargava, R., Sharma, Y., Sharma, S. (2016). Sentiment analysis for mixed script Indic sentences. 2016
International Conference on Advances in Computing, Communications and Informatics, 524–529. doi:10.1109/
ICACCI.2016.7732099
Blitzer, J., Dredze, M., Pereira, F. (2007). Biographies, bollywood, boom-boxes and blenders: Domain
adaptation for sentiment classification. ACL 2007 - Proceedings of the 45th Annual Meeting of the Association
for Computational Linguistics, 440–447. https://www.aclweb.org/anthology/P07-1056.pdf
Bonadiman, D., Castellucci, G., Favalli, A., Romagnoli, R., Moschitti, A. (2017). Neural Sentiment Analysis
for a Real-World Application. In Proceedings of the Fourth Italian Conference on Computational Linguistics
CLiC-it 2017 (pp. 42–47). Accademia University Press. doi:10.4000/books.aaccademia.2357
Cambria, E., Poria, S., Bajpai, R., Schuller, B. (2016). SenticNet 4: A semantic resource for sentiment analysis
based on conceptual primitives. COLING 2016 - 26th International Conference on Computational Linguistics,
Proceedings of COLING 2016: Technical Papers, 2666–2677. https://www.aclweb.org/anthology/C16-1251.pdf
Cruz, L., Ochoa, J., Roche, M., Poncelet, P. (2017). Dictionary-based sentiment analysis applied to a specific
domain. In A.-S. H. Lossio-Ventura J. (Ed.), Communications in Computer and Information Science: Vol. 656
CCIS (pp. 57–68). Springer. doi:10.1007/978-3-319-55209-5_5
D’Andrea, A., Ferri, F., Grifoni, P., Guzzo, T. (2015). Approaches, Tools and Applications for Sentiment
Analysis Implementation. International Journal of Computers and Applications, 125(3), 26–33. doi:10.5120/
ijca2015905866
Drus, Z., Khalid, H. (2019). Sentiment Analysis in Social Media and Its Application: Systematic Literature
Review. Procedia Computer Science, 161, 707–714. doi:10.1016/j.procs.2019.11.174
Dutta, S., Saha, T., Banerjee, S., Naskar, S. K. (2015). Text normalization in code-mixed social media text.
2015 IEEE 2nd International Conference on Recent Trends in Information Systems, 378–382. doi:10.1109/
ReTIS.2015.7232908
Guellil, I., Boukhalfa, K. (2015). Social big data mining: A survey focused on opinion mining and
sentiments analysis. 2015 12th International Symposium on Programming and Systems, 1–10. doi:10.1109/
ISPS.2015.7244976
Han, H., Zhang, Y., Zhang, J., Yang, J., Zou, X. (2018). Improving the performance of lexicon-based review
sentiment analysis method by reducing additional introduced sentiment bias. PLoS One, 13(8), e0202523.
doi:10.1371/journal.pone.0202523 PMID:30142154
He, R., McAuley, J. (2016). Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-
Class Collaborative Filtering. International World Wide Web Conference Committee (IW3C2), 507–517.
doi:10.1145/2872427.2883037
Hopken, W., Fuchs, M., Menner, T., Lexhagen, M. (2017). Sensing the Online Social Sphere Using a Sentiment
Analytical Approach. In Z. Xiang D. R. Fesenmaier (Eds.), Analytics in Smart Tourism Design Concepts and
Methods (pp. 129–146). Springer. doi:10.1007/978-3-319-44263-1_8
Hussein, D. M. E.-D. M. (2018). A survey on sentiment analysis challenges. Journal of King Saud University
-. Engineering and Science, 30(4), 330–338. doi:10.1016/j.jksues.2016.04.002

77
Iqbal, M., Karim, A., Kamiran, F. (2015). Bias-aware lexicon-based sentiment analysis. Proceedings of the
30th Annual ACM Symposium on Applied Computing - SAC ’15, 845–850. doi:10.1145/2695664.2695759
Jivani, A. G. (2011). A Comparative Study of Stemming Algorithms. International Journal of Computer
Technology and Applications, 2(6), 1930–1938. https://pdfs.semanticscholar.org/1c0c/0fa35d4ff8a2f925eb95
5e48d655494bd167.pdf
Kansal, N., Goel, L., Gupta, S. (2020). Cross-domain sentiment classification initiated with Polarity Detection
Task. EAI Endorsed Transactions on Scalable Information Systems., doi:10.4108/eai.26-5-2020.165965
Karamibekr, M., Ghorbani, A. A. (2012). Verb oriented sentiment classification. 2012 IEEE/WIC/ACM
International Conference on Web Intelligence, 327–331. doi:10.1109/WI-IAT.2012.122
Kaur, J., Saini, J. R. (2016). Punjabi Stop Words: A Gurmukhi, Shahmukhi and Roman Scripted Chronicle.
ACM Symposium on Women in Research 2016, 32–37. doi:10.1145/2909067.2909073
Lo, S. L., Cambria, E., Chiong, R., Cornforth, D. (2017). Multilingual sentiment analysis: From formal to
informal and scarce resource languages. Artificial Intelligence Review, 48(4), 499–527. doi:10.1007/s10462-
016-9508-4
Medhat, W., Hassan, A., Korashy, H. (2014). Sentiment analysis algorithms and applications: A survey. Ain
Shams Engineering Journal, 5(4), 1093–1113. doi:10.1016/j.asej.2014.04.011
Pang, B., Lee, L. (2004). A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization
Based on Minimum Cuts. Proceedings of the ACL. https://arxiv.org/abs/cs/0409058
Pang, B., Lee, L. (2008). Opinion Mining and Sentiment Analysis. Foundations and Trends® in Information
Retrieval, 2(1–2), 1–135. 10.1561/1500000011
Patwa, P., Aguilar, G., Kar, S., Pandey, S. (2020). SemEval-2020 Task 9: Overview of Sentiment Analysis of
Code-Mixed Tweets. https://arxiv.org/abs/2008.04277
Peng, L., Cui, G., Zhuang, M., Li, C. (2014). What do seller manipulations of online product reviews mean
to consumers? In Digital Commons @ Lingnan University (HKIBS/WPS/070-1314). https://commons.ln.edu.
hk/hkibswp/70
Polanyi, L., Zaenen, A. (2006). Contextual Valence Shifters. In Computing Attitude and Affect in Text: Theory
and Applications (pp. 1–10). Springer-Verlag. doi:10.1007/1-4020-4102-0_1
Press Trust Of India. (2013, July 10). India to have the highest internet traffic growth rate. Business Standard.
https://www.business-standard.com/article/technology/india-to-have-the-highest-internet-traffic-growth-
rate-113071000014_1.html
Ravi, K., Ravi, V. (2015). A survey on opinion mining and sentiment analysis: Tasks, approaches and
applications. Knowledge-Based Systems, 89, 14–46. doi:10.1016/j.knosys.2015.06.015
Ruder, S., Plank, B. (2018). Strong baselines for neural semi-supervised learning under domain shift. ACL
2018 - 56th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
(Long Papers), 1, 1044–1054. doi:10.18653/v1/P18-1096
Saif, H., Fernandez, M., He, Y., Alani, H. (2014). On stopwords, filtering and data sparsity for sentiment
analysis of twitter. 9th International Conference on Language Resources and Evaluation, 810–817. http://oro.
open.ac.uk/id/eprint/40666
Serrano-Guerrero, J., Olivas, J. A., Romero, F. P., Herrera-Viedma, E. (2015). Sentiment analysis: A review
and comparative analysis of web services. Information Sciences, 311, 18–38. doi:10.1016/j.ins.2015.03.040
Shamsudin, N. F., Basiron, H., Sa’aya, Z. (2016). Lexical Based Sentiment Analysis – Verb, Adverb
Negation. Journal of Telecommunication, Electronic and Computer Engineering, 8(2), 161–166. https://journal.
utem.edu.my/index.php/jtec/article/view/976/566
Singh, P. K., Singh, S. K., Paul, S. (2015). Sentiment classification of social issues using contextual valence
shifters. International Journal of Engineering and Technology, 7(4), 1443–1452. http://www.enggjournals.com/
ijet/docs/IJET15-07-04-335.pdf

78
Shailendra Kumar Singh is currently a Research Scholar in the Department of Computer Science and Engineering
at Sant Longowal Institute of Engineering and Technology (SLIET), Sangrur, Punjab, India. He obtained his
B.Tech degree in Information Technology from UPTU (Lucknow) in 2012 and Master of Engineering in Software
Engineering from Birla Institute of Technology (BIT), Mesra, Ranchi (India). He has qualified national level exams
(GATE-13 UGC-NET-2018, 2019). His research interests include handwriting recognition, sentiment analysis,
natural language processing, data mining, human stress level detection and personality detection.
Manoj Kumar Sachan (PhD) is currently a Professor at Sant Longowal Institute of Engineering and Technology
(SLIET), India. He is associated with the Department of Computer Science and Engineering. He did his B.Tech in
Computer Science from Punjabi University, Patiala, India. He did M.E in Computer Science from Thapar Institute
of Engineering Technology, Patiala, and Ph.D from Punjab Technical University, Jalandhar, India. His research
interests include handwriting recognition, stress detection, opinion mining, medical image processing, natural
language processing, and data mining.
Singh, R. K., Sachan, M. K., Patel, R. B. (2020). 360 degree view of cross-domain opinion classification: A
survey. Artificial Intelligence Review. Advance online publication. doi:10.1007/s10462-020-09884-9
Singh, S. K., Sachan, M. K. (2019a). GRT: Gurmukhi to Roman Transliteration System using Character
Mapping and Handcrafted Rules. International Journal of Innovative Technology and Exploring Engineering,
8(9), 2758–2763. doi:10.35940/ijitee.I8636.078919
Singh, S. K., Sachan, M. K. (2019b). SentiVerb system: Classification of social media text using sentiment
analysis. Multimedia Tools and Applications, 78(22), 32109–32136. doi:10.1007/s11042-019-07995-2
Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M. (2011). Lexicon-Based Methods for Sentiment
Analysis. Computational Linguistics, 37(2), 267–307. doi:10.1162/COLI_a_00049
Turney, P. D. (2002). thumbs up or thumbs down? semantic orientation applied to unsupervised
classification of reviews. 40th Annual Meeting on Association for Computational Linguistics, 417–424.
doi:10.3115/1073083.1073153
Yu, Z., Liu, G. (2018). Sliced Recurrent Neural Networks. 27th International Conference on Computational
Linguistics, 2953–2964. https://www.aclweb.org/anthology/C18-1250
Yue, L., Chen, W., Li, X., Zuo, W., Yin, M. (2019). A survey of sentiment analysis in social media. Knowledge
and Information Systems, 60(2), 617–663. doi:10.1007/s10115-018-1236-4
ENDNOTES
1
Negation words reverse the polarity of the word or sentence, if these words appear before negative or
positive word in the sentence.
2
https://www.worldclasslearning.com/english/five-verb-forms.html
3
https://www.enchantedlearning.com/wordlist/verbs.shtml
4
https://snowballstem.org/demo.html
5
https://text-processing.com/demo/stem/

Classification of Code-Mixed Bilingual Phonetic Text Using Sentiment Analysis

More Related Content

What's hot (20)

Similar to Classification of Code-Mixed Bilingual Phonetic Text Using Sentiment Analysis (20)

Recently uploaded (20)

Classification of Code-Mixed Bilingual Phonetic Text Using Sentiment Analysis