![International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470
@ IJTSRD | Unique Paper ID - IJTSRD21532 | Volume – 3 | Issue – 3 | Mar-Apr 2019 Page: 89
2. Neung Viriyadamrongkij and Twittie Senivongse
(2017):
Online inquiry communities such as Question-Answer
Communities (QAC) have captured interest of online users
since they can share and search for any information from
any place in the world. The number of questions and
answers submitted to a popular community can increase
rapidly, and that can make it difficult for users who look for
the right" questions to answer. That is, from the view of
knowledgeable experienced users, theytendtolookforhard
challenging questions as an opportunity to share their
knowledge and to build respect with the community. Hence
it is desirable to distinguish difficult questions from easy
ones. Current researches estimate complexity of questions
based on the analysis of the features of the QAC without
considering the contents of the questions. This paper
presents a method to measure question difficulty levels
based directly on the question contents. In particular, we
analyze the difficulty of terms that appear in a JavaScript-
related question, based on the proposed JavaScript concept
hierarchy. In an evaluation of the performance of the
question difficulty estimation, our concept based measure
givessimilar performance to that of the existing measure
based on the features of the QAC.
3. Sasitorn Nuthong, Suntorn Witosurapot (2017):
This Automatic Quiz Generation system is utterly handy for
reducing teachers' workloads in quiz creation.Nevertheless,
by exploiting a coarse-granular concern inside difficulty
ranking mechanism, only a few number of automatic
generated quizzes can be obtained. In order to increase the
number of usable quizzes, we suggesthowa5-leveldifficulty
ranking score using a hybrid similarity measurement
approach together with property filteringofthekeydatacan
be potential for serving this propose. Based on experiment
results, our proposed similarity measure outperforms three
other candidates. Enabling users with finer options of
making sensible quiz generation. Hence,this mechanismcan
be regarded as a synergistic technology for improving
teachers' quality of life for the future.
4. Surbhi Choudhary, Abdul Rais Abdul Waheed,
Shrutika Gawandi, Kavita Joshi (2015):
In this modern world e-book has become a basic
requirement for the candidates to appear and prepare for
their competitive exams within college premises. In this
paper we are proposing a replica system for smart question
paper generation of universities.Themechanismbehindthis
system is that many random question papers are generated
along with the complexity level of the questions in terms of
percentage.[4] After generation that particular question is
then mailed to the respective university. In this system
administration of the database inputs set of question paper
with an option of check box to tick the accurate answer.
More ever weightage of the particular question in terms of
marks and hours and the complexity of the question is
determined. After this course whole question paperall along
with the weightage is stored in the database. Inside order to
make question paper for 100 marks, admin sets all the
weightage and difficulties to solve the problem. When the
difficulty and weightage is specified a pre doc le as per
selected format will be downloaded to the admin and an
electronic mail will be triggered.
Variety of difficulty may vary from easy, Medium and hard.
5. Pawel Jurczyk, Eugene Agichtein (2007):
Question-Answer portals such as Naverand Yahoo!Answers
are rapidly fetching rich sources of information on a lot of
topics which are not well served by general web search
engines. Unluckily, the quality of the submitted answers is
uneven, ranging from excellent detailed answers to snappy
and insulting remarks or even advertisements for
commercial content. Furthermore, user feed-back for many
topics is sparse, and can be insufficient to reliably identify
good answers from the bad ones. Hence, estimating the
ability of users is a critical task for this rising domain, with
potential applications to answer ranking, spam detection,
and incentive mechanism design. We present an analysis of
the link structure of a general-purpose question answering
community to discover authoritative users, and capable
experimental results over a dataset of more than 3 million
answers from a popular community QA site. We also explain
structural differences between question topicsthatcorrelate
with the success of link analysis for authority discovery.
Existing system:
There are very few such system previous available like a
method was proposed based on Measuring Difficulty Levels
of JavaScript online Questions in Question-Answer
Community Based on Concept Hierarchy.
III. PROPOSE SYSTEM
We propose a system to automate paper setting and to
measure the difficulty levels of questions in an Engineering
subjects by analyzing the Terminology that appears in the
questions.
IV. PROPOSE SYSTEM ARCHITECTURE
Figure4.1. System Architecture
V. CONCLUSION AND FUTURE SCOPE
Establishing mechanisms tocontrolandpredictthedifficulty
of assessment questions is clearly a big gap in existing
question generation literature. Our contributions have
covered the deeper aspects of the problem, and proposed
strategies, that exploit ontologies and associated measures,
to provide a better difficulty-level predicting model,thatcan
address this gap. We developed the difficulty level model
(DLM) by introducing three learner specific logistic
regression models for predicting the difficulty of a given
question for three categories of learners.Theoutputof these
three models was then interpreted using the Item Response](https://image.slidesharecdn.com/18comparativestudyofdifferentapproachesformeasuringdifficultylevelofquestion-190612074749/85/Comparative-Study-of-Different-Approaches-for-Measuring-Difficulty-Level-of-Question-2-320.jpg)
![International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470
@ IJTSRD | Unique Paper ID - IJTSRD21532 | Volume – 3 | Issue – 3 | Mar-Apr 2019 Page: 90
Theory to assign high, medium or low difficulty level. The
overall performance of the DLM and the individual
performance of the three regression models based on cross-
validation were reported and they are found to be
satisfactory. Comparison with the state-of-the-art method
shows an improvement of 8.5 difficulty-levels of benchmark
questions. The model proposed in this paper for predicting
the difficulty-level of questions is limited to A Box based
factual questions. It would be interesting to extend this
model to questions that are generated usingtheTBox-based
approaches. However,the challengestobead-dressed would
be much more, since, in the T Box-based methods, we have
to deal with many complex restriction types (unlike in the
case of A Box-based methods) and their influence on the
difficulty-level of the question framed out of them needs a
detailed investigation. For establishing the propositionsand
techniques stated in this paper, we have implemented a
system which demonstrates thefeasibilityofthemethods on
medium sized ontologies. It would be interesting to
investigate the working of the system on large ontologies.
REFERENCES
[1] UlrikePad o Question Difficulty How to Estimate
Without Norming, How to Use for Automated Grading"
Proceedings of the 12th Workshop on Innovative Use
of NLP for Building Educational Applications,
Copenhagen, Denmark, September 8, 2017.
[2] Neung Viriyadamrongkij and Twittie Senivongse
Measuring Difficulty Levels of JavaScript Questionsin
Question-Answer Community Based on Concept
Hierarchy"978-1-5090-4834-2/17/31.00 c2017 IEEE
[3] Itziar Aldabe Edurne Martinez ArikIturri: an
Automatic Question Generator Based on Corpora and
NLP Techniques" Conference Paper inLectureNotesin
Computer Science June 2006
[4] Sarah K. Two Methods for Measuring Question
Difficulty and Discrimination in Incomplete Crowd
sourced Data "Association for the Advancement of
Artificial Intelligence 2013
[5] Asma Ben Abacha, Marcos Da Silveira, and Cedric
Pruski. Medical ontology validation through question
answering. In AIME, pages 196{205, 2013.
10.1007/978-3-642-38326-730.
[6] Maha Al-Yahya. Ontology-based multiple choice
question generation. The Scientific World Journal, Vol
2014, page 9, ID: 10.1155/2014/274949, 2014.
[7] T. Alsubait, B. Parsia, and U. Sattler. A similarity based
theory of controlling mcq difficulty.In eLearningande-
Technologies in Education (ICEEE), 2013 Second
International Conference on, pages 283{ 288, Sept
2013.
[8] T. Alsubait, B. Parsia, and U. Sattler. Generating
multiple choice questions from ontologies: Lessons
learnt. In Proceedings of the 11th Inter-national
Workshop on OWL: Experiences and Directions
(OWLED 2014), volume 1265, pages 73{84, Oct 2014.
[9] Tahani Alsubait. Ontology-based multiple-choice
question generation. PhD thesis, School of Computer
Science, The University of Manchester, 2015.
[10] Tahani Alsubait, Bijan Parsia, and UlrikeSattler.Mining
ontologies for Analogy questions: A similarity-based
approach. Volume849 of CEURWorkshopProceedings.
OWL Experiences and Directions, 2012.
[11] Tahani Alsubait, Bijan Parsia, and Ulrike Sattler.
Generating multiple choice questions from ontologies:
Lessons learnt. Volume1265 of CEUR Workshop
Proceedings.OWL Experiences and Directions, 2014.
[12] Xinming An and Yiu-Fai Yung. Item response theory:
What it is and how you can use the irt procedure to
apply it. In SAS Global Forum, 2014.](https://image.slidesharecdn.com/18comparativestudyofdifferentapproachesformeasuringdifficultylevelofquestion-190612074749/85/Comparative-Study-of-Different-Approaches-for-Measuring-Difficulty-Level-of-Question-3-320.jpg)
Comparative Study of Different Approaches for Measuring Difficulty Level of Question
- 1. International Journal of Trend in Scientific Research and Development (IJTSRD) Volume: 3 | Issue: 3 | Mar-Apr 2019 Available Online: www.ijtsrd.com e-ISSN: 2456 - 6470 @ IJTSRD | Unique Paper ID - IJTSRD21532 | Volume – 3 | Issue – 3 | Mar-Apr 2019 Page: 88 Comparative Study of Different Approaches for Measuring Difficulty Level of Question Ayesha Pathan, Dr. Pravin Futane Department of Computer Engineering, PCCOE, Pune, Maharashtra, India How to cite this paper: Ayesha Pathan| Dr. Pravin Futane "Comparative Study of Different Approaches for Measuring Difficulty Level of Question" Published in International Journal of Trend in Scientific Research and Development (ijtsrd), ISSN: 2456- 6470, Volume-3 | Issue-3, April 2019, pp. 88-90. http://www.ijtsrd.co m/papers/ijtsrd215 32.pdf Copyright © 2019 by author(s) and International Journal of Trend in Scientific Research and Development Journal. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (CC BY 4.0) (http://creativecommons.org/licenses/ by/4.0) ABSTRACT Semantics-based information representations such asontologies arefoundtobe very useful in repeatedly generating important factual questions. Formativethe difficulty-level Of these system generated questions is helpful to successfully make use of them in various learningand specialized applications.The accessible approaches for result the difficulty-level of factualquestionsare verysimple and are limited to a few basic principles. We suggest a new tactic for this problem by considering an edifying theory called Item Response Theory (IRT). In the IRT, facts skill of end users (learners) are considered for assigning difficulty levels, because of the assumptions that a given question is apparent differently by learners of various proficiencies.Wehavedoneadetailed studyon the features/factors of a question statement which could perhaps determine its difficulty-level for three learner categories (experts, intermediates, and easy). KEYWORDS: Difficulty level estimation, Item responsetheory,Question generation, Automatic Quiz Generation, Difficulty Ranking, Semantic Similarity I. INTRODUCTION Question difficulty is important in test creationand question analysis. Nowadays, it is widely recognized that test construction is really time-consuming for teachers. The use of Computer Assisted Assessment reduces considerably the time spent By teachers by Constructing examinationspaper. There are many types of assessment or 'testing' to access student's learning curves. However, written examination is the most common approach used by any higher education institutions for students' assessment. Question is an element that is intertwined with the examination. Questions raised in the paper plays an important role in e orts to test the students' overall cognitive levels held each semester. Effective styleof questioning as described by Swart is always an issue to help students attend to the desired learning outcome. Furthermore, to make it effective, balancing between lower and higher-level question is a must Swart Bloom's Taxonomy, created by Bloom has been widely accepted as a guideline in designing reasonable examination questions belonging to various cognitive levels. The hierarchical models of Bloom's are widely used in education fields constructing questions to ensure balancing and student cognitive mastery. II. LITERATURE REVIEW 1. Hochschule fur Technik Stuttgart Schellingstr (2017): Empirically verify thatBloom's taxonomy,astandardtoolfor difficulty estimation during question creation. Question difficulty estimates guide test creation, but are too costly for small-scale testing. We empirically verify that Bloom's Taxonomy, a standard tool for difficulty estimation during question creation, reliably predicts question difficulty observed after testing in a short-answer corpus.Wealsofind that difficulty can be approximated by the amount of variation in student answers, which can be computed before grading. We showthat question difficulty and its approximations are useful for automated grading, allowing us to identify the optimal feature set for grading each question even in an unseen-question setting. Testing is a core component of teaching, and many tasks in NLP for education are concerned with creating good questions and correctly grading the answers. We look at how to estimate question difficulty from question wording as a link between the two tasks. From a test creation point of view, knowing question difficulty levels is imperative: Too many easy questions, and the test will be unable to distinguishbetween the more able test-takers, who all achieve equally good results. Too many hard questions, and only the most able test-takers will be clearly distinguishable from the (low performance result.) IJTSRD21532
- 2. International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470 @ IJTSRD | Unique Paper ID - IJTSRD21532 | Volume – 3 | Issue – 3 | Mar-Apr 2019 Page: 89 2. Neung Viriyadamrongkij and Twittie Senivongse (2017): Online inquiry communities such as Question-Answer Communities (QAC) have captured interest of online users since they can share and search for any information from any place in the world. The number of questions and answers submitted to a popular community can increase rapidly, and that can make it difficult for users who look for the right" questions to answer. That is, from the view of knowledgeable experienced users, theytendtolookforhard challenging questions as an opportunity to share their knowledge and to build respect with the community. Hence it is desirable to distinguish difficult questions from easy ones. Current researches estimate complexity of questions based on the analysis of the features of the QAC without considering the contents of the questions. This paper presents a method to measure question difficulty levels based directly on the question contents. In particular, we analyze the difficulty of terms that appear in a JavaScript- related question, based on the proposed JavaScript concept hierarchy. In an evaluation of the performance of the question difficulty estimation, our concept based measure givessimilar performance to that of the existing measure based on the features of the QAC. 3. Sasitorn Nuthong, Suntorn Witosurapot (2017): This Automatic Quiz Generation system is utterly handy for reducing teachers' workloads in quiz creation.Nevertheless, by exploiting a coarse-granular concern inside difficulty ranking mechanism, only a few number of automatic generated quizzes can be obtained. In order to increase the number of usable quizzes, we suggesthowa5-leveldifficulty ranking score using a hybrid similarity measurement approach together with property filteringofthekeydatacan be potential for serving this propose. Based on experiment results, our proposed similarity measure outperforms three other candidates. Enabling users with finer options of making sensible quiz generation. Hence,this mechanismcan be regarded as a synergistic technology for improving teachers' quality of life for the future. 4. Surbhi Choudhary, Abdul Rais Abdul Waheed, Shrutika Gawandi, Kavita Joshi (2015): In this modern world e-book has become a basic requirement for the candidates to appear and prepare for their competitive exams within college premises. In this paper we are proposing a replica system for smart question paper generation of universities.Themechanismbehindthis system is that many random question papers are generated along with the complexity level of the questions in terms of percentage.[4] After generation that particular question is then mailed to the respective university. In this system administration of the database inputs set of question paper with an option of check box to tick the accurate answer. More ever weightage of the particular question in terms of marks and hours and the complexity of the question is determined. After this course whole question paperall along with the weightage is stored in the database. Inside order to make question paper for 100 marks, admin sets all the weightage and difficulties to solve the problem. When the difficulty and weightage is specified a pre doc le as per selected format will be downloaded to the admin and an electronic mail will be triggered. Variety of difficulty may vary from easy, Medium and hard. 5. Pawel Jurczyk, Eugene Agichtein (2007): Question-Answer portals such as Naverand Yahoo!Answers are rapidly fetching rich sources of information on a lot of topics which are not well served by general web search engines. Unluckily, the quality of the submitted answers is uneven, ranging from excellent detailed answers to snappy and insulting remarks or even advertisements for commercial content. Furthermore, user feed-back for many topics is sparse, and can be insufficient to reliably identify good answers from the bad ones. Hence, estimating the ability of users is a critical task for this rising domain, with potential applications to answer ranking, spam detection, and incentive mechanism design. We present an analysis of the link structure of a general-purpose question answering community to discover authoritative users, and capable experimental results over a dataset of more than 3 million answers from a popular community QA site. We also explain structural differences between question topicsthatcorrelate with the success of link analysis for authority discovery. Existing system: There are very few such system previous available like a method was proposed based on Measuring Difficulty Levels of JavaScript online Questions in Question-Answer Community Based on Concept Hierarchy. III. PROPOSE SYSTEM We propose a system to automate paper setting and to measure the difficulty levels of questions in an Engineering subjects by analyzing the Terminology that appears in the questions. IV. PROPOSE SYSTEM ARCHITECTURE Figure4.1. System Architecture V. CONCLUSION AND FUTURE SCOPE Establishing mechanisms tocontrolandpredictthedifficulty of assessment questions is clearly a big gap in existing question generation literature. Our contributions have covered the deeper aspects of the problem, and proposed strategies, that exploit ontologies and associated measures, to provide a better difficulty-level predicting model,thatcan address this gap. We developed the difficulty level model (DLM) by introducing three learner specific logistic regression models for predicting the difficulty of a given question for three categories of learners.Theoutputof these three models was then interpreted using the Item Response
- 3. International Journal of Trend in Scientific Research and Development (IJTSRD) @ www.ijtsrd.com eISSN: 2456-6470 @ IJTSRD | Unique Paper ID - IJTSRD21532 | Volume – 3 | Issue – 3 | Mar-Apr 2019 Page: 90 Theory to assign high, medium or low difficulty level. The overall performance of the DLM and the individual performance of the three regression models based on cross- validation were reported and they are found to be satisfactory. Comparison with the state-of-the-art method shows an improvement of 8.5 difficulty-levels of benchmark questions. The model proposed in this paper for predicting the difficulty-level of questions is limited to A Box based factual questions. It would be interesting to extend this model to questions that are generated usingtheTBox-based approaches. However,the challengestobead-dressed would be much more, since, in the T Box-based methods, we have to deal with many complex restriction types (unlike in the case of A Box-based methods) and their influence on the difficulty-level of the question framed out of them needs a detailed investigation. For establishing the propositionsand techniques stated in this paper, we have implemented a system which demonstrates thefeasibilityofthemethods on medium sized ontologies. It would be interesting to investigate the working of the system on large ontologies. REFERENCES [1] UlrikePad o Question Difficulty How to Estimate Without Norming, How to Use for Automated Grading" Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, Copenhagen, Denmark, September 8, 2017. [2] Neung Viriyadamrongkij and Twittie Senivongse Measuring Difficulty Levels of JavaScript Questionsin Question-Answer Community Based on Concept Hierarchy"978-1-5090-4834-2/17/31.00 c2017 IEEE [3] Itziar Aldabe Edurne Martinez ArikIturri: an Automatic Question Generator Based on Corpora and NLP Techniques" Conference Paper inLectureNotesin Computer Science June 2006 [4] Sarah K. Two Methods for Measuring Question Difficulty and Discrimination in Incomplete Crowd sourced Data "Association for the Advancement of Artificial Intelligence 2013 [5] Asma Ben Abacha, Marcos Da Silveira, and Cedric Pruski. Medical ontology validation through question answering. In AIME, pages 196{205, 2013. 10.1007/978-3-642-38326-730. [6] Maha Al-Yahya. Ontology-based multiple choice question generation. The Scientific World Journal, Vol 2014, page 9, ID: 10.1155/2014/274949, 2014. [7] T. Alsubait, B. Parsia, and U. Sattler. A similarity based theory of controlling mcq difficulty.In eLearningande- Technologies in Education (ICEEE), 2013 Second International Conference on, pages 283{ 288, Sept 2013. [8] T. Alsubait, B. Parsia, and U. Sattler. Generating multiple choice questions from ontologies: Lessons learnt. In Proceedings of the 11th Inter-national Workshop on OWL: Experiences and Directions (OWLED 2014), volume 1265, pages 73{84, Oct 2014. [9] Tahani Alsubait. Ontology-based multiple-choice question generation. PhD thesis, School of Computer Science, The University of Manchester, 2015. [10] Tahani Alsubait, Bijan Parsia, and UlrikeSattler.Mining ontologies for Analogy questions: A similarity-based approach. Volume849 of CEURWorkshopProceedings. OWL Experiences and Directions, 2012. [11] Tahani Alsubait, Bijan Parsia, and Ulrike Sattler. Generating multiple choice questions from ontologies: Lessons learnt. Volume1265 of CEUR Workshop Proceedings.OWL Experiences and Directions, 2014. [12] Xinming An and Yiu-Fai Yung. Item response theory: What it is and how you can use the irt procedure to apply it. In SAS Global Forum, 2014.