http://ejel.org EJEL News en © 2011, EJEL Fri, 24 Jun 2011 09:22:09 +0000 Fri, 24 Jun 2011 09:22:09 +0000 EJEL EJEL EJEL RSS generator http://blogs.law.harvard.edu/tech/rss The recent National Students Survey showed that feedback to students was an ongoing problem in Higher Education. This paper reports on the extension of our past research into the provision of automated feedback for objective testing. In the research presented here, the system has been further developed for marking practical and essay questions and providing automated feedback. Recent research at the University of Hertfordshire was able to show that learners and tutors accept and value our automated feedback approach based on objective tests and Computer Adaptive Testing. The research reported in this paper is an important extension to this work. The automated feedback system developed for objective testing has been extended to include practical testing and essay type questions. The automated feedback system, which can be used within any subject area, is based on a simple marking scheme created by the subject tutor as a text file according to a simple template. Marks for each option and a set of feedback statements are held within a database on a computer. As marks are awarded for each question by the teacher an individual feedback file is created automatically for each learner. Teachers may also add and modify comments to each learner and save additional feedback to the database for later use. Each individual feedback file was emailed automatically to learners. The development of the system is explained in the paper and testing and evaluation with 350 first year (1 final practical test), 120 second year (1 written and 1 practical tests) and 100 final year (1 final practical test) undergraduate Computer Science students is reported. It was found that the time to mark practical and essay type tests was reduced by more than 30% in all cases compared to previous years. More importantly it was possible to provide good quality individual feedback to learners rapidly. Feedback was delivered to all within three weeks of the test submission date. In end of module tests it was very beneficial indeed as it had proven difficult to provide feedback in the past after modules had ended. Examples of the feedback provided are presented in the paper and the development of the system using a user-centred approach based on student and staff evaluation is explained. The comments of staff teaching on these modules and a sample of students who took part in this series of evaluations of the system are presented. The results of these evaluations were very positive and are reported in the paper, showing the changes that were made to the system at each iteration of the development cycle. The provision of fast effective feedback is vital and this system was found to be an important addition to the tools available. http://www.ejel.org/volume9/issue1/p1 Trevor Barker http://www.ejel.org/volume9/issue1/p1 Fri, 01 Apr 2011 01:01:01 +0000 The correct grading of free text answers to exam questions during an assessment process is time consuming and subject to fluctuations in the application of evaluation criteria, particularly when the number of answers is high (in the hundreds). In consequence of these fluctuations, inherent to human nature, and largely determined by emotional factors difficult to mitigate, it is natural that small discrepancies arise in the ratings assigned to similar responses. This means that two answers with similar quality may get a different grade which may generate inequities in the assessment process. Reducing the time required by the assessment process on one hand, and grouping the answers in homogenous groups, on the other hand, are the main motivations for developing the work presented here. We believe that it is possible to reduce unintentional inequities during an assessment process of free text answers by applying text mining techniques, in particular, automatic text classification, enabling to group answers in homogeneous sets comprising answers with uniform quality. Thus, instead of grading answers in random order, the teacher may assess similar answers in sequence, one after the other. The teacher may also choose, for example, to grade free text answers in decreasing order of quality, the best first, or in ascending order of quality, starting to grade the group of the worst answers. The active learning techniques we are applying throughout the grading process generate intermediary models to automatically organize the answers still not fixed in homogeneous groups. These techniques contribute to reduce the time required for the assessment process, to reduce the occurrence of grading errors and improve detection of plagiarism. http://www.ejel.org/volume9/issue1/p15 Nuno Escudeiro, Paula Escudeiro and Augusto Cruz http://www.ejel.org/volume9/issue1/p15 Fri, 01 Apr 2011 01:01:01 +0000 Research in automated creation of test items for assessment purposes became increasingly important during the recent years. Due to automatic question creation it is possible to support personalized and self-directed learning activities by preparing appropriate and individualized test items quite easily with relatively little effort or even fully automatically. In this paper, which is an extended version of the conference paper of Gütl, Lankmayr and Weinhofer (2010), we present our most recent work on the automated creation of different types of test items. More precisely, we describe the design and the development of the Enhanced Automatic Question Creator (EAQC) which extracts most important concepts out of textual learning content and creates single choice, multiple-choice, completion exercises and open ended questions on the basis of these concepts. Our approach combines statistical, structural and semantic methods of natural language processing as well as a rule-based AI solution for concept extraction and test item creation. The prototype is designed in a flexible way to support easy changes or improvements of the above mentioned methods. EAQC is designed to deal with multilingual learning material and in its recent version English and German content is supported. Furthermore, we discuss the usage of the EAGC from the users’ viewpoint and also present first results of an evaluation study in which students were asked to evaluate the relevance of the extracted concepts and the quality of the created test items. Results of this study showed that the concepts extracted and questions created by the EAQC were indeed relevant with respect to the learning content. Also the level of the questions and the provided answers were appropriate. Regarding the terminology of the questions and the selection of the distractors, which had been criticized most during the evaluation study, we discuss some aspects that could be considered in the future in order to enhance the automatic generation of questions. Nevertheless the results are promising and suggest that the quality of the automatically extracted concepts and created test items is comparable to human generated ones. http://www.ejel.org/volume9/issue1/p23 Christian Gutl, Klaus Lankmayr, Joachim Weinhofer and Margit Hofler http://www.ejel.org/volume9/issue1/p23 Fri, 01 Apr 2011 01:01:01 +0000 The aim of the paper is to present the scientific model and several methods for the expert evaluation of quality of learning objects (LOs) paying especial attention to LOs reusability level. The activities of eQNet Quality Network for a European Learning Resource Exchange (LRE) aimed to improve reusability of LOs of European Schoolnet’s LRE service for schools are analysed in more detail. As a pan-European service, the LRE particularly seeks to identify LOs that can “travel well” (i.e., reusable) across national borders and can be used in a cultural and linguistic context different from the one in which they were created. The primary aim is to improve the quality of LOs in LRE. eQNet is doing this by establishing a network consisting of researchers, policy makers, and practitioners (teachers) that develops and applies “travel well” quality criteria to both existing LRE content as well as that to be selected in future from national repositories. The vision driving the LRE is that a significant percentage of high quality LOs developed in different countries, in different languages and to meet the needs of different curricula can be re-used at European level. The main problem of all existing approaches in the area is a high level of the expert evaluation subjectivity. The authors analyse several scientific approaches, theories, methods and principles to minimise the subjectivity level in expert evaluation of LOs quality, namely: (1) multiple criteria decision analysis approaches for identification of quality criteria, (2) technological quality criteria classification principle, (c) fuzzy group decision making theory to obtain evaluation measures, (d) normalisation requirement for criteria weights, (e) scalarisation method and (f) trapezoidal fuzzy method for LOs quality optimisation. The authors show that the complex application of these approaches could significantly improve the quality of expert evaluation of LOs and noticeably reduce the expert evaluation subjectivity level. The paper also presents several examples of practical application of these approaches for LOs quality evaluation for Physics and Mathematics subjects. http://www.ejel.org/volume9/issue1/p39 Eugenijus Kurilovas, Virginija Bireniene and Silvija Serikoviene http://www.ejel.org/volume9/issue1/p39 Fri, 01 Apr 2011 01:01:01 +0000 In the globalized economies e-permeation has become a basic condition in our everyday lives. ICT can no longer be understood solely as artefacts and tools and computer-related literacy are no longer restricted to the ability to operate digital tools for specific purposes. The network society, and therefore also eLearning are characterized by fluidity and the key competence for social actors in this ever changing e-permeated environment is the ability to cope with change - or Castells’ conceptualisation self-programming. Castells’ theory has influenced international definitions of future key competencies. Both lifelong learning and digital literacy understood as "bildung" have emerged as central for the definitions of and standards for future key competencies. However, definitions and standards only tell us about the desired destination and outcome of digital competence building. They tell us nothing about how we may get there. In the educational system ICT and e-learning are becoming an everyday condition and the basic challenge for the educational system is twofold: 1) The actually making of digital literate and self-programming social actors – students and teachers; and 2) How to develop adequate designs for teaching and learning for that purpose. We need research that aims to describe the phenomenology of acquiring digital literacy and self-programming in order to be able to identify relevant learning objectives and scaffolding. Findings from such studies are expected to be relevant for eLearning scenarios as well as for ICT and designs for learning in general. This paper presents a case study that aimed to explore the phenomenological appearance of self-programming as agency and learning among postgraduate students who participated in a specially designed eLearning workshop in the autumn 2009. The findings relate to both the individual and collaborative barriers and proactive strategies that come into play among the students. Drawing on the findings, it is argued that the presented workshop design contributes to the networked society’s design for ICT, teaching and learning, as the design – at least for this small group of students – have proved to support the development of digital self-programming as a sustainable competence. In the autumn 2010 the study will be expanded to a larger group of students. http://www.ejel.org/volume9/issue1/p52 Karin Tweddell Levinsen http://www.ejel.org/volume9/issue1/p52 Fri, 01 Apr 2011 01:01:01 +0000 Nowadays, Web based platforms are quite common in any university, supporting a very diversified set of applications and services. Ranging from personal management to student evaluation processes, Web based platforms are doing a great job providing a very flexible way of working, promote student enrolment, and making access to academic information simple and in an universal way. Students can do their regular tasks anywhere, anytime. Sooner or latter, it was expected that organizations, and universities in particular, begin to think and act towards better educational platforms, more user-friendly and effective, where students find easily what they search about a specific topic or subject. Profiling is one of the several techniques that we can use to discover what students use to do, by establishing their user navigation patterns on Web based platforms, and knowing better how they explore and search the sites’ pages that they visit. With these profiles Web based platforms administrators can personalize sites according with the preferences and behaviour of the students, promoting easy navigation functionalities and better abilities to response to their needs. In this article we will present the application of Markov chains in the establishment of such profiles for a target eLearning oriented Web site, presenting the system we implemented and its functionalities to do that, as well describing the entire process of discovering student profiles on an eLearning Web based platform. http://www.ejel.org/volume9/issue1/p63 Alice Marques and Orlando Belo http://www.ejel.org/volume9/issue1/p63 Fri, 01 Apr 2011 01:01:01 +0000 Assessment surveys of students are often conducted in order to evaluate online learning activities. Most surveys measure responses to questions which are based on students’ subjective impressions. The purpose of this study is to examine participants’ assessments made during the transitional phase in an online learning environment which includes blended and fully online courses at a Japanese national university. Students were enrolled in two-unit Master’s or Bachelor’s degree courses which were taught by the same professor. The total number of students with valid survey data was 184 (92 Masters, 67 Bachelors for the blended learning course and 25 Bachelors for the fully online course). A survey questionnaire consisting of 10 questions measured the self-assessments of students’ online learning experiences. Three factors were extracted. There are no significant differences in all factor scores between the beginnings and the ends of the courses. These results show the coherence of students' assessments during the course. The correlation coefficients of the first factor scores (e-learning evaluation) between the beginnings and the ends of the courses are not high, however (Masters: r=0.35, Bachelors for blended learning: r=0.46, and Bachelors for fully online: r=0.33). Therefore, some participants have changed their evaluations between the two surveys. When the differences in factor scores from the initial and final surveys are compared between students who rated the course highly at the beginning (high raters) and students who did not (low raters), the scores for the high raters decrease and the scores for the low raters increase. Also, the relationships between students’ transitions and the metrics of their behaviour were investigated. http://www.ejel.org/volume9/issue1/p75 Minoru Nakayama and Hiroh Yamamoto http://www.ejel.org/volume9/issue1/p75 Fri, 01 Apr 2011 01:01:01 +0000 Improve students' ability to link knowledge with real life practice, through enhancing children or teenagers' ability to think critically by way of making observations, posing questions, drawing up hypotheses, planning and carrying out investigations, analysing data and therefore improve their decision making is an educational challenge. Learning through sports can be effective for developing life skills because sport has a potential to contribute over a wide range and is a discipline that most children like. The constructions of real situations or “Problems” must achieve and incorporate certain aspects such as (a) encourage curiosity, (b) be perceived by students as relevant to their personal goals, (c) represent a motivated challenge, (d) stimulate group collaboration for older students, (e) technological equipment as a way of support, to motivate the learning process, and (f) demonstrate how simple scientific concepts can improve everyday activities. The aim of this paper is to present and evaluate the usefulness of the representative tasks created by a systematic integration of approaches (electronic and non-electronic devices) with interactive situations. Four tasks were applied to 140 children between 6-10 years old at elementary school level. The tasks were constructed considering the follow proposals: (1) promote the benefit of physical activity and (2) explore some science concepts using sport. To evaluate the process effectiveness, two groups were formed, group A was submitted to a more theoretical explanation of the concepts and group B was exposed to problem solving through sport situations. Data were analysed by using quantitative methods. Results show that when children participate in an active way they are more motivated, and the use of their own movement or body to resolve a problem (with electronic devices) contributes for knowledge acquisition by adapting their actions and looking for the best window of possibilities to solve the task situation. Further and longitudinal studies are recommended to consolidate the results. http://www.ejel.org/volume9/issue1/p87 Kelly O’Hara, Paula Reis, Dulce Esteves, Rui Brás and Luísa Branco http://www.ejel.org/volume9/issue1/p87 Fri, 01 Apr 2011 01:01:01 +0000 Research methods modules have become a core component of a range of nursing and allied health professional educational programmes both at pre-qualifying, undergraduate level and at post-qualifying and Masters’ level, in keeping with requirements of professional bodies. These courses are offered both on a full time basis and part time for qualified practitioners working in the field accessing continuous professional development (CPD). Evaluation of these courses suggests that some students find research methods challenging to understand and the pace of sessions demanding, and has highlighted a need for additional ways to support learning and teaching. There are a number of existing electronic resources relating to research methods accessible to students via the internet, which could help to support learning and teaching in this area and meet the wide range of learning styles among students. However, many are not specific to health research. In addition, the quality of content can be variable and use/accessibility unpredictable. This, combined with the need for innovative ways to engage interest in research methods, suggested the need for a new electronic resource for health research, for use within the context of a classroom taught course. The process of developing an interactive resource incorporating a narrative element is described. A narrative approach recognises the power of story in capturing interest and transferring information and offers scope for imagination and intrigue within learning. A story of two fictional health practitioner characters working in a local health centre was created to weave around research methods theory. Interactive elements such as question-and-answer tasks, audio extracts, games and interactive graphics were added to offer varied and stimulating ways of presenting material to meet a range of learning styles. The resource also incorporates a number of self-assessment opportunities to reinforce learning. The use of voices heard in realistic scenarios arising in the health centre anchors learning in everyday practice aiming to help students appreciate the need for evidence and the value of research understanding. http://www.ejel.org/volume9/issue1/p98 Abigail Sabey and Sue Horrocks http://www.ejel.org/volume9/issue1/p98 Fri, 01 Apr 2011 01:01:01 +0000 Metaphors are figures of speech in which a word or phrase that denotes a certain object or idea is applied to another word or phrase to imply some similarity between them. Due to their ability to make speaking and writing more lively and interesting, metaphors have always been popular among students. While metaphors provide significant enhancement of contexts and build upon the sense of community, they can limit the boundaries of the communication between students and teachers. In order to carry out student oriented courses, teachers ought to consider the metaphors students use. In an effort to understand and fill in this communication gap, the authors of this paper have initiated a study that aimed to drive out the e-education students’ metaphors in order to suggest a vision for future e-courses. The authors have designed the “E-Education Metaphor Analysis Survey” that comprised 35 items and captured data about e-education students’ metaphors. The questionnaire was posted on Surveymonkey.com and was distributed to e-education students in two countries: Turkey and Cyprus. 352 students filled the questionnaire. The answers revealed that the metaphors students use are influenced by their way of life, their personal characteristics, their educational background and their feelings. Internet was the most common metaphor used for e-education. A very interesting fact was that 47% of the students considered E-Student to be equivalent to “rich students’ education” and that the term recalled them the metaphor “richness”. Although there were many research studies on common metaphors and their impact on e-education, there were no studies in the literature about eLearning metaphors. This paper presents an innovative approach that focuses on 7 key research questions and represents a first step of a more detailed future project undertaken by the authors. http://www.ejel.org/volume9/issue1/p105 Nazime Tuncay, Ioana Andreea Stanescu, and Mustafa Tuncay http://www.ejel.org/volume9/issue1/p105 Fri, 01 Apr 2011 01:01:01 +0000