1.
Introduction
Online learning environments (OLEs) can be an
extremely effective way of delivering course content to students and
supporting them in their studies. By enabling access to a range of
materials, learning tools and communication facilities, OLEs can be
ideal constructivist learning environments that allow students to become
more actively involved in developing their knowledge and understandings.
However, this is still a relatively new field of study, and research
increasingly shows that students do not always use online environments
in the ways designers and tutors expect or desire.
This paper examines the findings of the
Computer-Based Learning in Petroleum Engineering (CBLPET) project, which
developed online modules for Masters-level, workplace-based students of
Petroleum Engineering and evaluated their usage and effectiveness. The
students who participated in the research did not use the environment
entirely as anticipated, especially in how they progressed through the
course materials and in their use of communication facilities. While
some of these findings were not particularly surprising, they are
certainly worthy of reflection and discussion.
The evaluation of the CBLPET environment described
the ways in which students worked online, and is discussed in Beasley et
al. (in press). This paper takes the findings of this research a stage
further and reflects on the possible reasons why students used the
environment in the ways they did by drawing on the formal evaluation,
other recent research, and informal discussion with the student
participants. A number of limitations in the design and implementation
of the course are identified, and the ways in which the online
environment could be improved to encourage or enable students to make
fuller use of the various online resources are considered.
2.
Overview of online learning environments
This section examines the reported benefits of online
learning environments, and considers examples of actual student usage
from the literature.
2.1
Expected benefits of online learning environments
Taking a constructivist perspective, a learning
environment can be defined as ‘a place where people can draw upon
resources to make sense out of things and construct meaningful solutions
to problems’ (Wilson 1996). Constructivist learning environments
emphasise fostering long-term understanding through meaningful contexts
and interactions that reflect how knowledge is developed and used in the
real world. They are characterised by increased student responsibility,
opportunities for reflection, a focus on realistic tasks, purposeful
collaboration with peers and tutors, exposure to multiple perspectives,
and course materials that go beyond purely abstract descriptions of a
subject domain (Grabinger & Dunlap 1995; Jonassen 1999).
OLEs typically combine hypertext-based course
materials with asynchronous communication facilities, supportive
multimedia, and other interactive features to aid understanding. This
provides a single point of access on the web, facilitating ‘one stop
shop’ learning, with the obvious advantage over traditional environments
that learners can study when and for as long as they want, utilising
whatever resources they require at any particular moment (Hill 2000).
The claims made for the educational value of
hypertext generally rest on the increased level of learner control that
studying with it involves. Within the navigational parameters of
hypertext course material, it is the learner and not the instructor who
determines the order and depth of exploration of the content. It has
been proposed that this allows for more active and reflective knowledge
appropriation, as learners can study according to their own ability
levels and requirements. It is because hypertext allows the learner to
interpret information and choose paths of enquiry in this way that it is
claimed that ‘hypertext is necessarily a constructivist environment’ (Jonassen
1992).
More specifically, the opportunity to present course
material non-linearly means that hypertext may be an ideal medium for
enabling learners to develop a critical understanding of a domain. This
is the central premise of Cognitive Flexibility Theory (CFT) (Jacobson &
Spiro 1995), which advocates designing hypertext course material around
case studies and problem scenarios. Through the provision of alternative
navigational paths, the subject matter can be explored from different
conceptual and thematic perspectives, and always with a focus on
completing the task in hand. CFT contends that knowledge developed this
way is richer in breadth and depth, and so more readily transferable
than that developed ‘linearly’.
While asynchronous discussion facilities also offer,
through dialogue with peers and tutors, the opportunity for students to
appreciate multiple perspectives, their main benefit is in the manner in
which dialogue is enabled. The increased opportunity for students to
reflect on their own opinions and those of others before contributing to
an online discussion has the potential to lead to a deeper, more
reasoned exchange of views than is often possible in real-time
situations (e.g. Mason 1994; McConnell 2000). Allied with this is the
idea that students who are less forthcoming in face-to-face discussions
are more likely to participate online where they are relatively
anonymous, and have ample time to compose messages and be sure of their
communications. There are also obvious advantages for students enrolled
in distance courses, as online discussion may provide the only means for
social interaction and collaboration.
Often integrated within the hypertext of OLEs are
visual representations in the form of static, animated and interactive
graphics, which are generally referred to as ‘supportive multimedia’ as
their content is intended to complement or extend the textual
information (Najjar 1998). Of the many cognitive benefits associated
with supportive multimedia, the two most important from a constructivist
perspective are the authenticity of the learning environment and the
concept of ‘distributed cognition’.
Supportive multimedia can contribute to the
authenticity of a learning environment by presenting objects and
phenomena in forms closer to those encountered in the real world, rather
than describing them solely through text or highly abstract images. This
enables a qualitatively better comprehension of whatever is being
depicted, for example a physical item or process, therefore increasing
the likelihood of it being recognised and understood on future occasions
(CTGV 1993; Honebein et al. 1993).
The idea of distributed cognition is, in short, that
certain tools and artefacts within our environment can enhance our
cognitive abilities and ‘make individuals
smarter…while using them’ (Bell & Winn 2000). In relation to
supportive multimedia, the concern is whether still or moving images
help learners to understand what is being depicted more effectively than
would be possible in their absence (Scaife & Rogers 1996). In relation
to other types of tools and media, for example interactive glossaries
and self-tests, then the concern is similarly with the unique ways in
which these respective features might aid recall or help consolidate
understanding.
2.2
Actual student use of online learning environments
Despite the claims that can be made regarding the
educational potential of OLEs, it is becoming apparent that some
students, often including those who value what the learning environment
has to offer, do not interact with them in a manner conducive to fully
experiencing the benefits.
Many students have a tendency to procrastinate rather
than exploiting the opportunity for self-paced learning that exists
online, which typically leads to them ‘falling behind’ (Hiltz 1997). It
is also common to find that much studying actually occurs offline, and
is largely based around working with printed copies of material (Crook
1997; Ward & Newlands 1998). Furthermore, research into the influence of
learner differences in online contexts tends to indicate that only a
minority of more focused or active students will fully utilise the
materials and tools at their disposal, while the majority limit
themselves to working with core materials and only satisfy the basic
requirements for interacting with other features of their environments
(Light et. al. 1997; Gibbs 1999; Karuppan 2001). Finally, in relation to
online communication, it is widely accepted that students will rarely
participate in asynchronous discussion or collaboration simply because a
facility for this has been provided (Tolmie & Boyle 2000; Salmon 2002).
2.3
Background to the CBLPET Project
The CBLPET project was a one-year EU-funded research
project, completed in March 2003. Using the
ASTEP framework (MacKinnon et al. 1998) for designing learning
environments according to constructivist principles as a starting point,
the aim was to develop an online environment for mature, workplace-based
students of Petroleum Engineering, and to evaluate the effectiveness of
that environment for the target student group and the ways in which they
used it.
The CBLPET environment was designed so that students
had access to the complete materials for the modules online,
interspersed with activities – particularly real-world worked examples
and case studies, and instant-feedback assessment questions. This
combination of reading material and activities could be utilised in two
ways: each chapter of material could be read and the activities used for
testing and consolidating knowledge; or the students could start with
the activities and use them in a more problem-based manner, accessing
material when required to complete an activity.
As students were remote from academic staff,
discussion fora were made available as a means for them to communicate
with staff and peers. A search facility, comprehensive hyperlinked
glossary, and individual progress reports were also implemented and the
environment was rich in colour graphics, although the use of simulation
and animation was limited owing to the short development time.
Two distinct student groups took part in the
research: a group of 12 distance learning students who were already
studying the modules using paper-based materials; and an additional
group of four students who agreed to study the modules online and
participate in a more in-depth evaluation. The opinions of the former
group were evaluated with two questionnaires: a short questionnaire to
gather first impressions and a more in-depth questionnaire later on. The
latter group were evaluated using a series of interviews and a task
walkthrough.
All of the students who participated in the trial had
opted to take part in the study and use the online environment, and had
a fairly high degree of computer literacy and experience of independent
learning. The market analysis showed that the distance version of the
programme is usually undertaken by graduate engineering professionals
working in aspects of the industry, and who want to move into petroleum
engineering specifically. The qualification would be necessary for
advancement in this area, which would usually be accompanied by a
significant salary rise. It is therefore likely that the learners who
participated in the study may be more open to the idea of online
learning, but also more strategic in their approaches to studying by
having extrinsic motivations focused beyond the course itself (Entwistle
1997).
Interviews were also carried out with the company
mentors, academic, managerial, and technical staff, which examined their
expectations of the environment and whether or not they had been met.
3.
Discussion of the student experience
This section examines the key findings of the
evaluation and, drawing upon relevant research, reflects upon the
possible reasons behind them and highlights recommendations for future
practice in the delivery of the programme.
3.1
Preference for linear learning
As noted previously, the learning environment could
be used either with a materials focus or with an activity focus, and
despite the opportunity to explore the material in a more active,
non-linear fashion, students exclusively studied the material linearly.
From the evaluation there was evidence that all students knew that the
non-linear option was available but chose not to make use of it.
A straightforward explanation for this could be that
the students were used to working in a linear fashion, and that it was
lack of experience at working in a non-linear fashion that led to a lack
of use. The idea that some students do not possess the necessary skills
or mindset to learn online effectively, and when presented with online
materials will rely on the tried-and-tested methods of studying they
developed in conventional courses, is an established proposition (Bostock
1998; Ward & Newlands 1998).
Another possible explanation is that although
students were aware of the functionality that enabled them to access the
materials in an activity-based manner, they were not given any specific
information on how to study in this way. Previous research has found
that learners who are provided with clear navigational guidance embedded
within the content of educational hypertexts are much more likely to
explore in a more active manner (Veenman et. al. 1994; Relan & Smith
1996). The likelihood is that this form of support would be required to
ensure the students are not just aware of the non-linear functionality,
but actually know how to use it effectively. Therefore a key future
recommendation would be to provide explicit instructional guidance on
the purpose of these aspects of the OLE, how to operate them, and the
expected benefits of using them.
The individual progress reports were not used at all,
as essentially the students saw no need for a system that told them what
they had yet to study. When studying linearly they could simply start
from where they last left off to ensure that all material was covered –
something that was not so easily guaranteed if the students were working
via the non-linear hyperlinks. It would be interesting to discover
whether the reporting functionality would be of greater perceived value
had a non-linear approach been used, and how the students have would
used the site if they had access to the online materials only.
The students placed a high value on the self-tests
provided at the end of each unit and it is possible that the students
worked linearly in order to take full advantage of these tests, because
if students worked through the material non-linearly it would be more
difficult to ensure that they had covered all the relevant material for
the tests.
One factor students mentioned that could have
increased the non-linear use of the materials would be if they were
structured into timed lessons, because workplace-based students often
had distinct blocks of free time to spend studying. If the materials
were structured into timed units then more effective workplace study
could be facilitated, and adequate time ensured to follow the
alternative paths through the materials. Although the idea of tightly
timed units of online instruction is counter to the idea of self-paced,
needs-based learning via hypertext (Jonassen 1992), on a practical level
it may have been beneficial to the students on this particular course.
Two features of the environment that were used in a
non-linear manner were the hyperlinked glossary, which in particular was
cited as extremely useful, and also the search facility. A possible
explanation for this is that these features provided functionality not
available or easily accessible in the book version, as there is some
evidence to suggest that simple-to-use, immediately responsive hypertext
navigational tools of this kind can encourage more direct interrogation
of content (Najjar 1996).
3.2
Requirement for paper-based materials
The students who were only given access to the online
materials quickly requested an additional paper version and for all
students there was a preference for the paper-based material, used in
conjunction with elements of the online environment. Students
appreciated the flexibility of working in any place that the paper
materials allowed, with most learners studying predominantly on paper at
home and using the online materials at work.
Although both mediums were used, the two formats were
not used in conjunction. This is perhaps a failure of the design of the
online site, which largely mirrors the content of the paper-based
version and was not significantly re-written to go online. The heavy use
of the paper materials, which are linear by their very nature, may have
even encouraged linear usage of the online site, as this would have
enabled the students to transfer easily between paper-based and online
modes of studying.
An improvement to the online environment would be to
add visual, interactive elements that would also not be possible in
print form, such as animation and simulation, which could have two
potential benefits. The students reported going online to study the
supportive graphics, which they found easier to understand than the
print versions because they were in colour and allowed better
differentiation between aspects of the images. Some form of ‘distributed
cognition’ (Bell & Winn 2000) was therefore being experienced as the
graphics supported their learning in this respect. If an increased
number of supportive and interactive elements were added, the main
benefit would be in the enhanced opportunities for learning through such
features. Furthermore, if their presence meant that students would go
online to study them, then a secondary benefit could be in encouraging
more interaction with other aspects of the environment. This is highly
speculative, but what is accepted as certain is that the benefits of
interactive online features can only be realised when students are
actually online to use them (Crook 1997; Taraban et. al. 1999).
Feedback from students was that the site was fast,
usable and largely without technical problems, so technical issues can
be discounted as factors that may have adversely influenced the use of
the online materials. All those who participated in the study were
mature students and would not have grown used to reading online in
childhood, so it may well be the case that in future years, as student
cohorts have more experience of reading and working online, this
preference shifts the other way.
A preference for working with paper has been found in
similar studies (Crook 1997; Ward & Newlands 1998) and, consistent with
these findings, this investigation showed that most learners found
studying at work difficult owing to noise, interruptions, and large
amounts of time spent away from the desk, and favoured working at home
or in the office out of hours. The inability to annotate the web-based
material was also cited as a reason for preferring to work on paper, and
O’Hara & Sellen (1997) provide evidence that students may be
reluctant to make separate, paper-based notes while reading online.
This use of paper-based materials may well indicate a
reliance on ‘traditional’ studying strategies, but equally it could be
seen as learners being active in working around some of the
disadvantages of studying online – the unsuitability of the access
environment, and the reduced scope for making sense of materials through
personal annotations.
Overall, it is apparent that online materials
provision should not be seen as a replacement for the printed materials
currently used on the MSc programme, as the students valued and made use
of the flexibility that having both resources allowed them in fitting
their studying around their work.
3.3
Limited use of communication facilities
During the requirements analysis, the students
expressed a clear desire for online discussion facilities to communicate
with peers and academic staff, which was seen as particularly important
because they were often physically remote. An asynchronous discussion
forum was provided for students to ask subject-specific questions of
academic staff, and the purpose of the discussion boards was
communicated to the students via a guide to studying online that was
available on the web site. However, the students indicated that –
despite being aware the guide was available – most had never actually
looked at it. In practice, no use was made of the discussion facility
for either peer or tutor interaction, although it was used sparingly for
technical information and support.
The main reason given was that the students genuinely
saw no need for such a feature as they could gain all the support
required from the company mentor, and preferred to get that support
face-to-face. However some did report being discouraged by the lack of
activity in the forum, and so were presumably open to the possibility of
using it. Another key reason for the low usage levels is possibly that
the academic staff were not trained in how to use the forum effectively
to encourage worthwhile discussion, and were of the opinion that
students would use the facility if they needed it.
Although the students did not feel disadvantaged in
relation to the non-use of the discussion forum, it is arguable that the
opportunities this offered for interaction could have enriched the
learning experience, and in this respect the non-use of the facility can
be attributed to shortcomings in course design. The main failing was
that no tasks or activities were associated with using the discussion
board. Tasks for facilitating online discussion can take many forms,
from online seminars to more sophisticated collaborative projects, but
it is generally accepted that without there being a reason or purpose to
participate in online discussion, very few students will (Tolmie & Boyle
2000; Salmon 2002). To make online discussion purposeful, it is
essential that students are provided with a clear understanding about
the task or activity to be undertaken, and also what is expected of them
as participants. Although usage scenarios were provided in the training
materials, students were not overtly encouraged to make use of these
materials and, in addition, student use of the discussion boards was not
mandatory, which would have at least ensured some level of
participation.
The course could be improved by developing specific
learning activities based around the discussion forum to encourage peer
and tutor interaction and foster a sense of community, and by providing
explicit instructions and indication of purpose.
3.4
Use of self-assessment activities and worked examples
As regards the more active ways in which the students
interacted with the environment, the
vast majority studied all of the core reading material, using a
combination of paper-based and online learning, and also used
extensively the self-assessment exercises and worked examples that were
embedded throughout the materials. The activities were used
predominantly to test and consolidate understanding of what had just
been studied.
The reflective activities, worked-examples and quick
quizzes were all perceived as useful aids to enhancing and, where
applicable, testing understanding. Peat (2000) observes how in online
and other independent learning contexts, students often find
self-assessment opportunities invaluable as they provide for the kind of
responsive support for evaluating understanding that is often readily
available in conventional courses but which would otherwise be missing
in online equivalents.
This seems to be the likely reason why the
self-assessment tools and activities in the course where so heavily
used, and why the students were forthcoming in their requests for more
interactive online elements to support various aspects of their
learning.
For future developments it is important that the
online environment be used strategically to integrate added-value
interactive elements that complement and enhance the paper-based
materials. The evaluation brought forward several ideas for
interactivities that would aid understanding and learning. These
included glossary word-matching quick quizzes to test knowledge of the
complex terminology within the subject area; animated graphics of
geological and time-dependent processes where appropriate; the ability
to isolate separate data segments on complex graphs to simplify the data
representation; and the provision of hints, such as bringing up a
solution to a problem a line at a time so that keen learners can pick up
the problem from where they understand it to increase learner control
over how much relevant support they wish to receive.
4.
Conclusions
Although the students in this study did not use the
online learning environment in all the ways intended they did perceive
the environment to be a valuable resource. Yet from an instructional
perspective they were arguably not using the it in the most effective
ways, and in part this can be explained by the lack of explicit guidance
provided to the students, many of whom were new to learning online.
However, while it is likely this would have helped to improve the ways
in which the environment was used, ultimately it should be recognised
that the expectations of tutors are often different from the
requirements of the students.
It is felt that the lessons learned from reflecting
on the evaluation of this programme may have some practical worth for
those involved in the delivery of online courses, and who wish to
increase the likelihood of all students in a course utilising an online
environment as effectively as possible.
When developing an online learning environment, it is
important to:
·§
consider the
likely educational experiences of students, particularly in relation to
online learning, to identify the kinds of online learning support
required;
·§
give explicit
instructional guidance, within the environment itself, on the purpose of
aspects of the environment and how to operate them, including specific
guidance on how to study course material via non-linear hypertexts;
·§
include
appropriate supportive visual and interactive multimedia to increase
potential opportunities for enhanced learning and transferable
knowledge;
·§
provide
self-assessment tools and materials to allow students to test their
understanding and reflect on the effectiveness of their learning;
·§
ensure there
is a purpose, in the form of clearly communicated tasks or activities,
for students to participate in online discussion.
References
Beasley, N.,
Ford, J. and Tomes, N. (In Press). “Enabling Postgraduate Learning in
the Workplace”, in Etrain: Training Solutions for Professional
Organisations, Kluwer.
Bell, P. and
Winn, W. (2000) “Distributed cognitions, by nature and design”, In
Theoretical foundations of learning environments, D.H. Jonassen and
S.M. Land (Eds), Lawrence Erlbaum, pp 123-144.
Bostock, S.J.
(1998) “Constructivism in mass higher education: a case study”,
British Journal of Educational Psychology, Vol 29, No 3, pp 225-240.
Crook, C. K.
(1997) “Making hypertext lecture notes more interactive: undergraduate
reactions”, Journal of Computer Assisted Learning, Vol 13, pp
236–244.
CTGV
(Cognition and Technology Group at Vanderbilt) (1993) “Integrated media:
toward a theoretical framework for utilizing their potential”,
Journal of Special Education Technology, Vol 12, No 2, pp 71-85.
Entwistle, N.
(1997) “Contrasting perspectives on learning”, in The experience of
learning: implications for teaching and studying in higher education
(2nd ed.), F. Marton, D. Hounsell & N. Entwistle (Eds.), Scottish
Academic Press, pp 3-22.
Gibbs, G.
(1999) “Learning to learn in a virtual learning environment for
philosophy”, Journal of Computer Assisted Learning, Vol 15, No 3,
pp 221-231.
Grabinger, R.S.
and Dunlap, J.C. (1995) “Rich environments for active learning: a
definition”, ALT-J, Vol 3, No 2, pp 5-34.
Jacobson, M.J.
and Spiro, R.J. (1995) “Hypertext learning environments, cognitive
flexibility and the transfer of complex knowledge: an empirical
investigation”, Journal of Educational Computing Research, Vol
12, No 5, pp 301-333.
Jonassen, D.H.
(1992) “Cognitive flexibility theory and its implications for designing
CBI”, in S. Dijkstra, H.P.M. Krammer and J.J.G Van Merrienboer (Eds),
Instructional models in computer-based learning environments,
Springer-Verlag, pp 385-403.
Jonassen, D.H.
(1999). “Designing constructivist learning environments”, in
Instructional design theories and models: a new paradigm of
instructional theory, C.M. Reigeluth (Ed), Lawrence Erlbaum, pp
215-239.
Honebein,
P.C., Duffy. T.M. and Fishman, B.J. (1993) “Constructivism and the
design of learning environments: context and authentic activities for
learning”, in Designing environments for constructivist learning,
T.M. Duffy, J. Lowych and D.H. Jonassen (Eds), Springer-Verlag, pp
87-109.
Hill, J.R.
(2000) “Web-based instruction: prospects and challenges”, in
Educational Media and Technology Yearbook (Volume 25), R.M. Branch
and M.A. Fitzgerald (Eds), Libraries Unlimited, pp 141-155.
Hiltz, S.R.
(1997) “Impacts of college-level courses via asynchronous learning
networks: some preliminary results”, Journal of Asynchronous Learning
Networks, Vol 1, No 2, pp 1-19.
Karuppan, C.M.
(2001) “Web-based teaching materials: a user’s profile”, Internet
Research Electronic Networking Applications, Vol 11, No 2, pp
138-148.
Light, P.,
Colbourn, C. and Light, V. (1997) “Computer-mediated tutorial support
for conventional university courses”, Journal of Computer Assisted
Learning, Vol 13, pp 228-235.
Mason, R.
(1994) Using Communications Media in Open and Flexible Learning,
Kogan Page.
MacKinnon, L.
M., McAndrew, P. and Flockhart, S. (1998) ASTEP Framework User Manual,
[online],
http://www.cee.hw.ac.uk/~astep/deliverables/d24v12.pdf
McConnell, D.
(2000) Implementing Computer Supported Cooperative Learning (2nd
Edition), Kogan Page.
Najjar, L.J.
(1996) “Multimedia information and learning”, Journal of Educational
Multimedia and Hypermedia, Vol 5, No 2, pp 129-150.
Najjar, L.J.
(1998) “Principles of educational multimedia user interface design”,
Human Factors, Vol 40, No 2, pp 311-323.
O’Hara, K. and
Sellen, A. (1997) “A
Comparison of Reading Paper and On-Line Documents”, Proceedings
of CHI-97, Special Interest Group on Computer-Human Interaction.
Peat, M.
(2000) “Online self-assessment materials: do these make a difference
to student learning?”, Vol 8, No 2, pp 51-57.
Relan, A. and
Smith, W.C. (1996) “Learning from hypermedia: a study of situated versus
endemic learning strategies”, Journal of Educational Multimedia and
Hypermedia, Vol 5, No 1, pp 3-21.
Salmon, G.
(2002) E-tivities: the key to active online learning, Kogan Page.
Scaife, M. and
Rogers, Y. (1996) “External cognition: how do graphical representations
work?”, International Journal of Human-Computer Studies, Vol 45,
pp 185-213.
Taraban, R.,
Maki, W.S. and Rynearson, K. (1999) “Measuring study time distributions:
implications for designing computer-based courses”, Behavior Research
Methods, Instruments and Computers, Vol 31, No 2, pp 263-269.
Tomlie, A. and
Boyle, J. (2000) “Factors influencing the success of computer-mediated
communication (CMC) environments in university teaching: a review and
case study”, Computers and Education, Vol 34, No 2, pp 119-140.
Veenman, M.V.J,
Elshout, J.J. and Busato, V.V (1994) “Metacognitive mediation in
learning with computer-based simulations”, Computers in Human
Behavior, Vol 10, No 1, pp 93-106.
Ward, M. and
Newlands, D (1998) “Use of the Web in undergraduate teaching”,
Computers & Education, Vol 31, pp 171–184.
Wilson, B.G. (1996) “What is a
constructivist learning environment?”, in Constructivist learning
environments: case studies in instructional design, B.G. Wilson
(Ed), Educational Technology Publications, pp 3-8 |
