1.
Introduction
There are numerous clear theoretical advantages of online instructional methods.
Firstly, such methods provide for flexible learning, meaning that the student
can progress at his or her own pace; secondly, such methods provide the facility
for student centred learning, making the student responsible for his/her own
learning. Finally, implementing online methods of instruction, means that
material can be made available on demand from anywhere at any time provided the
learner has the facility for taking advantage of such a system.
A
variety of different online learning paradigms are now being utilised across
higher education and therefore it would now seem timely to evaluate such systems
in terms of their effectiveness. Three online methods are utilised in this
paper. These are a literature search, an online discussion and an online
assessment system. These three methods were chosen as being representative of
the types of tasks students typically engage in through the medium of
e-learning. It is also suggested that individual difference factors such as
attitudes towards computer-based learning and cognitive learning style may be
relevant to include in this investigation. The rationale for this is given below
in sections 1.2 to 1.4. However, firstly a description of cognitive style is
given.
1.1
Cognitative Style
Riding (1991) suggested that all cognitive styles could be categorised according
to two orthogonal dimensions. These are the wholist-analytic dimension and the
verbaliser-imager dimension.
1.1.1
Wholist-analytic style
Wholist-analytic cognitive style can be defined as the tendency for individuals
to process information either as an integrated whole or in discrete parts of
that whole. In practical terms, analytics are able to apprehend ideas or
concepts in parts, but have difficulty integrating such ideas into complete
wholes. However, wholists are able to view ideas as complete wholes, but are
unable to separate these ideas into discrete parts (see Figure 1).
Figure 1:
Analytic and Wholist views of information (Riding, 1991)
1.1.2
Verbaliser-imager cognitive style
The verbaliser-imager cognitive style can be defined quite simply as an
individual’s tendency to process information either in words or in images.
Verbalisers are superior at working with verbal information, (Riding and
Mathias, 1991; Riding and Watts, 1997) whereas imagers are better at working
with visual and spatial information.
Both the wholist-analytic and verbaliser-imager cognitive styles can be assessed
using the Cognitive Styles Analysis (CSA) detailed in section 2.2.1 below.
1.2
Online literature search
Searching for information sources online is now a skill with which most
undergraduate students have to be familiar. Previous research suggests that the
skill of searching for information is in some respects related to cognitive
style. For example, cognitive style differences have been noted in searching for
information in a database and this topic was investigated by Ford, Wood and
Walsh (1994) and Wood, Ford and Walsh (1992). In these studies, searching
strategies were classified in terms of relative breadth or depth. A high usage
of the operator ‘OR’ to link keywords represents a relatively broad strategy,
whereas a use of ‘AND’ a relatively narrow strategy. Their results showed that
wholistic learners displayed a broader approach than analytic learners, in that
they made significantly greater use of OR in searching. However, they also used
more truncation than analytic learners, and made more use of ‘AND’, a finding
not in accord with their hypothesis. While the issue of the use of different
search strategies between individuals with different cognitive styles seems
unsettled, the success rate at searching for information may yield more useful
data. It is this issue that the current study seeks to address.
1.3
Online discussion
It would seem to be generally accepted that educational environments where
students interact in seminars leads to good collaborative learning. Research
shows that there are clear educational advantages to be derived from
collaborative learning activities (Del Marie Rysavy and Sales, 1991; Slavin,
1996). When students work in groups and small teams, the interactions and
activities frequently involve higher order and reflective thinking. Face to face
talk therefore theoretically assists students to share knowledge and
interactions often lead to the creation of new ideas.
However, the issues surrounding online discussion are perhaps less well
understood. In a traditional face-to-face environment, support for learners can
be provided immediately. Yet, with online systems, support for learners in the
form of interaction with instructors is not always so immediate.
Furthermore, in computer-based learning environments, the language through which
new ideas are expressed are reduced to print and graphics and interactions
between learners and instructors are reduced to levels that can be supported by
the technology. Also, in online discussion sessions, other factors such as
non-verbal cues are removed, making discussion between participants more
difficult.
Given these factors it is pertinent to investigate whether attitudes to
educational technology and cognitive style are useful learner characteristics to
take into account when designing learning environments that include an element
of online discussion. This is principally because cognitive style also has a
bearing on the way in which individuals interact socially. For example
verbalisers are typically more outgoing than imagers (Riding, 1991), therefore
it is theoretically possible that verbalisers will be less reluctant to engage
in online discussion compared to imagers.
1.4
Online assessment
Online assessment may be defined as a method of using computers to deliver and
analyse tests or exams and such systems have been around since the seventies.
Yet in many ways the internet provides a new way of delivering assessment
material. This is because it is independent of time and place. Assessment can
essentially be divided into two types. Firstly, formative assessment at the end
of a period of study, whereby the results are used in order to determine
examination outcome. Secondly, summative assessment, which is an assessment
which may be administered during the presentation of a course as a means of
checking on student learning. Furthermore, students may also assess themselves
periodically in order to check on progress.
Within any assessment system question types may vary. For example, questions may
include short essay type questions, true or false type questions, or
multiple-choice questions. There are many potential advantages of online
assessment to learners. For example, tests are available on demand and at any
time. Furthermore, computerised assessment systems give immediate feedback to
the user; therefore users learn by taking the test. However, online assessment
systems also have a drawback in that students who perceive themselves as
possessing poor IT skills may be disadvantaged. Therefore a study of individual
differences in attitudes towards computer-based learning is relevant here.
Furthermore, individual differences in approach to different question types have
been found between individuals possessing different cognitive styles, (Riding
and Read, 1996) and therefore it is possible that this may have an impact on the
success with which they engage with online assessment.
1.5
Summary
In summary then, this study seeks to evaluate by comparing student attitudes
towards computer-assisted learning, cognitive style and student feedback, three
different types of online learning and assessment methods, an online literature
search, an online discussion, and finally an online assessment system.
2.
Method
2.1
Participants
Participants in this study were fifty, first year undergraduate university
students, (9 males and 41 females). The mean age was 23.24 with a standard
deviation of 7.49. Ages ranged from 18 to 46. All participants were single
honours psychology students who received credit for participation in this study.
2.2
Instruments
2.2.1
Cognitive Styles Analysis (Riding, 1991)
The Cognitive Styles Analysis is a computer presented test used to determine an
individual’s position on the Wholist-Analytic and Verbal-Imagery style
dimensions. It consists of three subtests. The first contains items relating to
the verbaliser-imager style, the second set of items relates to the wholist
dimension of style and the third set of items relates to the analytic dimension
of style. The test taker is required to react by simply pressing either a ‘true’
or ‘false’ button in response to each question item. The computer then
calculates an individual’s position on each style dimension by comparing
response times between the verbal and imagery items and the wholist and analytic
items on the test.
Test-retest reliability of this instrument as reported by Peterson et al (2002)
is as follows. For the verbaliser-imager scores (r=0.70 p <0.00) and for the
wholist-analytic scores (r=0.81, p <0.00). For the purpose of data analysis, WA
categories of wholist, intermediate and analytic were identified according to
the following scores, <1.02 wholist, 1.03 - 1.35 intermediate, >1.36
analytic. The VI categories of verbaliser, bimodal and imager were identified as
<0.98 verbaliser, 0.99 - 1.09 bimodal and >1.10 imager. This
procedure is according to the standardisation scores for this style dimension
(Riding, 1991).
2.2.2
Computer Attitude Test (Smalley, Graff and
Saunders 2001)
This computer attitudes test developed by Smalley Graff and Saunders (2001)
consists of thirty seven items assessing three components of attitudes towards
computers, namely, affective, behavioural and cognitive. Responses to each item
are made on a five point Likert type scale.
Firstly, internal consistency was calculated using Cronbach’s Alpha for each of
the three components, affective (0.93), behavioural (0.65) and cognitive (0.65).
These coefficients indicate a high level of internal consistency for the each
attitude component. Cronbach’s Alpha for the original development study are
affective (0.95), behavioural (0.71) and cognitive (0.88) and total (0.95).
Correlations were calculated for the scores between each of the four components,
and with the total score. These are shown in table 1 below.
Table 1:
Correlations between attitude components
|
|
Behavioural |
Cognitive |
Total |
|
|
|
|
|
|
Affective |
0.52** |
0.76** |
0.94** |
|
Behavioural |
|
0.74** |
0.75** |
|
Cognitive |
|
|
0.90** |
|
|
|
|
|
** p<0.01
The correlations between the scores on each of the four components of the scale
and with the total score indicate that the components and the scale are
significantly correlated with each other. All correlations reach significance at
p<0.01, illustrating that each component contributes to the total score.
Test retest reliability from the original development study (Smalley, Graff and
Saunders 2001) is (r=0.84 p <0.001).
3.
Online literature search
3.1
Procedure
This study involved an online search whereby participants were required to
retrieve information in response to fifteen questions, the answers to which
could be found on the WWW. Typical tasks involved retrieval of simple pieces of
information such as the names of journal editors etc. Participants were awarded
1 point for each completely correct answer to any of the questions. No strict
time limit was set for the search activity.
3.2
Results
3.2.1
Attitudes to computers
Firstly, Table 2 presents the correlations between each attitude component and
total attitude score with the scores achieved for the literature search. None of
the correlations are significant indicating no relationship exists between
attitudes to computers and the literature search task.
Table 2:
Correlations between attitudes to computers and scores for the litertaure search
task
|
|
Affective |
Behavioural |
Cognitive |
Total Attitude |
|
Literature Search |
0.06 |
-0.07 |
0.01 |
0.03 |
3.2.2
Cognitive style
Figure 2 displays the mean scores for literature search task performance and
wholist, intermediate and analytic cognitive styles.
Figure 2:
Wholist-analytic cognitive style, and scores for search performance
Intermediates performed best whereas wholists performed least well. A one-way
ANOVA was carried out for wholist, intermediate, analytic cognitive styles for
search performance scores, however, the results did not reach significance.
Figure 3 displays the mean scores for literature search task performance and
verbaliser, bimodal and imager cognitive styles.
Figure 3:
Verbaliser-imager cognitive style, and scores for search task performance
Bimodals performed best whereas imagers performed least well. A one-way ANOVA
was carried out for verbaliser, bimodal and imager cognitive styles for search
performance scores, however, the results did not reach significance.
3.2.3
Student Evaluation Questionnaire data
Finally, Figure 4 shows participant ratings for the literature search task.
Figure 4:
Student ratings for library search task
No statistical analysis was performed here, however, the results illustrate that
most participants rated this type of task as good.
4.
Online discussion
4.1
Procedure
This study involved students engaging in an online discussion about a question
set by their lecturer. Students were awarded a score for the amount of
substantive discussion engaged in during this task.
4.2
Results
4.2.1
Attitudes to computers
Table 3 presents the correlations between each attitude component and total
attitude score with the scores awarded for the online discussion. None of the
correlations are significant indicating no relationship exists between attitudes
to computers and ability at the online discussion task.
Table 3:
Correlations between attitudes to computers and scores for the online discussion
task
|
|
Affective |
Behavioural |
Cognitive |
Total Attitude |
|
Online Discussion Scores |
0.01 |
-0.02 |
0.10 |
0.02 |
4.2.2
Cognitive style
Figure 5 displays the mean scores for the online discussion and wholist,
intermediate and analytic cognitive styles.
Figure 5:
Wholist-analytic cognitive style, and scores for online discussion
Intermediates performed best whereas wholists performed least well. A one-way
ANOVA was carried out for wholist, intermediate, analytic cognitive styles for
search performance scores, however, the results did not reach significance.
Figure 6 displays the mean scores for the online discussion and verbaliser,
bimodal and imager cognitive styles.
Figure 6:
Verbaliser, bimodal, imager cognitive style, and scores for online
discussion
Bimodals performed best on this task, whereas imagers performed least well. A
one-way ANOVA was carried out for verbaliser, bimodal and imager cognitive
styles for online discussion scores. An effect approaching significance was
observed (F (2,40) = 3.11, p = 0.06). A Tukey post hoc test indicated
significant differences between bimodals and imagers. However, there were no
significant differences observed between verbalisers and bimodals or between
verbalisers and imagers.
5.
Online Assessment
5.1
Procedure
This study involved participants answering questions online regarding
information from a module they were taking. A variety of question types were
utilised in this part of the project which were free response, true / false
questions, multiple-choice questions and an essay question. Some questions gave
immediate feedback on the accuracy of the answer and others did not.
Participants were awarded points for correct responses. No time limit was set
for this activity.
5.2
Results
5.2.1
Attitudes to computers
Table 4 presents the correlations between each attitude component and total
attitude score with the results for the online assessment. None of the
correlations are significant indicating no relationship exists between attitudes
to computers and results for the online assessment.
Table 4:
Correlations between attitudes to computers and scores for the online assessment
|
|
Affective |
Behavioural |
Cognitive |
Total Attitude |
|
Online Assessment Scores |
-0.22 |
-0.10 |
-0.18 |
0.28 |
5.2.2
Cognitive style
Figure 7 displays the mean scores for the online assessment and wholist,
intermediate and analytic cognitive styles.
Figure 7:
Wholist-analytic cognitive style, and scores for online assessment
The performance of wholists and analytics was approximately equal although the
performance of intermediates is inferior to the other two styles. A one-way
ANOVA was carried out for wholist, intermediate, analytic cognitive styles for
online assessment scores. A significant effect was observed here, (F (2, 38) =
3.91, p < 0.05). A Tukey post hoc test indicated significant differences between
wholists and intermediates. However, there were no significant differences
observed between wholists and analytics or between analytics and intermediates.
Figure 8 displays the mean scores for the online assessment for verbaliser,
bimodal and imager cognitive styles.
Figure 8:
Verbaliser, bimodal and imager cognitive style, and scores for the online
assessment
Very little difference can be observed between verbalisers, bimodals and
imagers. A one-way ANOVA revealed no significant differences between cognitive
styles.
5.2.3
Student Evaluation Questionnaire data
No statistical test was performed for this part of the study, however, Figure 9
shows participant ratings for the online assessment. The results illustrate that
the general response to online assessment was generally good.
Figure 9:
Student ratings for online assessment task
6.
Discussion
The overall findings from this investigation suggests that attitudes toward
computers are not related to performance on each of the online tasks employed
here, although there are some connections between cognitive style and
performance on these tasks.
For the online search task, the results show no relationship between attitudes
towards computers and performance at this task. Similarly, the results show no
differences in performance on the online search task for participants identified
as possessing different cognitive styles. The findings of Ford, Wood and Walsh
(1994) and Wood, Ford and Walsh (1992) suggested that individuals possessing
different cognitive styles employ different search strategies. If this were the
case in this study, then this did not result in differences in performance. The
evaluation questionnaire data for perceived usefulness of this task however,
suggests that the majority of participants found it useful.
For the online discussion, the results again show no relationship between
attitudes to computers and online discussion performance. However, for cognitive
style wholists outperformed analytics, which is consistent with the idea of
wholists, being typically more outgoing than analytics (Riding, 1991).
Furthermore, a relationship approaching significance was noted between and
cognitive style and the online discussion task, with bimodals outperforming
verbalisers and imagers. On a more practical note, several issues were
encountered in the implementation of this activity. Firstly, it took students a
little time to get used to this system of online discussion, when they were more
used to face to face interaction. Furthermore, management of such a system of
seminars required extra time from the tutor in judging just when to contribute a
comment in order to keep the discussion active. However, one of the advantages
of this activity was that because the tutor monitored the contribution to the
discussion by students, all students were encouraged to contribute. Those who
might naturally be more reserved had the opportunity to consider their
contributions rather than being forced to make them too spontaneously. More work
on the techniques involved in such a system is however needed in order to make
improvements.
For the online assessment, the results revealed, as with the above tasks, that
no relationship was evident between attitudes to computers and performance.
However, a significant effect was noted for wholist-analytic cognitive style
with analytics and wholists outperforming intermediates. This would seem
therefore to be an important consideration for the design of such systems. No
differences were observed between individuals with a verbaliser bimodal or
imager style. Further research looking at the methods of online assessment would
need to focus on the types of questions preferred and performed best by
individuals with different cognitive style characteristics.
Generally, it is suggested that the overall culture of using online methods for
instruction is an issue which needs to be assessed. Traditionally, courses are
taught without online support, and one of the areas would seem to involve
educating students to utilise online methods more readily.
7.
Conclusion
This study looked at three different areas of online delivery and methods of
assessment, which were online searches, an online discussion and an online
assessment system. These methods were chosen as being the types of task with
which learners would typically engage throughout higher education. In terms of
individual differences in the efficacy of such methods the results may be
summarised as follows. Few differences were found on each of the three tasks
between individuals with differing attitudes towards computers. However, some
differences were found between individuals identified with different cognitive
styles. Evaluation of the methods used from the participants in this study was
generally positive.
References
Del Marie Rysavy S.,
and Sales, G. C. (1991) ‘Cooperative learning in computer-based instruction’
Educational Technology Research and Development, Vol. 39, No. 2, pp.70-79.
Ford, N., Wood. F. and
Walsh, C. (1994) ‘Cognitive styles and searching’. Online and CDROM review,
Vol. 18, No. 2, pp.79-86.
Peterson, E. R. Deary,
I. J. and Austin, E. A. (2002) ‘The reliability of Riding’s cognitive styles
analysis test’ Personality and Individual Differences (in press)
Riding, R. J. (1991) Cognitive Styles Analysis User Manual. Birmingham:
Learning and Training Technology.
Riding, R. J. and Mathias, D. (1991) ‘Cognitive styles and preferred learning
mode, reading attainment and cognitive ability in 11-year-old children’
Educational Psychology, Vol.11, pp. 383-393.
Riding, R. J. and Watts, M. (1997) ‘The effect of cognitive style on the
preferred format of instructional material’ Educational Psychology, Vol.
17, Nos. 1&2 pp. 179-183.
Riding, R. J. and Read, G. (1996) ‘Cognitive style and pupil learning
performances’ Educational Psychology, Vol. 16, pp. 81-106.
Slavin, R. (1996) ‘Research and Cooperative Learning and achievement: what we
know, what we need to know’ Contemporary Educational Psychology, Vol. 21,
pp. 43-69.
Smalley, N., Graff, M. G. and Saunders, D. (2001) ‘A revised computer attitude
scale for secondary students’ Educational and Child Psychology, Vol. 18,
No. 3, pp. 47-57.
Wood, F., Ford, N. and
Walsh, C. (1992) ‘Online searching and cognitive styles’ Final report to the
British Library. London: British Library. |
