| LIBRES:
Library and Information Science Research Electronic Journal ISSN 1058-6768 2001 Volume 11 Issue 2; September 30 Bi-annual LIBRES 11N2 |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Students’
Perceptions of Online Learning: A Case Study of
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Abstract
With
the emergence of the Internet and its related technologies, many educators
assert that there are substantial benefits to reap from online learning
and educational technology. This study examines the effectiveness
of online learning and to provide insights into the experiences related
by participants in the A total of 657 first-year business students responded to a questionnaire administered as part of this study that examined the issues of the accessibility, usefulness, and effectiveness of online learning and its relation to improvement in subject grades. The findings suggest that the “better” students were more receptive to VBUS, while the “weaker” students found VBUS more of an added burden than an aid to their already heavy workload. There was no clear indication that VBUS played a significant role in improving students’ grades. More positive reactions to VBUS came from students who use VBUS on an average of one to two times a week, with each access lasting between fifteen and thirty minutes, and whose median time spent on studying a subject is around two hours per week. IntroductionOn
VBUS was set up primarily as a vehicle to accomplish two main objectives—to develop online resource-based learning courseware on the Internet and to enable every student to access these media-rich courseware anytime from anywhere. In this respect, courseware cannot be downloaded but can be interactively accessed to facilitate learning. Such objectives are in line with those advocated by Barron in his definition of distance education in which he noted that distance education takes advantage of currently available technologies to achieve the two objectives of teaching and learning by providing equal access to quality education and in meeting the unique learning needs and style of individuals (Barron, 1999). OBJECTIVES OF THE STUDY With the use of VBUS at TP for some time, this study was a timely interjection to assess its effectiveness (Peh, 2000). In particular, this study aims to address four objectives:
METHODOLOGYStephen Ehrmann (1988) has argued that different students learn different things and that their learning cannot be tested on discrete skills tests and be quantified. Thus, assessment of the effectiveness of online resources such as VBUS should be planned in conjunction with defining course goals and instructional designs. Based on this premise, the evaluation of the VBUS project was subsequently carried out using the two methods of formative and summative evaluation. Formative evaluation seeks to identify the sources of problems in order to suggest possible improvements. Its primary goal is to serve as feedback from participants (students). For this reason, data should be collected and analysed continuously and used to guide lecturers or tutors involved with the project. Summative evaluation is conducted for the benefit of external parties, agencies funding research or educational institutions investing in online technologies (Harasim, Hiltz, Teles, and Turoff, 1995). Generally, summative evaluation is conducted for funding purposes. The data collected is usually used to derive generalized conclusions resulting from the online project on which the institution has embarked. Cost-benefits analysis is a significant component of such evaluation. This study emphasizes formative evaluation, and the quantitative survey method was employed to address the aforementioned objectives. The data to be collected is, therefore, mainly concerned with the students’ profile (demographics, computer literacy skills, resources such as personal computer (PC) ownership and Internet access), users’ behavioural patterns in using VBUS (average duration of VBUS sessions, time of access, and frequency of usage), and perceptions about the system. In the end, the data collected should show a pattern of the students’ participation in using VBUS. A more detailed analysis of the data that follows should yield a clearer picture of the type and amount of activity that transpires on VBUS. Thus, the study should reveal both the typical profile of students using VBUS and their perceptions of using the system. Questionnaire Design The questionnaire was structured with four parts comprised of a total of seventeen multiple-choice questions and two open-ended questions (Appendix 1). The questions are logically grouped in a specific order for simplicity and clarity and numbered from 1.1 to 1.17 due to a requirement of the scanning software. The first part of the questionnaire, “General VBUS Access Habits,” contains nine questions directed toward determining students’ access patterns in using VBUS. The second part, “Access to Computer Fundamentals on VBUS,” contains four questions intended to co-relate the use of VBUS to the Computer Fundamentals compulsory core subject taken by all participants in the previous semester and to determine if they thought VBUS played a role in improving learning, culminating in improved test and examination grades in the subject. The third part of the questionnaire contains four demographic-related questions. The final part of the questionnaire is an open-ended section to seek comments and suggestions from participants in order to further improve VBUS and to list new features that are currently unavailable. Findings and DiscussionConvenience sampling was used to administer the questionnaire to all 1057 year-one business students who attended a core subject in the course curriculum during a randomly selected week. Class representatives handed out the questionnaire to students and asked them to complete it before they left the lecture halls. The self-completion method was applied in the data collection process using an Optical Mark Reader (OMR) form. Students were given fifteen minutes to complete the questionnaire. A total of 657 returns from students majoring in various diplomas were collected. The returned data was subsequently scanned using a scanning software that provided a summary of the results. The results were then saved into an MS Excel spreadsheet and analysed in detail using Statistical Products and Service Solutions (SPSS), a statistical package. Profile of Respondents Of the 657 returns, a different number of valid responses was received for each of the seventeen questions. Almost all questions received a ninety-five percent response rate from respondents except for questions 1.8 (eighty-three percent response rate), 1.12 (sixty-six percent response rate) and 1.17 (eighty-six percent response rate). A problem in distinguishing the students’ shaded response was encountered by the scanning software that resulted in the lower number of valid responses for question 1.12. The findings indicate that the majority of respondents were from the diploma course in business (47.9 percent) followed by the diploma courses in accounting and business (eighteen percent), legal studies (12.2 percent), tourism (11.3 percent) and hospitality (10.7 percent). Females accounted for 76.2 percent of the returns with males making up the remaining 23.8 percent.
PC Ownership and Literacy Skills The majority of respondents rated themselves as fair to competent PC users (fifty-nine percent), while twenty-four percent rated themselves as good or very good and the remaining seventeen percent as poor or very poor in PC literacy and competency. A very high proportion of respondents (eighty-eight percent) has a PC at home with access to the Internet. Study Patterns and Internet Access Respondents typically spend approximately between four to six hours (29.1 percent) or two to four hours (23.6 percent) per week studying for the six subjects in their respective diploma courses. The majority of respondents (seventy percent) spend up to two hours per week on the Computing Fundamentals (CF) core course. Access to VBUS VBUS
access was divided roughly between home PC access (fifty-one percent)
and campus access (forty-nine percent). Most respondents accessed
the VBUS servers during school hours between The average duration for each VBUS session is shown in Table 1 for all students across all diploma courses. On average, 34.9 percent of the respondents access VBUS for fifteen to thirty minutes per session, 26.2 percent for fifteen minutes per session, and 22.5 percent for thirty to forty-five minutes per session.
Figure 1. Average Duration of VBUS Sessions by All Users For each VBUS session, the time spent includes a waiting time and an actual access time component. Waiting time is defined as including the time to gain access to VBUS, to log in, to download material, and so on. Actual access time refers to the time when the user is actively engaging VBUS to carry out certain tasks such as interacting with courseware material, reviewing materials found on recommended external links, and so on. The findings indicate that the actual access times for VBUS sessions reported by respondents were relatively low, ranging from twenty to forty percent (thirty percent), zero to twenty percent (twenty-four percent) and forty to sixty percent (twenty-four percent), thus indicating that the amount of waiting time is significant. Respondents were also asked to indicate the VBUS facilities that were used during these sessions. The findings indicate that VBUS was used mainly for downloading lecture material (eighty-two percent). Other uses included newsgroups (30.3 percent), RealMedia (12.6 percent), databases (12.2 percent), and File Transfer Protocols (FTP) (3.8 percent). Perceptions of the Ability of VBUS to Enhance Learning and Improve Grades When asked their perceptions regarding the effect of VBUS on enhancing learning, and, in particular, in improving test and examination grades for all subjects as well as the specific subject Computer Fundamentals (CF), the findings reveal that 39.7 percent (all subjects) and 37.5 percent (CF) of respondents felt the improvement was “somewhat” while 29.5 percent (all subjects) and 31.1 percent (CF) indicated improvement was “not much.” Up to as many as 11.4 percent and 14.2 percent of respondents thought that VBUS did not help at all (Figure 2). The general perception of VBUS appears to be the same, whether for all subjects or for a particular, singled-out subject. Respondents from the accounting and finance course (53.5 percent) appear to indicate a more positive response to VBUS (Figure 3) than their counterparts in the hospitality course, who had the least positive response (29.2 percent) of all groups.
Figure 2. Perception of VBUS Improving Subject Grades
Figure 3. Perception of VBUS Improving Subject Grades (by Course) Suggestions for Enhancement of VBUS The open-ended questions at the end of the questionnaire were used to elicit respondents’ views on ways to improve VBUS, including identifying additional features that are currently unavailable on the system. Many respondents suggested an increase in Internet bandwidth. They claimed that the downloading time from their homes took them too long (no quantitative figures were reported). In their own words, they requested "faster access," "more speed," and to "make the machines go faster." With a T3 Internet bandwidth, the computer systems at TP were already accessing information at the most rapid rate possible. The speed limitations experienced by respondents were imposed by the computer configurations of students who were accessing from their homes. Another suggestion was to introduce an interest-free loan program for students who cannot afford to buy a computer for home. This would enable them to own a PC and thus more conveniently access VBUS. The high percentage of PC ownership (eighty-eight percent) reflected in the current survey seems to indicate that those without PCs see themselves as handicapped when compared with their counterparts who have home PCs. Another possibility is that these respondents were relating concerns of their friends taking other diploma courses at TP. Nonetheless, this particular comment was taken up by the school, which has recently, in collaboration with IBM, offered an interest-free method for students to purchase laptop computers. In terms of enhanced features of VBUS, the main suggestion was a call for a higher degree of interactivity in VBUS. "More interactivity," "more fun stuff," "more chat rooms," "make the materials fun," and "add on more music" were some of the comments offered. These suggestions were taken into consideration, and VBUS has been working in the direction of adding more color and creativity to future versions. Correlation Analysis A correlation analysis was also conducted to explore further the perceptions of VBUS’s role in improving subject grades in general and the grades in the core CF subject (Figures 2 and 3) against other variables to determine if some form of relationship exists between them. The Spearman Correlation was used to calculate the association between these perceptions (Q1.9 and Q1.13) and other variables using a two-tail, ninety-nine percent level of confidence test. This coefficient was chosen because the variables were non-metric, non-ratio/interval and do not assume a normal distribution. It was found that as the frequency of access and duration of each session increases (Q1.11 and Q1.12), the perception of VBUS’s usefulness in improving CF grades becomes more positive. This similar observation also applies to the increase in the number of hours spent in studying CF (Q1.6). Regarding the perception of all the subjects’ courseware on VBUS, it was found that these responses were strongly correlated with whether VBUS courseware was favourably perceived (i.e., similar to results shown in Figure 2). As a result, the perception of all subjects’ VBUS courseware was closely tied to the determinants of the perception of CF courseware identified earlier (Q1.6, Q1.11 and Q1.12). From this, we can conclude that a student who is more hard working in terms of time spent on CF tends to find the VBUS courseware for CF more useful. A student who accesses the CF courseware more regularly and for a longer duration per session tends to find VBUS more useful with a positive impact on improving CF grades. A student who uses the CF courseware on VBUS is also likely to use courseware for all other subjects in VBUS. This appears to point to a “spill over” effect and may indicate that there is a need to adopt an integrated approach to the design of online courseware for all subjects since students expect quality standardisation across all courseware. Computer literacy and PC ownership do not seem to have any impact on the usefulness of VBUS. This could be due to the low level of knowledge needed to use and navigate around VBUS. In addition, the amount of time spent on accessing and using the Internet has no bearing on the perception of the usefulness of the courseware on VBUS. As a further attempt to analyse the data, a CHAID analysis (using the SPSS Answer Tree)—a statistical procedure often used for market segmentation—was applied to derive the significant predictor variables related to the perception of the extent to which VBUS helps to improve CF grades. CHAID is most useful when the variable responses are categorical in nature and a relationship is sought between the predictor variables and a categorical outcome measure. This analysis reveals that the most significant predictor is in the frequency of access to the CF courseware outside lesson hours (Figure 4).
Figure 4. CHAID Analysis for Predictors of Perception of Usefulness of CF VBUS Courseware As expected, it can be seen that those students who seldom or never access the CF courseware did not find it useful. About nineteen percent of those who access it one to two times during the week found it helped to improve the CF grades “greatly” or “quite a lot,” forty-six percent found it somewhat useful, while the remaining thirty-five percent found that it did not really help. Among those who access the courseware more than three times a week, however, the percentage reporting significant improvement more than doubled. About forty-one percent of these reported improvement in CF grades “greatly” or “quite a lot” after using the courseware, while eighteen percent felt the courseware did not help in improving grades. This seems to suggest that time and effort need to be invested by students in the learning process if students wish to reap the rewards of using the courseware. The findings also indicate that the group of students who reported positive improvement in CF grades accessed VBUS an average of one to two times per week with each duration lasting between fifteen and thirty minutes and with a median time spent on studying the subject of around two hours per week. SummaryThis study attempts to examine the perception of students regarding online learning through the VBUS project to determine if it indeed helps students learn and perform better. The findings support the observation that VBUS helps only a certain profile of students, specifically those who place a strong emphasis and effort on their studies. These students are characterized by their higher frequencies of access to VBUS, spending more time interacting with VBUS, and studying more for their subjects than other students. Weaker students appear to perceive VBUS less favorably, some even viewing it as an added burden to their workload. The key variables that may contribute to a positive perception of VBUS are likely to be familiarity and competency. Students who frequently access VBUS become familiar with the system. Once accustomed to VBUS, they are able to navigate through the course materials with ease, and, in turn, enjoy a higher level of appreciation and satisfaction in using the system and learning from it. Conclusion This study, though simplistic and exploratory in nature, has provided some useful insights regarding the VBUS online learning project at TP. It has addressed the relationship between students’ perception of the VBUS system and student grades. When viewed as a tool to facilitate learning, a student’s attitude and approach become important. The findings suggest that different user groups interacting with the system perceive the usefulness of the system differently. The more-hardworking students appear to leverage VBUS to their advantage and satisfaction, while the weaker students are not able to utilize fully the benefits of VBUS and may even find it burdensome. For the future, it is important to determine how VBUS can be enhanced to address the needs and concerns of these different groups of users. One potential solution is to require students to access VBUS on a weekly basis through compulsory structured assessments at the end of each completed module. Another possibility is to enhance the presentation of course materials through improving the degree of interactivity and by introducing more visually appealing materials as well as other aesthetic elements. Aspects of improved human-computer interaction can also be incorporated to improve students’ online experiences. Online learning is a two-way partnership between students and courseware providers (lecturers). Students need to take responsibility for their own learning and expend the effort to utilise regularly the courseware for their advantage and benefit. At the same time, the courseware providers need to ensure that courseware materials exhibit relevance, conciseness, currency, and appropriateness coupled with effective modes of presentation and interaction. References Barron, Daniel. (1996). Virtual School Library—Gateway to the Information Superhighway. Kuhlthau, C. Carol, (ed.), Libraries Unlimited, Inc. Ehrmann, Stephen C. (1988). Adult Learning in a New Technological Era. Organization for Economic Cooperation & Development. Trade Paperback. Harasim, L., Hiltz, S., Teles, L, & Turoff, M. (1995). Learning Networks: A Field Guide to Teaching & Learning Online: MIT Press. Peh, Wee Leng. (2000). “The Impact of Online Learning—A Case Study of the Temasek Business School VBUS project.” MSc. dissertation. Nanyang Technological University: Division of Information Studies. Available: http://www.s-one.gov.sg/overview/s1def01.html [October 2000]. VBUS (2000). Milestones of Temasek Polytechnic. Available: http://www.tp.ac.sg/corporate/milestones/1998.htm [November 2000]. |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Appendix 1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
TEMASEK
POLYTECHNIC VIRTUAL SURVEY OF VBUS PROJECT IN AY1999/00 SEMESTER 1 Dear student, We invite you to participate in a survey conducted to study the general user’s habits in using VBUS as well as the frequency and duration of your access to the Computer Fundamentals online course materials through VBUS during the last Semester 1 (June – December 1999). Please take a few moments to complete the following questionnaire. Please be as accurate and honest as possible with your answers. Your individual responses will be collected and analysed at an overall level. All information supplied will be kept in strict confidence and be solely used for the purpose of this study. Thank you very much for participating in the survey. Instructions (1) Do not write on this questionnaire, but use the OMR form provided. (2) Use a 2B (or darker) pencil to shade your answers on the OMR form. (3) In place of Academic Year, fill in your Surname (4) In place of Subject, fill in your Name. PART I: General VBUS Access Habits 1.1
Where do you access VBUS most of the time, excluding the sessions
in tutorials?
1.2
How do you access the VBUS servers most of the time? Via:
1.3
Which part of the day do you most often access VBUS outside
lecture/tutorial hours?
For Questions 1.4 to 1.6, estimate the following on a weekly basis. Please exclude tutorial and lecture hours in your estimation. Choose and shade one of the following:
1.4
Number of hours that you are logged onto the Internet. 1.7
On VBUS, which of the following utilities do you access?
1.8
On VBUS, which of the following utilities do you access most
often?
1.9 Overall, to what extent do you think VBUS helps in improving your tests and exam grades for the subjects you took in Semester 1 (June-Dec 1999)?
PART II: Access to Computer Fundamentals on VBUS, AY1999/00 Semester 1For Q1.10 - 1.12, please provide estimates based on access outside lecture and tutorials hours. 1.10
For the subject Computer Fundamentals, how often do you access VBUS
for courseware
1.11 Please estimate the duration of each VBUS login session:
1.12 During your VBUS login sessions, time is spent on either:
Roughly what percentage of your time is usually spent on actual access?
1.13 To what extent do you think VBUS helps in improving your test and exam grades specifically for Computer Fundamentals in semester 1 (June-Dec 1999)?
PART III: Tell us about yourself 1.14 Gender
1.15 Rate yourself on your PC skills (literacy and competency):
1.16 Do you have a personal computer (PC) or laptop or notebook at home?
1.17 For the computer system (PC, notebook, laptop, etc.) that resides in your home, does it have Internet capabilities and access?
PART IV: Comments and Suggestions Write the answers for these last two questions on the BACK of the OMR sheet. 2. Write down your suggestions (if any) to further improve VBUS. 3. Please list the features that you would like to see on VBUS that are presently not available. End of Questionnaire. Thank you for participating in the survey. ---------------------- |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
This
document may be circulated freely Wee Leng Peh and Schubert Foo (ASSFOO@ntu.edu.sg) Division of Information Studies, School of Computer Engineering Nanyang
Technological University, To
subscribe to LIBRES send e-mail message to Return
to Libres 11n2 Contents |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
