|
|
Learning Technology publication
of IEEE Computer Society's |
|
|
Volume 9
Issue 2 |
ISSN
1438-0625 |
April 2007 |
Learners’ profiling by data driven
approaches
HotKey Coach: Training users to
interact efficiently with software applications
Evaluation of Training – A Critique
of Kirkpatrick’s Evaluation Model
Welcome to the April 2007 issue of Learning Technology.
Many advances in learning technologies are happening throughout the world. This issue focuses in bringing these new developments and emerging technologies to the readers. This issue contains papers ranging from practical learning technology solutions to evaluation of training.
Wanless-Sobel and Alcorn looks at ways to increase the student participation in
discussion forums in eLearning with the use of student mentors. Wanless-Sobel
and Alcorn argues that by removing the residual hierarchy in online
instruction student participation in discussion forums will be increased.
Hence, they use student mentors as facilitators of these forums instead of
lecturers to find out if student participation increases. Viola et. al. looks
at learner profiling using data driven approaches. Colace and DeSanto outlines
the URO Project which is designed to use eLearning as a bridge between the
patient and a health professional. eLearning bridge in this project provides
the patients with the necessary answers from a professional who works at home, in the community
and in the hospital.
Bose
provides a critical analysis of Kirkpatrick’s Evaluation Model while Veglis describes how Microsoft
Homepage Starter Kit can be used to teach Content Management Systems. Finally,
Krisler and Alterman look at a system which trains users to become experts in
using software.
This newsletter focuses publishing new and emerging technologies in education focussing on advanced learning technologies and its usage in different contexts. Please feel free to bring forward your ideas and views.
Besides, if you are involved in research and/or implementation of any aspect of advanced learning technologies, I invite you to contribute your own work in progress, project reports, case studies, and events announcements in this newsletter. For more details, please refer author guidelines at http://www.ieeetclt.org/content/authors-guidelines.
|
Ali Fawaz
Shareef, PhD Director, Centre for Open Learning |
Abstract: Current
pedagogical theory promotes high social presence and democratic space in online
instruction. This case study shares an experiment with four online classes
featuring an online student mentor to increase social presence and democratic
contact zone quotients by way of the mentor’s coaching personality as well as
the creation of a “safe house” discussion forum in each class, completely
administered by the mentor. The preliminary trial indicates increased social
presence, along with increased student satisfaction with the online learning
experience, although a good portion of student participation was observational
and peripheral,
suggesting the safe-house design requires further
development for nurturing an active democratic contact zone.
Keywords: Distance learning, social presence, democratic contact zone, online student mentor, pedagogical safe-house, lurking, peripheral participation
Introduction
21st
century online educators need to situate students in prime and authentic
learning environments (Herrington, 2003), and one feature of a prime learning
environment is the social presence of the instructor and class members. Research
shows that social presence in computerized instruction—the presence of
humans—increases student satisfaction with courses as well as instructors’
satisfaction (Richardson & Swan, 2003; Tu & McIsaac, 2002). Prime
social presence classes are often choreographed with a variety of social
presence opportunities, allowing for students’ individual preferences (Reeves
& Nass, 1996): frequent and timely
posting; online office hours in synchronous chat; embedded instructor audio
files or other speakers; human avatars with synthesized voices (SitePal.com);
and constructivist, collaborative work with peers in asynchronous or
synchronous discussion.
High social presence
humanizes online education and democratizes power relations (Foucault, 1979,
1980; Zembylas, 2007). Establishing a “democratic contact zone” (Brookfield, 2001,
p.206; Pratt, 1991) in online instruction entails creating and maintaining a
course climate where students feel safe and empowered to engage in public
discourse not only about course material and assignments but also emotional
components of their online experiences, such as
frustration from unreliability of the Internet; stress from juggling
school, work and personal life; apprehension and anxiety about class
assignments; “public” exposure of course work; embarrassment on inability to
perform tasks or perform tasks at a brisk pace; social isolation; and
complaints about the course instructor (O’Regan, 2003.) Because affective domains of knowledge are
recognized and valued in a democratic contact zone, there is tacit
encouragement for critical discourse, along with associational qualities for
democracy, which, in turn, nurture autonomous and thoughtful class activity
(Zembylas, 2007).
In online
instruction, high social presence and a democratic contact zone are worthy
goals, but they are also utopian intentions that easily fall short, leaving
typical dystopian outcomes (Miller, 2001), such as unsuccessful course
completion rates; student dissatisfaction with the online learning; and the
prevalence of the online silent or passive majority, whose lack of interaction,
intellectually and affectively, inhibits their active, constructivist learning.
Dystopian outcomes, especially in regard to student passivity, have been
typical for Colette Wanless-Sobel, an online instructor and one of this essay’s
authors. In an effort to maximize social
presence and a democratic contact zone, the over-riding question for
Wanless-Sobel has been, “What is hindering social presence and a
more democratic, participatory culture in online instruction?”
After
reflection, Wanless-Sobel hypothesized that, power relations in online classrooms are
still residually hierarchical and, therefore, inhibit social presence and
democratic activity, especially in terms of majority interaction. In other words,
although Wanless-Sobel casts herself as learning facilitator and fellow
knowledge-builder, students, who still cast themselves as subordinates and who
rightfully question how much power students actually have vis a vis the instructor and the institution, may be wary of
affective and democratic pronouncements, especially since Wanless-Sobel is
still the person in charge--the individual who dispenses the grades. (This
issue is an important one for social presence and democratic space in online
instruction but is beyond the scope of this case study.) Furthermore, residual
hierarchy in online instruction is attributable to students’ (and the
instructor’s) years of built-up experience of how one is supposed to be and act in a class: knowledge resides
with only the instructor, who maintains the dominant discourse; course work and
activity is issued only from the instructor; students follow instructions;
students do not create their own knowledge base; and emotional and affective
expressions have strict rules of conduct (Zembylas, 2007).
Residual hierarchy,
whatever the cause, is antithetical to social presence and a democratic
zone. Perhaps, Wanless-Sobel surmised,
hierarchy or power needs to be disrupted, and a “safe house” zone needs to be
actively nurtured in order to increase the level of social presence and
“democracy.” The adoption of the term safe house for this purpose is to connote online space where students can
seek sanctuary and protection from academic authority—namely, the
instructor’s-- as well as an
infrastructure to safely experiment with “subversive,” non-hierarchical course
activity and new norms of classroom behaviour.
Based on this idea,
Wanless-Sobel devised a pedagogical design test to see in what ways the social
presence-democratic zone quotient would increase by adding two inter-related
components to the online classroom: an online student mentor as student
coach–advocate and a “safe house” discussion forum completely administered by a
student mentor, where students can post anonymous questions and concerns about
the course all term, and where the course instructor is limited to lurking (ATIS Telecon Glossary).The student mentor’s job would be to respond to
forum questions, using as a knowledge base her or his personal experience as a
student in online instruction, along with instructor consults. Wanless-Sobel conceived both mentor and
safe-house forum as pedagogical elements that would subvert not only the
instructor’s course dominance but also the teacher-student hierarchy, while simultaneously
encouraging peer to peer dialogic activity. Thus, rather than information
dispensed only from the instructor, students could opt to “informally” gather
information from a peer, the online mentor, in a student platform that provides
a supportive knowledge culture, with the instructor having no visible presence.
This short essay
provides a case study on the design and test-run of these social presence-democratic
zone components, bringing together lessons learned about how to
choreograph online engagement, and a point of departure for asking questions
about future instructional design directions.
Situating the Case: Institutional and Student Frameworks
The online mentor and
safe house discussion forums were tested in four online classes at Inver Hills
Community College (hereafter, “Inver Hills”), Inver Grove Heights, MN, an
eastern suburb of the Twin Cities, where one of the authors, Wanless-Sobel,
teaches part-time and the other author, Amanda Alcorn, is a student in law
enforcement. Innovation in online education is a priority at Inver Hills, whose
course management system is Desire 2Learn (hereafter, “D2L.”). Inver Hills supports faculty efforts in
online course development through grant money, technology support, lap top
computer allocation, continuing education (CTL, 2004), and, most recently, an
Online Student Mentoring Program, developed by Landon Pirius, Director of
Enrolment and Online Services at Inver Hills, who used Walden University’s Online Concierge
Program and Lake Superior College’s Online Student Mentor Program as models
while also allowing more flexibility in terms of how instructors can make use
of the mentors in their classes (Pirius, 2007).The rationale for the Online
Student Mentor Program
is the online mentor is a D2L veteran and can share her or his experiences with
current D2L students, making practical suggestions as the class proceeds and
offering coaching support, although the online student mentor can also be used
to assist with the heavier clerical and record-keeping workload accompanying
online instruction (Cavanaugh, 2005; Doube, 2000).
Online Mentors at
IHCC are recruited from the student body, which ranges in
age from sixteen on up and comprises a diverse demographic: high school students; two-year college students;
paraprofessional students; four year degree candidates; displaced workers; and
life-long learners. African Americans, Asian Americans, and Mexican
Americans often comprise the student body as do recent émigrés from various
countries in Africa,
In
spring 2007, Wanless-Sobel had four D2L classes she planned on integrating a
student mentor and safe-house:” Research Writing in the Disciplines,” two
sections; “Creative Problem Solving”; and “Human Sexuality.” Alcorn, a former
“Research Writing in the Disciplines” student, approached her about working as
a student mentor, and Wanless-Sobel readily agreed, as Alcorn had previously
demonstrated intellectual ability, a personable online presence, and a high
comfort level with computers and online instruction. Until her work with
Alcorn, Wanless-Sobel had no previous online student mentor experience,
although she had worked with teaching assistants in classes she had taught at
the
The
venture was new for both Alcorn and Wanless-Sobel, but the goals were certain: increased social
presence and scaffolding for a democratic contact zone. The
questions were, “How to make best use of Alcorn as the student
mentor?” and “How should the safe-zone be constructed in the online
environment?”
The Student Mentor’s Autogenesis and the Creation of Central Perk
Out of habit, Wanless-Sobel, as the educational “expert” and
“professional,” solely set about answering these questions, bypassing Alcorn,
the student mentor. When Wanless-Sobel realized she was falling into old
teacher-student behaviour patterns of the analogue educational world,
however, she paused and reflected: In a
digital and democratic educational environment, her role, as an educator, is not to
oversee public access to knowledge; no longer does she act as gatekeeper to
what is produced, what is shown and how it is interpreted. In a digital world of infinitely replicable and
malleable
content, this behaviour pattern no longer applies. The full
implications of digital environments for educators, educational institutions
and students are not yet clear, although private, public and third sector
innovations, from Amazon to Wikipedia to Second Life and MySpace,
are revolutionizing the digital landscape and suggesting possible directions.
The conclusion from
these ruminations is that, perhaps Wanless-Sobel is not the person who should
solely direct the creation of the online safe space or the role of the online
student mentor. If she truly wishes to
disrupt hierarchical relationships in the online classroom and maximize social
presence, then her own behaviour and role in the online environment must change
and evolve, and what better way to begin than by allowing Alcorn, the online
mentor, autonomy and independence, both in her mentoring job and in the online
safe-houses she would be operating, although Wanless-Sobel could oversee all
activity behind the scene. In other
words, Wanless-Sobel needed to give up control and empower Alcorn by letting
her characterize her mentoring role to some extent and also design the online
safe-house.
Alcorn and Wanless-Sobel conferenced in-person and online for two weeks,
discussing and negotiating Alcorn’s mentoring responsibilities and how she
should conduct the safe-house. The actual design of the safe house was
uncertain for a time, although both women agreed that Wanless-Sobel would
create a D2L DISCUSSION forum in each of the classes for Alcorn, placing each
forum as the top forum in each class DISCUSSION, increasing its visibility and
accessibility for students. Alcorn would then conceptualize the safe-house
design and Wanless-Sobel would then enter the design into D2L.
The design Alcorn came up with for the safe-house was ingenious. Her idea was to integrate or “mash” the
highly popular Generations X / Y television series Friends, which is syndicated world-wide (http://www.answers.com/topic/friends-1/),
with the D2L Discussion forum, making use of the series’ coffee shop, Central
Perk, as an auto-poetic expression of friendliness and connectedness, a student
space that acknowledges affective and pleasurable dimensions of the class. Accordingly, Alcorn christened her safe-house
forum Central Perk and even included
a colourful poster image of Central Perk (http://www.allposters.com/-sp/-Posters_i855117_.htm)
from Friends to greet students and
set the tone. Essentially, Alcorn
conceptualized Central Perk as a student forum with a heart beat, possessing a
tangible positive social energy. She, Alcorn, would be Central Perk’s host and
manager. Here is the text Alcorn created to accompany the poster:
Central Perk, a forum in
DISCUSSION, is a place for students to ask Amanda (Mandy) Alcorn, "the
online mentor," questions about the class, assignments, or anything that
they feel like talking about within reason. The course instructor, Colette,
does not participate in Central Perk. So, if you have a question, ask. Who
knows, by posting a question in here, you (the student) may also be helping out
a fellow student. :) Posts are set for anonymous. Central Perk and its
transcripts will be available all term.
The
student mentor and the online safe-house were ready to launch. At the beginning
of spring term 2007, Wanless-Sobel advertised Central Perk on each class’s
HOMEPAGE and also posted class E-mails, informing students about this new class
accommodation. Alcorn also posted class
E-mails, introducing herself and providing a brief biographical statement.
Central Perk and Alcorn were in operation.
Central Perk in Action
Quantitatively, in
terms of posting and without attempting to account for lurker activity, which
was high (see discussion below), Central Perk participation was active in each
of the four online classes through the first eight weeks of the semester, with
activity falling off around the twelfth week of the term and then with occasional
use until the end of the term. Cumulatively, student posts to Central Perk were
four to five times higher than when
Wanless-Sobel had maintained anonymous question and answer forums in previous
online classes. Although almost all the
posts in Central Perk were anonymous, frequency of student posts indicates the
forum was perceived as a “safe zone,” where students would ask questions or network for information more comfortably
than in an instructor-run question and answer forum.
Central Perk participants’
posts centred around five types of information and communication: factual and
clarification information (e.g. When is
this assignment due); cross-functional human skills, especially regarding
technology (“How do you post a PDF file?”);
access to others’ online class experiences (”Why does this class have so much collaborative learning?”); moral
support and thanks (“Hang in there.”);
and mild flaming and criticism about class peers, the instructor, and
collaborative class work (“I haven’t
heard from any of my team members. It’s been two weeks. What should I do?”). Although most posts were addressed to the
student mentor, who responded to questions within twenty-four hours (usually
twelve), Central Perk also operated as an “open administration” forum,
especially as the term progressed, where students responded to posted queries,
too, if they thought they knew the answers.
One example:
Q. Am I to understand correctly that
for this week, we are to select a job ad and submit it in dropbox and also write
a rhetorical essay? (Due on the 12th @ midnight) The actual writing of the
resume comes later, right? Or, are we supposed to post a resume now for
team discussion?
A. No
resume. The work this week is foundation work for the resume, which will
be required in an upcoming assignment.
Central Perk’s Social Presence Efficacy
Tentatively, going by
post frequency, Central Perk and the student mentor had social presence
benefits for all four courses. In an
effort to qualitatively measure the benefits of Central Perk and the mentor,
Alcorn and Wanless-Sobel created an online survey, using Survey Monkey.com’s
free survey utility (http://surveymonkey.com/),
and administered the survey to all four classes at the end of the
semester. Along with collecting some
demographic information, such as age and gender, the survey asked students to
gauge Central Perk for nineteen social presence indicators, such as impersonal or personal, insensitive or sensitive, helpful or unhelpful.
Out of the four
classes with a total of ninety-seven students, forty students formally
responded to the survey; seven responded informally to Wanless-Sobel in
personal E-mails; ten responded to Wanless-Sobel in course feedback at the end
of the term; and forty students chose not to respond in any way. Most
respondents found Central Perk a positive feature and appreciated its safety
zone features. Thirty-two survey respondents, however, commented that they,
themselves, did not post questions to Central Perk, but, nonetheless,
frequently checked in to see what questions had been posted and found Central
Perk and the online mentor helpful. Four lurkers said they
wanted to post, but certain circumstances deterred them. These included lack of time for posting;
computer access issues, making posting difficult; and computer literacy. Finally, three
respondents said they did not find Central Perk or the mentor useful and
stopped checking in to the forum early in the term, with reasons such as
repetitious, inane questions; perceived mentor abrasiveness; and resistance to
subverted class authority; meaning: some students preferred to acquire
information only from the instructor.
Alcorn and
Wanless-Sobel had not realized peripheral participation was as high as it was
until reading the survey results. As it
turns out, the high incidence of lurking poses questions about Central Perk’s
social presence and how well it scaffolds a democratic contact zone for
students. These issues are discussed below.
Lurkers and Peripheral Participation
Internet studies show that, much
of the time people spend on any forum or list is with reading, not posting, and
that the primary motivation for participating or frequently logging in to the
forum-list is learning by reading the question / answer exchanges (Nonnecke & Preece,
2000; Lakhani &
von Hippel, 2003). What this means is that, posting is performed by a small
percentage of active forum / list participants, with a majority of people
participating peripherally as lurkers. As it turns out, the same holds true
with the Central Perk forums, where a small percentage of students in each
class of twenty-five students, participated as active and frequent posters and
a majority participated as observers or peripheral participants.
Not all lurking in Central Perk
was voluntary as mentioned above. In the survey, some lurkers said they wanted to
post, but certain circumstances
deterred them, some of which Wanless-Sobel can not easily influence. These
include lack of time for posting; computer access issues; computer skills
literacy, such as how to cut and paste text into a Discussion post; and
intimidation by hyperactive interactive class peers, who expected immediate
responses to posts.
Since lurking was so high in
Central Perk, what are the implications for social presence and the democratic
zone as well as online course design?
Although the active Central Perk participants likely had a stronger
sense of community and personal satisfaction than observers (Preece, Nonnecke, and Andrews,
2004), peripheral participation also entailed social presence, as evidenced by
the survey responses. Central Perk lurkers offered a variety of reasons for
experiencing social presence, ranging from satisfaction in receiving class
“mail”; comfort in knowing other students were experiencing difficulties with
an assignment; and ability to ask a peer questions about the course instructor
and her expectations.
High lurking activity with
Central Perk, then, does not necessarily negate its social presence efficacy.
At the same time, in terms of the democratic zone efficacy, especially in
regard to scaffolding a high level of interactivity, the silent student
majority is an issue for constructivist and collaborative pedagogy, which is
based on students functioning as active, individual learners. The high lurker
activity indicates many students are still passive learners, perceiving
knowledge acquisition and even interrogative posting as activity for only
certain individuals. In this regard, Central Perk is not successful, and
Wanless-Sobel’s future design efforts now need to focus on revising Central
Perk to provide optimal conditions for an active learning democratic zone.
Design Challenges and Opportunities
In retrospect, after a one
semester trial period with Central Perk, Wanless-Sobel sees a number of design
challenges and opportunities for future classes. One idea emerging from this case study is
that, her conception of a democratic contact zone in Central Perk is incorrectly
based on the notion of an online class as a “homogenous collective” (Foth,
2006), affectively, socially, and intellectually. As a result, Central Perk
does not provide students with a choice for peer-to peer networking
opportunities (Foth, 2006) or individual empowerment. In other words, Central Perk, though somewhat
successful in nurturing social presence, is interactively
too one-dimensional and parochial; it does not offer students the choice of
what to do, how to do it, whom to “socialize” with, and whom to gather
information from. As an online intellectual and affective space, it cannot
fulfill the full range of students’ needs for democratic empowerment (Satchell,
2003).
Additionally, although the
goal of Central Perk is to subvert classroom hierarchy, Wanless-Sobel’s use of
the discussion board to achieve this goal may need to be reviewed and
re-evaluated. For one thing, Central
Perk’s current design does not guarantee there still are not asymmetrical power
relations in the classroom (Miller, 2001). For another, its discussion board
design may ironically foster hierarchical relationships, which, in turn,
encourage lurking (Foth, 2006). Marcus
Foth, whose scholarship addresses computer-supported cooperative work in RL
(real life) inner-city neighbourhoods, addresses the inherent hierarchy of
discussion board architecture, and says:
A discussion board that affords collective many-to-many broadcast
style interaction does not support true social networks that are personal,
tacit, intricate, and peer-to-peer. Social networks possess tribal and swarming
qualities which [prove] to be incompatible with the mostly hierarchical nature
of tools that are designed to support dispersed online communities. (Foth,
2006)
Wanless-Sobel’s original
concept of a democratic contact zone had not taken into account either online
tribal or swarming behavior, meaning polarization of interests, personalities
and learning styles. Swarming and
tribalism are not necessarily positive features (Kelly, 1995) in an online
environment, where the goal is for more of a self-organized knowledge culture,
but they are often entailed in active networking and relevant to individual and
emotional control in the online environment (Zembylas, 2007) and need to be
recognized and addressed in course design. Foth’s ideas for digital
interactivity of inner-city neighborhood communities suggests that online
instructional design for a democratic zone requires features that animate
students’ interaction (Foth,2006), while acknowledging some degree of lurking
will be inevitable.
Central Perk is still a
useful online feature, but its present design needs to be revised and
expanded. Its discussion board format is
too conventional, encouraging students to resort to old classroom behaviour
patterns, such as looking for answers from one person or from the dominant
online personalities and hiding out in the “crowd.” Instead, what students need
is a variety of democratic zone choices, especially zones designed to
accommodate different learning styles and personalities. The next logical step
in Wanless-Sobel’s design experiment is to enlist other media and utilities as
democratic zone designations — designations that will allow more tribal and
swarming activity and provide individual empowerment: cell phones, with voice
and text messaging (Satchell, 2003) and micro-blogging, using a utility like
Twitter (http://twitter.com/) ; I-PODs; audio
editors, such as Audacity (http://audacity.sourceforge.net/);
file sharing facilities; mailing lists; digital video; and perhaps a virtual
environment, such as Second Life (http://secondlife.com/), where students can take advantage of visual and
spatial components to interact. Individualizing the
democratic contact zone will provide a stronger scaffold for class interaction
and active learning in the online environment (Gilchrist, 2000).
Conclusion
In sum, the present case study suggests a student mentor and a safe
house discussion forum do increase social presence in online instruction for
students, although a considerable percentage takes advantage of these course
amenities peripherally as lurkers. Furthermore, some students are unreceptive
to these course accommodations. Central Perk is less successful for
establishing a student majority democratic contact zone. To actively engage
more students, especially lurkers and non-participants, a more complex and
nuanced design of the online mentor and safe house forum is needed, including
the creation of other safe-house zones using a variety of media and utilities
to meet students’ social presence and democratic contact zone needs.
References
ATIS Telecon Glossary T1.523. (2001). Definition
of Lurker.
Accessed on 08/05/07 at http://www.atis.org/tg2k/_lurker.html
Audacity.Net.
Accessed on 24/05/07 at http://audacity.sourceforge.net/
Burns, Philip, J. (1999).
Supporting deliberative democracy: Pedagogical arts of the contact zone of the
electronic sphere. Rhetoric Review, 19
(1), 128-46. Accessed on 25/04/07 at http://links.jstor.org/sici?sici=0735
Cavanaugh,
Joseph. (2005). Teaching online—A time comparison. Online
Journal of Distance Learning Administration, 8 (1).
Accessed
on 07/04/07 at http://distance.westga.edu/~distance/ojdla/spring81/cavanaugh81.pdf
Center
for Teaching and Learning (CTL / MNSCU). (2004). CTL grant guidelines, 2004-
05. http://www.ctl.mnscu.edu/programs/grants/guidelines-05.htm
“Central
Perk.” Entry in Wikipedia. Accessed on 25/04/07 at http://en.wikipedia.org/wiki/Central_Perk
Central
Perk Poster. Accessed on 11/01/07 at http://www.allposters.com/-sp/-Posters_i855117_.htm
Doube, W. (2000). Distance
teaching workloads. Association for Computing Machinery, Special Interest
Group on Computer Science Education Bulletin, 32 (1), 347-351.
Foucault, M. (1979). Discipline
& punish: The birth of the prison. Alan Sheridan trans.
Foucault, M. (1980).
Power/knowledge: Selected interviews and other writings 1972-1977. Colin Gordon,
et al., trans.
Foth, Marcus. (2006). Facilitating social networking in inner city
neighborhoods. Computer, 39 (9),
44-50. Accessed on 13/05/07 at http://eprints.qut.edu.au/archive/00004750/01/4750.pdf
“Friends Television Series.” (1994-2004). Entry in Answers.com
Accessed on 20/04/07 at http://www.answers.com/topic/friends-1
Gilchrist,
Alison. (2000). The well-connected community: networking to the "edge
of chaos." Community Development
Journal 35(3), 264-75
Gunawardena, C.N. (1995). Social presence theory and implications for interaction
and collaborative learning in computer conferences. International Journal of
Educational
Telecommunications,
1(2/3),
147-166.
Gunawardena, C.N. and Zittle, F.J. (1997). Social presence as a
predictor of satisfaction within a computer-mediated conferencing environment. The American
Journal of Distance Education, 11(3), 8-26.
Herrington, J., Oliver,
R., & Reeves, T.C. (2003). Patterns of engagement in authentic online learning
environments. Australian Journal of Educational Technology. 19 (1),
59-71.
Kelly, Kevin. (1995). Out of control: The new biology of machines,
social systems, and the economic world.
Accessed on 25/04/07 at http://www.kk.org/outofcontrol/ch2-f.html
Lakhani, K. and
von Hippel, E. (2003). How open source software works:
“free” user-to-user assistance. Research Policy, 32, 923–943.
Accessed on 13/05/07 at http://opensource.mit.edu/papers/lakhanivonhippelusersupport.pdf
Miller, N. (2001). The
politics of access and communication: Using distance learning technologies. In
R. M. Cervero, & A. L. Wilson (Eds.), Power in Practice: Adult Education
and the Struggle for Knowledge and Power in Society (pp. 187-205).
Nonnecke, B., and J. Preece. (2000). Lurker demographics: Counting the silent.
CHI 2000,
O’Regan, Kerry. Emotion and e-learning. Journal of Asynchronous
Learning Networks, 7
(3), 78-92.
Pirius, Landon, Director
of Student Enrollment Services, Inver Hills Community College, Inver Grove
Heights, MN 55076. E-mail to Amanda Alcorn, 25 April 2007.
Pratt, Mary Louise.
(1991). Arts of the contact zone. Profession,
91, 33-40.
Preece, J., Nonnecke,
B. and Andrews, D.C. (2004). The top
five reasons for lurking: Improving community experiences for everyone,” Computers
in Human Behavior, 20 (2), 201-223.
Reeves, B. and Nass, C.
(1996). The media equation: How people treat computers, television, and new
media like real people and places.
Richardson, Jennifer C.
and Swan, Karen. (2003). Examining social presence in online courses in relation
to students’ perceived learning and satisfaction. Journal of Asynchronous
Learning Networks, 7 (1), 68-88.
Satchell, C. (2003). The
swarm: Facilitating fluidity and control in young people’s use of mobile phones.
Proc. OZCHI 2003: New Directions in Interaction, Information
Environments, Media and Technology, Information Environments
Program,
Second Life.Com. Accessed on 17/05/07 at http://secondlife.com/
Short, J., Williams, E., and Christie, B. (1976). The social psychology of
telecommunications.
SitePal.Com. Accessed on 22/04/07 at http://sitepal.com/
Suchman, L. A. (1987). Plans and situated action: The problem of
human-machine communication: Cambridge University Press,
Survey Monkey.Com. Accessed on 20/04/07 at
http://surveymonkey.com/
Swan, K., Shea, P., Frederickson, E., Pickett, A. Pelz, W., and Maher, G.
(2000).
Building knowledge building communities: Consistency, contact, and
communication in the virtual classroom. Journal of Educational Computing
Research, 23 (4),
389-413.
Tu, Chih-Hsiung and McIsaac, Marina.
(2002). The relationship of social presence and interaction
in online classes. American Journal of Distance Education, 16 (3), 131-50.
Twitter.Com. Accessed on 22/05/05 at http://twitter.com/
Zembylas,
Michalinos. (2007). A politics of passion in education: The Foucauldian legacy.
Educational Philosophy and Theory, 39
(2),135–149.
|
Colette Wanless-Sobel, Ph.D. Department
of English and Interdisciplinary Studies colette.wanlesssobel@GMAIL.COM Amanda Alcorn Student
in Law Enforcement |
Learners’ profiling by data driven approaches
Data driven approaches for learners’ profiling
In recent
years, E-Learning became an opportunity for re-thinking learning itself in
terms of personalized learning. Personalization in this context is meant
as the capacity of the system to adapt to the individual differences and the
needs of learners: this implies the capacity of representing these differences
and needs and their evolution. “Personalized” learning is said both regarding
individuals and regarding groups.
Differently
from traditional educational settings, E-Learning allows tracking the users’
behaviour in the Electronic Learning Environments: these data are fully
authentic. Therefore, to improve personalization, data driven approaches (e.g.
[1]) that are accurate, precise and flexible, should be tested for analyzing
and modelling online interactions. The approaches have to deal with the
complexity of learning, taking into account:
- the evolution of the behaviour
during time;
- the pedagogical, cognitive and
metacognitive features of the learning process;
- the variables affecting
learning (social and situated learning, impact of each artefact on the
learning process);
- the uncertainty of the domain,
and the unexpected patterns that can arise;
- the individuality of each
profile (or of each group in case of collaborative work).
In past
years, an increasing attention has been dedicated to data driven approaches for
analyzing educational data within different Research Communities (eg Data
Mining, User Modelling and ITSs). Today, the field is growing and maturing
[11], but there are still many open problems.
Experimental settings, mathematical settings and results
Our dataset
comes from the V Framework European WINDS Project, and is made by sessions
data. The attention was focused on establishing if data driven approaches – in
particular the ones attested in the literature for modelling complex processes,
such as biological, medical or economic processes - can be applied to learning
to attain a useful and meaningful characterization of pedagogical, cognitive
and metacognitive features.
From the
introduction of data driven approches we expected improvements in accuracy and
precision in modelling complex domains, and smaller cost in terms of human and
material resources. From a mathematical viewpoint, sessions data are
heterogeneous, both for length and for the number belonging to each individual;
moreover, being they collected outside any experimental design settings,
classical statistical frameworks premises do not hold; eventually, appropriate
measures have to be selected in order to avoid biased results.
Data are
the absolute and joint frequencies of usage of different kinds of objects
provided by the WINDS ALE, including traditional objects (units, paragraphs),
interactive objects (cases, exercises), collaborative objects (annotations,
forums), non sequential navigation tools (glossaries and maps). Data are
expressed on a numeral scale.
Two views
have been considered: the first, called “profiles view”, focuses on the average
preferences of each learner; the second, called “sessions view” focuses on the evolution
and changes of the profiles during time.
We have
applied:
- multivariate methods, such as
Principal Component Analysis (PCA, [3]) and Multiple Correspondence
Analysis (MCA, [2, 4]), together with appropriate proximity measures
and/or additional data (test scores) where available; MCA has been also
compared with classical methods (p values and chi-square test);
- “frequent episodes discovery”
algorithms (FED, [5]) in sessions treated as sequence data;
- time series methods, such as
autocorrelation and cross correlation, in order to get synthetic indexes
of the evolution of the profiles along time.
Results
shows the effectiveness of these methods. In fact, PCA detects unobservable
dimensions coherent with important features of technology-enhanced learning
(sequential, non sequential, multimodal, collaborative strategies) [6]. MCA
detects different strategies according to the learning outcomes. Moreover,
these methods are more sensitive than classical statistical ones in detection
of differences even when they are not statistically significant [7]. FED-based
sessions analysis and time series methods show two different ways of non
sequential navigation patterns whose difference is statistically significant
[8, 9].
Figure 1 – The data driven approaches
Conclusions and further work
From our
experience, data driven approaches are effective and promising for detecting
learning process patterns; moreover, the unobservable dimensions revealed are
coherent with some important features of learning, and helpful in individual
differences detection. In order to improve precision and accuracy, it is
remarkable to consider the need for:
- multi-source heterogeneous data
integration inside the profiles;
- multi-models cross-validations
to obtain profiles as coherent as possible;
- efficient proximity measures
for heterogeneous data;
- techniques for high dimensional
data visualization.
Improvement
can be foreseen in: intelligent patterns discovery, monitoring systems,
evaluation and assessment tasks, adaptivity/personalization of responsive
environments, supporting collaborative learning tasks, and research; however,
limitations should be taken into account as well (eg the easiness of use and of
understanding by unskilled people in real-life tasks, and the impact on
educational contexts).
More can be
found at http://www.del.univpm.it:8080/del/Members/silvia/pubblicazioni_html
References
Cherkassky
& V. Mulier, F. (1998). Learning from data, Wiley.
Greenacre,
M. (1984). Theory and Applications of Correspondence Analysis, Academic
Press.
Jolliffe,
T. I. (1986). Principal Component Analysis, Springer.
Lebart, L.,
Morineau, A.& Warwick, K. M. (1984). Multivariate Descriptive Statistical Analysis, Wiley.
Mannila, H.,
Toivonen, H. & Verkamo, A. I. (1997). “Discovery of Frequent Episodes in Event
Sequences”, Data Mining and Knowledge Discovery, 1:259-289.
Viola, S.
R., Giretti, A.& Leo, T.(2005). “Discovering learning process patterns by
multivariate analysis of usage frequencies data in e-learning courses”. ICL
2005 Proceedings, 1B-4.
Viola, S.
R., Giretti, A.& Leo, T. (2006). “Exploring attitudes of learners with
respect to different learning strategies and performances using statistical
methods”. International Journal of Emerging Technologies in Learning,
1(1), June 2006, ISSN 1863-0383, http://www.online-journals.org/index.php/i-jet
(retrieved March 22, 2007)
Viola, S. R.,
Giretti, A.& Leo, T. (2006). “Differences in meaningful learning strategies
of navigation: an empirical model.” IEEE ICALT 2006 Proceedings,
pp.441-445.
Viola, S.
R., Giretti, A.& Leo, T. (2006): “Non sequential pattern analysis along
time for “meaningful learning” strategies of navigation detection”. ICL 2006
Proceedings, 6A–3.
Viola, S.
R.(2006): Learning Process Patterns Extraction and Characterization using
mathematical and statistical methods, Ph. D. Thesis, Universita’ Politecnica
delle
http://www.del.univpm.it:8080/del/Members/silvia/tesiViola.PDF
http://www.educationaldatamining.org/
(retrieved March 22, 2007)
|
Silvia
Rita Viola, PhD Universita’ Politecnica delle DIIGA, Dip. Ing. Informatica, Gestionale e
dell’Automazione “Maurizio Panti” Via delle Brecce Bianche – 60100 Alberto
Giretti Universita’ Politecnica delle DACS – Dipartimento di Architettura, Costruzioni e Strutture Via delle Brecce Bianche – 60100 Tommaso
Leo Universita’ Politecnica delle DIIGA – Dip. Di Ingegneria Informatica, Gestionale e
dell’Automazione “Maurizio Panti” Via delle Brecce Bianche – 60100 |
Introduction
ICT technologies should
be useful tools for the acquisition and the increase of knowledge and skills in
the field of medicine. The application of ICT in the health sector could
provide a better quality of life to the citizens and an easier job environment
for physicians and other health care workers. ICT can be used wherever it has a
clear benefit, such as reaching remote populations, offering the tools for
building a national health network and providing continuous training for
doctors and hospital attendants, sharing knowledge between institutions and
countries. In this latter field ICT services, and in particular E-Learning,
could allow an effective improvement of training, management and organization
of medical staff. So the introduction of e-health (health services and
information delivered or enhanced through the Internet and related
technologies) represents the promise of information and communication
technologies to improve health and the health care system. The Uro project is
set in this scenario. In fact it aims to prepare a “new professional”, named
Content Therapist, in the area of Urinary Incontinence by integrating specific
preparation in the field of urology with disciplines principally concerned with
gynecology, obstetrics, physical and rehabilitative medicine, neurology,
coloproctology, geriatrics, paediatrics. It is foreseen the extensive use of
New Educational Technologies and of E-Learning approach. The main activities
are concentrated on the preparation of a curriculum for appropriate studies for
the training of human resources, from both Asian and European countries and
course material (content definition, teaching tools, production/printing of
course material), their dissemination and the demonstration of the realized
training services. In particular the target groups are medical attendants of
Main aims of the Uro Project
ICT and its
applications are increasingly looked upon as the potential answer to the
requirement of a modern society, with demands for better healthcare, improvements
in medical outcomes, and maintenance of relatively high quality of life,
especially with the onset of chronic health conditions coming to the fore as a
key issue. Furthermore, implying a view of utilizing the technology as a tool
for readdressing the prevailing state of affairs, ICT tools and applications
are also seen as having a potential to support an enhanced access to health
information in general and indeed, to the health system itself, in particular
[1]. Today, Continuous
Medical Education (CME) becomes a crucial factor, because the life of knowledge
and human skills in the field of medicine is shorter than ever. E-learning
comes with solutions and methods, which can be very helpful in supporting
doctors with access to the up-to-date medical knowledge and achievements. It
allows creation of interactive model of learning, which stimulates knowledge
acquisition. Another advantage is that e-learning provides flexibility in both
time and location, while accessing medical curriculum presented online [2].
There is a possibility of collaboration between teachers and students from
different universities and countries, which allows exchange of knowledge and
experiences. In this field the URO project aims to give the main contribution.
In fact it aims to make available a new curriculum,
supported by ICT technologies, for appropriate studies for the training of
human resources, from both Asian and European countries. The overall objective
of the URO project is the preparation of a “new professional”, named
“Continence Therapist”, who works at home, in the community and in the hospital
and is the fulcrum of the interaction between the needs of the patient and the
necessary social and specialist medical answers. We have to underline that the
“Continence Therapist” highlights the need for a multi-dimensional and, above
all, cross-discipline professional preparation able to offer integrated and
sequential management in cooperation with specialists in the sector and/or centres
in the world, for continence and the pelvic floor. In this scenario a central
role is played by the ICT and in particular by the e-learning methods. Any
e-learning platform and provision should give opportunities to improve the
quality and the variety of teaching and learning not otherwise achieved through
traditional methods. The particular requirement is the capacity of the
e-learning provision to offer the additional services and tools for the needs
of the target groups of the Uro project. We have two different kinds of target
groups: ultimate target groups and direct target groups. The Ultimate Target
Groups will be targeted within the participating countries. The indirect target
groups are: Registered Nurses, Licensed Practical Nurses, Nurse Assessment Coordinators,
Midwives and Physical Therapists, Occupational Therapists. Precise numbers of
the above categories are impossible to quantify. In
· Students and their relationships are
to be in the centre of attention
· Enhancing the learning scenario by
allowing communication richness
· Focus on the social environment
· Acknowledging the individuality of
learning styles
Definition of Learning and training needs of the final users
The first activity of the project had the
objective of showing the needs of the final users in terms of the
characteristics of the learning service, interface and contents. The aim was to
recognise in a clear manner the learning needs of each target group. In order
to achieve this aim a series of interviews of members of the direct target
groups was performed to highlight the needs of users with regard to the
learning system to be implemented. We adopted a validated questionnaire, ILS –
Index of Learning Styles questionnaire, for evaluation of preferred learning
styles. ILS is an instrument used to assess preferences in four dimensions
(active/reflective, sensing/intuitive, visual/verbal and sequential/global) of
a learning style model and was designed by Richard M. Felder and Linda K.
Silverman. We used the ILS questionnaire on two direct target groups:
Slovenian Target group (N = 205)
The behaviour
of the average Slovenian Target group:
[ACT/REF] =
has a moderate preference on the active direction. It tends to discuss the
information or explains it to others rather than to reflect on it.
[SNS/INT] =
has a moderate preference on the sensing direction. It prefers to learn facts,
to be patient with details rather than grasping new concepts. It is more
practical than theoretical.
[VIS/VRB]
=has a moderate preference on the visual direction. It remembers best what is
seen - pictures, diagrams, films - rather than written or spoken explanations.
[SEQ/GLO] =
has a moderate preference on the sequential direction. It tends to gain
understanding in linear steps, with each step following logically from the
previous one.
Indian Target group (N = 61)
The behaviour
of the average Indian Target group:
[ACT/REF] =
has a moderate preference on the active direction. It tends to discuss the
information or explains it to others rather than to reflect on it.
[SNS/INT] =
has a moderate preference on the sensing direction. It prefers to learn facts, to be patient with
details rather than grasping new concepts. It is more practical than
theoretical.
[VIS/VRB]
=has a moderate preference on the visual direction. It remembers best what is
seen - pictures, diagrams, films - rather than written or spoken explanations.
[SEQ/GLO] =
has a moderate preference on the sequential direction. It tends to gain
understanding in linear steps, with each step following logically from the
previous one.
|
|
|
|
Description
of Slovenian Target Group |
Description
of Indian Target Group |
Conclusions
The study
content of the Continence Therapist study programme will be hence designed and
structured to meet the recognised learning style preferences of the three
Target groups. Given that the learning styles of Slovenian and Indian groups
are strikingly similar, no major regional differences in form and structure of
the study programme has to be taken in consideration for Europe and
Bibliography
The Euser Project.
eUser – Workpackage 1: Conceptual and Analytical Framework. D1.1: eUSER
Conceptual and Analytical Framework (Part A and Part C). European Commission, 6th
Framework Program Contract number: IST-2002-507180
P. Rudowski, R.
Rudowski (2006). E-Learning In Medicine – Myth Or Reality, in Malina Jordanova
and Frank Lievens Proceedings of Med-e-Tel 2006, 5-7 April 2006,
Luxembourg
R. M. Felder, L. K.
Silverman (1988). Learning and
Teaching Styles in Engineering Education, Engineering Education, 78(7),
674-681 (1988)
|
Francesco Colace DIIIE-Università degli Studi di Salerno Via Ponte Don Melillo, 1 84084 Fisciano (Sa) Massimo De Santo DIIIE-Università degli Studi di Salerno Via Ponte Don Melillo, 1 84084 Fisciano (Sa) |
HotKey Coach: Training users to interact efficiently with software
applications
Introduction
Once people become
fluent with the basic workings of a software package, they turn their focus
towards productivity. To achieve productivity, a user must become efficient
with the execution of the most frequent tasks. While using a software
application, a user can save a considerable amount of time over the course of a
typical work session by substituting keyboard shortcuts for mouse actions.
However, no useful method exists for helping an intermediate software user
transition from relying on the mouse to using the keyboard. Consequently, many,
if not most, users settle for less efficient methods of interaction with the
application.
In our lab at
Background
As outlined by Anderson [1], the acquisition of a skill progresses through three stages: cognitive stage, associative stage and autonomous stage. Bederson [2] defines these three stages in terms of a graphical user interface (GUI) interaction as:
· (Novice) Cognitive Stage: Uses the GUI for interface exploration.
· (Intermediate) Associative Stage: Knows GUI elements, but still requires feedback.
· (Expert) Autonomous Stage: Requires no feedback from the interface.
When users execute a keyboard shortcut, they have minimal interaction with the interface. By reducing the feedback between user and user interface, while at the same time eliminating the need to transfer the hand between the keyboard and the mouse, users reach the autonomous stage and achieve maximum efficiency [3].
However, most software
users fall into the associative stage, with some forays into the autonomous
stage with the most repetitive tasks, such as save, copy and paste. Most users
fail to achieve significant progress at autonomous interaction with the
application because the learning required to advance using current methods is
too time consuming. For example, cheat sheet cards and GUI widget tool-tips
require extra time and effort to learn because they do not fit in with the
normal flow of the user's tasks. This limitation results in only the most
dedicated users developing a strong shortcut repertoire.
Our Approach
Our goal is to develop
a method that helps the user become more efficient with their everyday software
tasks. For example, if a user of Microsoft Word frequently switched between the
Normal View and the Page Layout View via the mouse, learning the keyboard
shortcuts for these options would, over the course of multiple operations,
produce significant time savings. The net effect of this learning would be to
replace user actions to move the hand from the keyboard, to the mouse, navigate
to the menu option, select it, with a simple keystroke operation. It also
reduces cognitive load.
We have devised a
system that teaches keyboard shortcuts within the current task as the user
proceeds with their work. When a user selects an operation from a menu item, a
window displays the keyboard shortcut for the operation. At this point, the
user has the option of performing the keyboard shortcut to reinforce the
learning, or they can ignore the tip by either continuing with their task, or
selecting a check box within the tip indicating that they no longer wish to be
coached for that specific operation in the future. See Figure 2 as an example.
Figure 2: Coach Tip for Paragraph, Format in Word
By providing the shortcut
for an operation directly after the operation is executed via the interface,
the user sees the shortcut while the operation is still fresh in memory, making
a direct link between the operation and the shortcut. Working within the
typical usage patterns of the software, we ensure that only the frequently used
commands are taught first. Allowing the user the ability to ignore the tip
keeps the opportunity to learn from interfering with the workflow.
Preliminary Results
Currently, our software
runs on the Mac OS X operating system. The HotKey Coach software is available
to a limited test base and early results have demonstrated direct examples of
learning. The data shows numerous instances where a user starts out solely
using the mouse for a specific operation, but over time transitions into
keyboard usage for common operations.
There is also evidence of transfer, where once a user learned the
keyboard shortcut for an operation, they used that shortcut for the same
operation in a different application.
Our data shows users
making the first step towards autonomous software interaction after just a
short amount of time using our coaching software. Because our method of
coaching during the normal workflow provides a more natural approach to
learning with minimal interruption, a user is more likely to spend the time
required to learn the shortcuts for their commonly used commands. Our next
steps are to expand the supported application features beyond menus to toolbar
and palette operations. We are also investigating methods for assigning
shortcuts for common operations that do not currently have a shortcut. Given
the current trend to move towards network-centered applications, we are also
working on applying our coaching method to web-based applications, such as
GMail, that now support direct keyboard interaction.
References
Bederson, B.B. (2004). Interfaces for staying in the flow. Vol. 5, ACM Press. 1-1
Lane, D.M., et
al. (2005). Hidden
Costs of Graphical User Interfaces: Failure to Make the Transition from Menus
and Icon Toolbars to Keyboard Shortcuts. International Journal of
Human-Computer Interaction. 16(2):
p. 133-144
|
Brian
Krisler Brandeis University Richard
Alterman Brandeis
University |
Evaluation of Training – A Critique of Kirkpatrick’s Evaluation Model
Evaluation of training means measuring the effectiveness of a training program. Evaluation helps in defining the learning outcomes more sharply, remove unnecessary training content, and ensure that the training method meets the training needs of the learners and consequently of the business.
This effectiveness is as much about short-term retention as about the long-term retention and application by learners. Today most organizations talk about measuring the effectiveness of the training programs. While most traditional institutions have mechanisms in place where they measure the effectiveness of the classroom training programs, most modern day training programs are still not evaluated objectively or accurately.
There are several approaches used to measure the effectiveness of training. One popular approach is Kirkpatrick's Four Levels of Evaluation. In this model, each successive evaluation level is built on information provided by the earlier lower level. Let’s look at a brief description of each of the four levels and the respective criticisms.
Level 1 Evaluations: Reactions
As the name suggests, evaluation at this level captures the immediate reaction of the participants about the training. It assesses learners’ satisfaction with the training. While this is not an accurate measure of effectiveness, it certainly can be a filter for bad training. Positive reactions may or may not result in good learning but negative reactions can certainly be roadblocks to learning.
Criticism: Research shows that reactions should be observed and not asked. Kirkpatrick’s model has not defined the mechanisms of capturing the reactions. The model depends on the participants to provide this information, which is not a measure of the effectiveness. At best, it can be an indicator.
Level 2 Evaluations: Learning
At this level, evaluation moves beyond learner satisfaction to assess the learning that has happened in terms of acquisition of skills, knowledge, or attitude. Common methods used to test this are pretest and post test assessments to determine the amount of learning that has occurred.
Criticisms: Pre-assessment and post-assessment can capture knowledge and comprehension level skills. Also this is a measure of how effective the training is in transferring this information to the learner’s short-term memory. It does not measure the effectiveness of training on the learner’s cognitive skill, which will ultimately determine whether the learning will be applied at the workplace or not. In short, this phase does not measure learning; it only measures “rememberability”.
The other point of criticism is that at no point is there a measurement against the specified learning goals and outcome. There is no mechanism to compare the % of fulfillment of the specified learning goals and outcome.
Level 3 Evaluations: Transfer
This level is a critical measure of the amount of transfer of learning or amount of application of learning at the workplace. Evaluating at this level attempts to answer the question - Are the newly acquired skills, knowledge, or attitude being used in the everyday environment of the learner? According to Kirkpatrick, measuring at this level is difficult as it is often impossible to predict when the change in behavior will occur, and thus requires important decisions in terms of when to evaluate, how often to evaluate, and how to evaluate.
Criticisms: This is the most critical level of evaluation because it helps measure the most primary purpose of training. However, Kirkpatrick’s model does not specify the mechanisms of capturing and analyzing the information obtained in this level. While it is impossible to predict a change in behavior post training; information about learner behavior should have been captured during learner analysis (pre-training) so that the training is designed keeping in mind the said behavior. Once, this variable is factored in the evaluation metrics, evaluating the change in behavior or acquisition or transfer of skill will be much more effective and accurate.
Kirkpatrick’s model also ignores the other dependencies at this level of evaluation, such as “who will measure this information, what are the interventions needed, when to intervene, and what happens if intervention is delayed or absent…” The actual measurement of training effectiveness will depend on all these dependencies that are not defined in the Kirkpatrick’s model.
Level 4 Evaluations: Results
At this level, evaluation is done to measure the success of the program in terms that managers and executives can understand - increased production, improved quality, decreased costs, reduced frequency of accidents, increased sales, and even higher profits or return on investment. However, most organizations do not address this form of evaluation because they think determining results in financial terms is difficult to measure, and is hard to link directly with training.
Criticisms: This evaluation level is the most critical test for business and it’s the least explained in Kirkpatrick’s. The major criticism against Kirkpatrick is that its fourth level does not actually estimate the business impact of the training. It does not provide the outcome as ROI. While it makes an effort to measure the success of the program from the perspectives of managers and executives, it actually does not evaluate how the program relates to the measures of business success.
There is an interesting article Alternatives for Measuring Learning Success by Kaliym Islam that talks about how with the advent of the electronic delivery of training programs and the increased capital investment required to develop these programs, the Kirkpatrick model has become obsolete. Some interesting read about criticisms of Kirkpatrick’s Models:
- http://www.e-learningguru.com/articles/art5_1.htm
- http://www.sibm.edu/home/pdfs/research/srp/hr/SRP_Developing_a_quantitative_model_to_measure_training_effectiveness.pdf
- http://www.ltimagazine.com/ltimagazine/article/articleDetail.jsp?id=85206
The next article in this series will talk about an evaluation methodology evolved by Kern. This methodology not only overcomes some of Kirkpatrick’s deficiencies but also talks about a paradigm shift in the way eLearning should look at evaluation.
|
Geeta
Bose |
Introduction
A Content Management
System (CMS) is a computer software system used to assist users in the process
of content management. A CMS supports the organization, control, and
publication of a large body of documents and other multimedia content. CMS often facilitates the collaborative
creation of documents. Usually a CMS is a web content management system that is
a content management system with additional features to ease the tasks required
to publish web content to websites (Wikipedia).
CMSs are often used
for storing, controlling, versioning, and publishing industry-specific
documentation such as news articles, operators' manuals, technical manuals,
sales guides, and marketing brochures. This kind of environment is an essential
part in a web design course for postgraduate Journalism students. The problem
is that the majority of the available for downloading CMSs require installing
additional software in the host server (a database -usually MSQL, - PERL, or
PHP etc.) (CMS review). The installation and customization of such software is
not an easy task. Media Informatics Lab at the Department of Journalism and
Mass Communication in Aristotle University of Thessaloniki solved the previous
problem with the deployment of the Microsoft Homepage Starter Kit. It is an
ASP.NET 2.0 based Content Management System. No ASP knowledge is required. If
the host server is running Microsoft Windows 2003 Server Edition, the ASP.NET
2.0 can be installed automatically with the optional updates. The only thing
the administrator has to do is to activate the ASP.NET and add the read permission
for the ASPNET user in a specific
directory.
Description of the Homepage Starter Kit
First thing before a
user starts creating content is to set some global parameters and information
for the new website. Thus he is able to change some global parameters and even
change the complete layout of your website, even after all content has been
added. The user can customize the navigation and can create new pages. The
pages are listed in the order in which they appear in the navigation. Indented
items are subpages of the page above them and will be shown in a secondary
navigation. The CMS seems to supports many level of
navigation, although the user must not exceed three levels. After the third
level the CMS does not display the navigation levels. Each page can be moved within the navigation
hierarchy or can be made a subpage of another page.The CMS usually allows by
default everyone to visit the pages, but it also supports restricted area for
members.
Page and Navigation customization
The CMS provides a
simple yet powerful user management that can be used to create restricted
areas. There are 2 levels of rights available: Registered
Users: Pages that are not visible to everyone can be visited by registered
users only. There is only one level of registered users. Administrators: Among the registered users there can be one or more
administrators. Administrators have full access to the environment and can edit
the site, create and delete pages and mange user profiles.
Every new page is
empty and can be customized by adding 10 different modules. You can add as many
modules to your page as you like and arrange them in any order. Many modules
will have a rich text editor to create new content. This editor allows the
basic text formatting tasks and even table creation. If you are familiar with
HTML, you can toggle views between the design view and the HTML view.
HTML editor
Conclusion
The Microsoft Homepage
Starter Kit can be downloaded from Codeplex at http://www.codeplex.com/Wiki/View.aspx?ProjectName=MyWebPagesStarterKit.
You can unpack it directly on the server and start building web sites. If you
want to be more involved, using Visual Studio or the Visual Studio Web
Developer Express Edition, you can customize and extend the features of the
CMS. Our experience in using it in our lab courses has shown that the Homepage
Starter Kit is an extremely useful tool in teaching web design. All data are
stored in the App_Data folder. So you
can easily collect lab exercises by simply copying the files from the App_Data folder. Also by deleting all
files from the previous folder you can reset the CMS to its initial state,
ready to be used by another student.
References
Wikipedia the free encyclopedia,
Available at http://en.wikipedia.org/wiki/Content_management_system
(retrieved on 04/01/07).
CMS review available
at http://www.cmsreview.com/ (retrieved on 04/01/07).
The Microsoft Homepage
Starter Kit,
Available at http://www.codeplex.com/Wiki/View.aspx?ProjectName=MyWebPagesStarterKit
(retrieved on 04/01/07
|
Andreas Veglis Media Informatics Lab. Dept of Journalism & MC 54006 Thessaloniki GREECE |