\documentclass[]{article}
\usepackage{url}
\usepackage{graphicx}
\usepackage{verbatim, code, hyperlatex}

\textheight 8.6in \textwidth 6in \topmargin -0.25in \oddsidemargin
0.25in \evensidemargin 0.25in

\newcommand{\proposalTitle}{Computer-Supported Collaborative Learning %
in Support of SMET Undergraduate Retention:\\ %
A Practice-Oriented CSCL Research Agenda }
\newcommand{\mycomment}[1]{\textbf{\emph{#1}}}

\begin{document}

% Definition of title page:
\title{
    \proposalTitle
}
\author{
    PI: Mark Guzdial \\    % insert author(s) here
}
\date{May 2003}    % optional

\maketitle

\thispagestyle{empty}

\vspace*{.5in}

\noindent {\Large\bf Project Summary}

\vspace{.5em}

\noindent Collaborative learning in general and Computer-Supported
Collaborative Learning (CSCL) in particular can positively impact
student learning. Several studies have shown significant impacts
in SMET and other courses \cite{hoadley-linn,miyake,
coweb-english}. How it impacts student learning is still an open
matter. We know that dialog has an impact on student learning
\cite{roschelle,jeong}, but we know that that's not the only
mechanism engendered by collaboration which impacts learning, and
it may not even be a necessary one \cite{nodialog}.

One way that collaboration seems to impact student learning is by
sustaining student interest and motivation (to remain active
within a class) which can result in improved retention. We have
examples in the literature of collaborative learning situations
dramatically impacting retention in a SMET course with
historically low retention (e.g., \cite{pair-program,mediacomp}).
However, we know that, in very similar courses, collaboration can
fail to have any impact at all \cite{collab-no-work}.

We propose in this project to explore the ways in which CSCL can
impact student interest and motivation as factors in retention in
SMET courses. Our goal is to be able to make predictions about
student retention based on classroom culture and CSCL practices.
We then propose to directly apply the results by selecting courses
with low retention and testing our predictions by attempting to
improve retention by direct intervention in classroom practices
and through implementation of CSCL-based activities. The broader
impact is a deeper understanding of retention in SMET and
developing a research agenda around directly applying learning
science technologies toward classroom practice problems.

This work is in ROLE Quadrant Three, in that we are seeking to
improve SMET learning by improving retention via a CSCL-based
approach.  The third year of the project, where we seek to change
particular classes through our approach, will requirement
development of a systemic approach to identifying and addressing
retention problems, which starts moving the research into Quadrant
Four.

\newpage

\thispagestyle{empty}

\begin{center}
    {\Large \proposalTitle
}
\end{center}

\tableofcontents

\newpage

\begin{center}
    {\Large \proposalTitle
}
\end{center}

\pagenumbering{arabic}

\section{Introduction: The potential of CSCL to impact SMET learning}

Collaboration can be a significant factor in student learning.
Through collaboration students can reach understandings that none
of the collaborators held previously \cite{roschelle}.  Through
collaboration, students can learn how to collaborate better-a
significant result in fields where professionals must collaborate,
e.g., Engineering. Accreditation for Engineering curricula
requires collaboration \cite{abet} both because of the belief that
it aids in learning and because of the desire for students to
learn about collaboration itself \cite{augustine}.

Computer-supported collaborative learning (CSCL) is promising for
providing the benefits of collaboration in a form that can be more
effectively instituted in a wide range of classrooms.
Collaboration in its face-to-face form can be challenging to
institute in classes \cite{cohen}. Forming and maintaining groups,
arranging for meeting times, or possibly using class time can be
impediments to using collaborative learning opportunities. With
CSCL, collaboration can be just-in-time or asynchronous, and much
larger groups can participate without diminishing the learning
opportunities \cite{hoadley-linn}.

What we are starting to learn is that collaboration can also
influence retention.  Besides the opportunities that it provides
for learning, collaboration can influence factors that themselves
are important for learning.  In a couple of recent studies in
computer science (one of which used CSCL), collaborative learning
situations dramatically improved retention-students stuck with the
class longer than students had in similar classes without
retention, and (presumably) learned more than they would have had
they dropped out.

Retention is a significant issue in science, math, engineering,
and technology (SMET) education \cite{seymour, astin, hilton}.  Among minority populations and women, high attrition rates exacerbate the problem of underrepresentation in many scientific and technical fields. 
\cite{nsf-women, nsf-indicators}. It's clear that we lose many students in the SMET
educational pipeline. If we can influence retention
positively, we can improve SMET education. \mycomment{Allison,
Andrea-can we find any papers on this?}

\textbf{The goal of the proposed project is to understand how CSCL
can positively influence retention in SMET education, test our
understanding, and then use this understanding to change classroom
practice.}  We believe that CSCL can be effective at improving
retention and can be (relatively) easily instituted into classes.
Our three year study will develop a theory for how CSCL influences
retention in SMET classes, will test the theory by making
predictions about classes and matching them to observed classes,
and then going into SMET classes with low retention and seeking to
change them through use of CSCL-based activities in order to
improve retention.

This project is using a model for education and learning science
research that's related to design experiments \cite{brown-design},
but with different purposes.  Much learning science research draws
from psychology as its model-our goal is deep understanding of how
learning takes place.  The model in this research is more of an
engineering model than a science model.  We have a desired end
state, of a larger percentage of students remaining in classes
while learning. The point of our research is to study and attempt
to directly influence the factors that influence our end state
(e.g., motivation and students' interests), as opposed to deep
understanding of the factors themselves.  We expect to make
contributions to understanding about factors that influence
retention, but while we are attempting to directly change
education practice and outcomes.

\subsection{Where and How CSCL can Influence Retention}

The literature on collaboration does provide strong examples of
where collaboration has facilitated learning, e.g., Roschelle's
study of learning physics collaboratively \cite{roschelle}, or
Jeong and Chi's study of students learning anatomy in a
peer-learning setting \cite{jeong}, or the work on peer-tutoring
\cite{reciprocal}. Our definition of collaboration is where more
than one student is interacting with other students with a goal of
one or more of the students' learning.

We are also learning that CSCL can also be effective in
facilitating learning.  The earliest examples of CSCL showed
marked improvement on standardized test scores
\cite{higher-agency,csile-world3}. Later studies have shown that
students have been facilitated in their learning by CSCL in
physics \cite{hoadley-linn}, cognitive science \cite{miyake}, and
in our own work, English Composition \cite{coweb-english}.
CSCL-using students in our study even performed better on a
post-test than students not collaborating.

Our theory of how collaborative learning takes place says that
much of the learning arises from the dialog between students
\cite{roschelle, jeong}. Through their discussions, students make
hypotheses, test their hypotheses (often simply through the
discussion), and reformulate their hypotheses \cite{roschelle}.
However, CSCL practice suggests that other mechanisms are in play,
too \cite{nodialog}.  In actual practice, must less dialog is
observed than our theory says is needed for learning to occur, yet
we can measure learning.

It seems that CSCL plays more of a role in facilitating learning
than simply providing a forum for dialog.  In CSCL, the dialog is
visible and persistent.  Students can come back to a discussion
later, look at what others have discussed, search among
discussions, and even continue discussions that seemed to have
ended.  In our interviews with students, we find that CSCL can
play several roles in learning.  For some students, it's a
catalyst for individual learning, e.g., ``I saw what was posted,
and decided to go explore before posting anything.''  For others,
it's a catalyst for impromptu face-to-face collaborations, e.g.,
``We discussed what we saw in the [collaborative Web space].''

In recent studies, we've also found that collaboration can be a
factor in improving retention.  Retention is not a cognitive
construct, but a measurable outcome of other cognitive constructs
such as student interest (if you're more interested in a topic,
you'll more likely stay in class (cite?)) and motivation (your
desire to succeed in the class, sometimes despite interest).
\mycomment{Allison, help here, please.}  Collaboration, even in a
computer-supported version, has been shown to play a role in
improving retention.

\subsection{Improving Retention in SMET Classes with
Collaboration}

Two recent studies in computer science education research have
highlighted the potential for collaboration to improve retention.
The first used face-to-face collaboration, while the second used
CSCL.

Pair programming is a relatively new practice developed in
industry where pairs of programmers work together
\cite{williams2}. Pair programmers are strictly forbidden to use
any program code that was written by either individual programmer.
Instead, all code is written with one person typing and the other
person observing-sometimes critiquing, sometimes making
suggestions, sometimes just thinking it through with the
programmer at the keyboard.  Typically, the pairs switch roles
regularly.

Researchers such as Laurie Williams of North Carolina State
University have been exploring the use of pair programming in
introductory computer science.  Introductory CS has historically
low success rates-50\% of the class withdrawing or failing is
common \cite{retention-sigcse,failurerates}. Williams and her
colleagues have found that pair programming has not had
significant influence on performance in introductory classes
(e.g., the resultant code is not significantly better than
individuals' code) though it has in other classes
\cite{williams-phd}, nor has pair programming significantly
improved learning as measured by post-test performance (CSCL03).
However, there has been a significant improvement in retention.
Students in pair programming stick with the class more than
students who do not. \mycomment{Allison or Andrea, please insert
here some of her statistics about majors.}

The disadvantage to pair programming is that it's difficult to
manage. What happens when one of the pair doesn't show for a
session? Do you set up triplets or have the sole programmer work
alone (potentially at a disadvantage)? How do you form pairs? How
do you handle dysfunctional pairs? If CSCL in large groups could
have similar effects, we could have the advantages of
collaboration in improving retention without the cost of managing
the groups.

We recently developed and evaluated a pilot introductory computer
science course aimed particularly at non-majors \cite{mediacomp},
funded in part by NSF CCLI grant {\#}0231176. The course
\textit{Introduction to Media Computation} emphasized the
communications aspects of computation, by having students
manipulate pictures, sounds, text, and movies as the context for
their programming and learning. Collaboration played a significant
role in the course.

\begin{itemize}
\item Students were encouraged to collaborate on a subset of the
homework assignments. These were not explicitly organized by the
instructor and mostly occurred face-to-face. There were also
pre-quiz activities that students worked on explicitly and
collaboration was encouraged.
\item The course also had its own \textit{CoWeb} (Collaborative Webspace) \cite{coweb]-jls}
 which was used extensively in the course. Students could ask
 questions and make comments about the course (sometimes
 anonymously). Students were encouraged to post the media
 resulting from their work to share in \textit{Galleries}. Exam
 review questions were posted on the CoWeb with space created for
 questions, comments, and proposed solutions. The instructor would
 participate in the exam review to provide feedback on answers and
 help respond to questions. A \textit{Soapbox} that appeared at
 the top of every page in the CoWeb website was student editable
 and thus served as a way to raise awareness in the student
 community.
\end{itemize}
The course had a 90{\%} success rate---only 10{\%} of the class
withdrew or received an F or D. In contrast, non-majors in the
pair programming CS1 class had a 66{\%} success rate
\cite{pair-programming}. On a final survey, students in the
\textit{Media Computation} class were asked what was the most
important thing \textit{not} to change about the class. Nearly
20{\%} of respondents named the collaboration tool CoWeb and over
20{\%} mentioned collaboration in general. While we do not know
the relative influence of the course context vs. collaboration
(and collaboration vs. CSCL) in the improved retention in this
course, student responses suggest that collaboration may have
played a significant role.

\subsubsection{The Possible Roles of CSCL in Improving Retention}
\label{subsubsec:mylabel2}

Why should collaboration play a role in improving retention, if it
does? That's one of the significant research questions of the
proposed research. We do have some early hypotheses.

One of the factors that influences a students' motivation to
remain in the class is a sense of anonymity \cite{donald}. If a
student will be missed if she doesn't attend class, there is more
motivation to attend class. This is one of the explanations for
lower retention in large lecture-based classes. In a class that
uses face-to-face collaboration heavily (as in pair programming),
you will always be missed if you do not attend. Your collaborators
rely on you.

CSCL (at least, when the technology is asynchronous and
text-based) does not influence the same factor in the same way.
While students may develop a sense a community in the on-line
space \cite{amy-cscw}, thus reducing the sense of anonymity, there
is not the same pressure to physically be present in class.
Nonetheless, CSCL can inculcate some similar motivation to remain
a part of the class.

More powerful may be CSCL's influence on student interest and
motivation, factors in retention \cite{donald}, through projects.
SMET education is often project-based \cite{abet,augustine}.
Students work on homework, often requiring multiple sessions to
complete. Sustaining motivation is a critical factor in
project-based learning \cite{pbl} and technology in general can
play a role in sustaining that motivation \cite{krajcik}. If
students are interested in the project, they tend to be more
successful at it \cite{djoseph}, and technology can play a role in
maintaining interest \cite{djoseph2}.

Our analysis of interviews with students in the \textit{Media
Computation} class suggests that CSCL influences both motivation
to continue in the course and student interest. The particular
role that CSCL plays is not unlike the role of a case library in
supporting project-based learning \cite{lbdsw, cbr}. CSCL can
serve as a \textit{source for models} for both process and product
and a \textit{source for lessons-learned}, as a \textit{source for
feedback} well as a place to get questions answered. (Emphases
added below.)

\begin{itemize}
\item As a \textit{source for models}, we found that students used
the CoWeb to find out what others have done (e.g., in the
\textit{Galleries}) to be used as examples. They would also use
students' questions in the CoWeb to guide their own process.

\textbf{Q. What do you think about the homework galleries on the
coweb?}

\textbf{Student 2:} ``Oh like where you can put your picture up.
\emph{It's nice to see other people, like what they did with it.
Cuz, I'm not the most creative person and even if I was I wouldn't
know how to be creative in this!} And there is no better feeling
than getting something done and knowing that you've done it right,
\emph{like the Soapbox sometimes is just like `I did it!' and
posting to the CoWeb just adds onto that.}''

\item As a \textit{source for lessons-learned}, we found that
students would go to the CoWeb when first starting a project to
see the challenges that others were facing and sometimes to
jump-start their own process.

\textbf{Q: Do you think the CoWeb is beneficial? Why?}

\textbf{Student 1: }``Very beneficial, it keeps you on track,
keeps you organized. \emph{You get to see what people are having
problems with and maybe see... I always start off looking at what
people have had problems with.}''

\textbf{Q. How do you plan to study for the final exam?}

\textbf{Student 2: }``Hard. I have actually read the book. I read
it before the first test. I'll go through all the lecture slides
and look at old tests. And the online review [on the CoWeb] is
awesome and I'm very grateful he did that. I like that you can
make comments, \emph{I can see stuff where other people had
questions cuz they ask stuff that I might not have thought of.}
And I'll study with a group.''

\item As a \textit{source for feedback}, the CoWeb served as a place
to ask questions, inform others where you were in your process
(e.g., the Soapbox comment above), and to get feedback on the
clarity of your process. Anonymity, which is not possible easily
in face-to-face collaboration, played a role in putting students
at ease in asking questions---an important factor in retention
\cite{comfort-cs1}.

\textbf{Q. Have you ever posted to the CoWeb?}

``I think I've posted to everything. Sometimes I'll just make
random comments. \emph{Sometimes I ask a specific question and he
[the professor] asks for clarification.}''

\textbf{Do you think the CoWeb is beneficial? }

``Yes. \emph{And there's no reason to feel uncomfortable because
if you feel dumb, just don't put your name at the end!} I did that
a few times. ``
\end{itemize}

\subsection{When CSCL has Failed to Improve SMET Learning and Retention}
\label{subsec:mylabel2}
At the same time that we know that
collaboration, and CSCL specifically, \textit{can} play a role in
learning and retention, we also know of specific SMET learning
situations where it does \textit{not}. Part of our work on our NSF
REPP grant ({\#}REC-9814770) involved coming to understand where
collaboration is \textit{actively resisted} in SMET classes.

The initial focus of the project was to encourage integrative
learning among Engineering students by encouraging collaboration
around a common theme across disciplines. The theme we selected
was \textit{computer modeling}. We teach computer programming
(including MATLAB) in Computer Science (and now, Engineering)
freshmen courses, differential equations in Mathematics, and
implementation of differential equations in programs to model
Engineering problems in upper-division courses. The problem is
that these classes are not at all integrated: Terms for the same
things change between courses, problems used in earlier courses
are not at all like those that students see in later courses, and
the faculty don't communicate about these courses--faculty
communication across disciplinary boundaries is quite uncommon
(Cuban).

Our plan was to provide a collaboration space, the CoWeb (or
Swiki), which was being used successfully in several other courses
\cite{coweb-jls}. The idea was to provide a place where students
could go for help on any aspect of computer modelling, but since
the space encouraged collaboration, the help might come from
students in other disciplines taking other classes whose overlap
might inform one another. For the Senior struggling to remember
differential equations, the Sophomore taking it just then is the
best help. For the Sophomore, the Senior's problems tell her what
she has to look forward to.

The actual execution was much more disappointing. Students
actively avoided our efforts.

\begin{itemize}
\item To encourage collaboration in the CoWeb, we created a
mandatory assignment that required collaboration between a
Chemical Engineering and a Mathematics course. The students in
Chemical Engineering created simulations that generated data for
the Mathematics students to analyze, and then provide the results
back to the Chemical Engineers. 40{\%} of the Mathematics students
accepted a zero on the assignment rather than collaborate with the
Chemical Engineers.

\item One semester, we started using the CoWeb in a Freshman
Architecture course ($n$=171) at the same time that we started in
a Senior Chemical Engineering course ($n$=24). After ten weeks
into the semester, the Architecture students had generated over
1500 pages, with some discussion pages having over 30 authors. In
the Chemical Engineering course, not a single student had made a
single posting yet. In another semester, in a Computer Science
course of 340 students, only 22 students participated.

\item We had a hypothesis that part of the inhibition to
participate in the Engineering and Mathematics class was a
technical one. The content of many of these courses involves
equations, and equations are difficult to post on the Web. If
students couldn't ``talk'' in the modalities that were the most
comfortable for them, it would make sense that they would avoid
our tool. So, we created an applet-like tool that allowed users to
create equations by simply dragging and dropping components from
palettes, and then drop the equations into a bucket for rendering
to a GIF format and for easy posting. We installed it in a CoWeb
for a Mathematics class and for a Chemical Engineering class.
Faculty used it and praised it. Not a single student even
\textit{tried} it in either class.
\end{itemize}

After two years and over a dozen relatively-unsuccessful attempts
to get collaboration activities working in Engineering, some CS,
and Mathematics courses, we shifted our focus from pushing the
collaboration agenda to understanding why there was such
resistance to it. We found three main reasons for the lack of
collaboration \cite{coweb-fie,coweb-nowork}:

\begin{itemize}
\item \textit{Rational response to competitive conditions}: If
students perceived the course to be highly competitive, they often
believed it to be graded on a curve--even if there was evidence to
the contrary. When the course is graded on a curve, it's only
rational to avoid collaboration.

\item \textit{Learned helplessness}: In several courses, students
expected to do badly or expected to get little help, so
\textit{requesting} such help (in a collaborative, public setting)
was to label oneself and would probably not result in any help.

\item \textit{A lack of models}: Students don't see faculty
collaborating across disciplinary boundaries. They don't know what
to do, nor that it's valued.
\end{itemize}

The conclusion is that CSCL is no panacea. Students will not
necessarily participate, and without participation, we can have no
impact on learning nor retention. Thus, any theory that attempts
to explain how CSCL might influence retention must take into
account classroom culture and inculcating student interest even in
the collaboration itself.

\section{Project Plan: A Practice-Driven Process for Research, a Research-Driven
Process for Impacting Practice} \label{sec:mylabel2} The process
that we are proposing in this project means to coordinate research
with practice in such a way that we learn more about practice and
directly impact the practice. The research starts from a real
problem in practice, and then proceeds to develop a process for
applying the results of the research in practice to address the
original problem. Our proposed project seeks directly to improve
retention in SMET undergraduate classes using interventions based
on CSCL. We use the projects described
\cite{mediacomp,pair-programming} as evidence that collaboration
\textit{can} have an impact on retention. Our proposed work seeks
to understand when collaboration may be successful at improving
retention, test our understanding, and then apply that
understanding to change classes with historically low retention
rates.

This is a different model than most research in the learning
sciences. While it's in the vein of a design experiment
\cite{brown-design}, our goals are somewhat different. Like a
design experiment, we begin in real classrooms with all of their
complexity, and our goal is to understand better what's going on
in the situation. But while a design experiment seeks to
understand individual student psychology, our work seeks to
understand the classroom dynamics and not focus on the individual
student. In some sense, our approach is more of an engineering
approach than a scientific approach. While scientists seek to
understand all of a phenomenon, an engineer studies enough to make
it work \cite{kay-foreword}. In so doing, the engineer often
gathers that data that drives the scientific enterprize. We plan
to play that kind of role: Learning how to directly affect
retention rates through manipulation of factors related to
collaborative learning, and then using that knowledge to impact
courses with historically low retention rates.

A brief sketch of our plan follows:

\begin{itemize}
\item Year One: We develop a theory about how classroom culture and
collaboration practices relate to retention, by studying some of
the 40 courses per term that use our CoWeb collaborative software
and by studying comparable courses that do not use any form of
collaboration. We essentially want to develop a qualitative
regression equation that describes how factors such as perception
of competition and the availability of models relate to
intermediate factors such as student interest and motivation which
drive our factor of interest, retention.

\item Year Two: We then test our theory and develop an
intervention strategy. Each term during Year Two, we will select a
sample of courses, some using the CoWeb and some not using
collaborative learning activities at all, and develop a prediction
based on our theory of anticipated retention. We use CSCL because
we expect that if will be the easiest for teachers to adopt in our
Year Three efforts. We will study those courses (e.g., use surveys
to establish student interest and motivation in the course) and
refine our theory.

\item Year Three: We apply our theory. Given our theory, we will
develop a system to identify courses with historically low
retention rates which might be amenable to our interventions,
e.g., changing grading practices to reduce competition, and
provide models and discussion spaces via the CoWeb. We will work
with faculty in those courses to bring about change, and then
measure our factors to look for impact of our intervention and
match to our theory.
\end{itemize}

\subsection{Year 1: Creating a theory about retention}
\label{subsec:mylabel3} During the first year of the study, we
plan to develop a theory about how collaboration impacts
retention. We are not under any illusions that collaboration is
the \textit{sole} factor impacting retention in any post-secondary
course, however we do believe that collaboration can be a
\textit{significant} factor, and it is a factor that we have some
control over.

Collaborative activities encompass student experiences from
explicitly collaborative activities created by and/or sanctioned
by the course instructor through collaborations that the course
instructor allows and implicitly encourages by a liberal
collaboration policy and a grading policy that does not penalize
students for working together. We have learned that the implicit
encouragement is perhaps more critical for successful
participation in collaborative activities than the explicit
encouragement \cite{coweb-fie,collab-no-work}, but the explicit
encouragement is noted by the students and does have an impact.

Our strategy for developing a theory about collaboration is to
measure these variables in studying four kinds of classes.  We
will choose a sample of each of these classes to study during each
of the three semesters in Year 1, for a total of some 20 courses
over the course of the year.

\begin{itemize}
\item SMET classes with high retention. Our analysis of courses at
Georgia Tech suggest that high retention may be a relative
concept---rarely in first and second year courses is the WDF rate
(percent of enrolled students who Withdraw, or who earn a grade of
D or F) in single digits.

\item SMET classes with historically low retention.
Table~\ref{tbl:wdfrates} summarizes the WDF (Withdraw, or earned
grade of D or F) rates of several potential target courses at
Georgia Tech.

\item Similar SMET classes using collaborative activities. Not all
sections of these courses are taught in the same way---the
statistics hide the activities of particular instructors and the
use of collaborative activities within those classes. We do not
have evidence right now that there \textit{are} sections of
similar classes that are using collaborative activities without
technological support, but we plan as part of our survey to seek
out these kinds of activities.

\item As a significant subset of the second kind, we are interested
in similar SMET classes that are using CSCL for collaborative
activities of some kind. We know that simply using CSCL tools does
not imply participation nor successful learning (FIE01, ICLS02).
However, use of a CSCL tool indicates the potential for
incorporating successful collaborative learning activities, and
use of a tool is easier to incorporate into courses that have no
collaborative component. Moreover, CSCL-using classes with low
retention help us to understand better what CSCL can impact and
what it cannot.
\end{itemize}


At Georgia Tech, there are a reasonable number of courses in that
last group of SMET classes. Last semester, some 45 classes used
our CoWeb collaborative technology. Some 20 of these were in
Computer Science, 5 in Mathematics, and another 10 were in
Chemical Engineering. Currently, \textit{all} undergraduate
Chemical Engineering classes are automatically provided with a
CoWeb at the start of each semester.

\begin{table}[htb]
\begin{center}
\begin{tabular}{||l|p{1 in}|l|l|l|l|l||}
\hline\hline Course& Title& Term& $N$ & W& D/F&
DFW \\
\hline \hline
CS1050 & Understanding and constructing proofs& Fall
2001& 234& 10.68{\%}& 14.10{\%}&
24.79{\%} \\
\cline{3-7}

 & & Fall 2002& 213& 4.23{\%}& 17.37\% & 21.6\%
 \\ \hline

CS1321 & Introduction to Computing & Fall 2001 & 1168 & 7.96\% &
21.06 \% & 29.02\% \\ \cline{3-7}

 & & Spring 2002 & 935 & 8.88\% & 25.99\% & 34.87\% \\ \cline{3-7}

 & & Fall 2002 & 1316 & 7.07\% & 18.77 \% & 25.84\% \\ \cline{3-7}

 & & Spring 2003 & 879 & 10.69\% & 27.08\% & 26.02\% \\ \hline

CHE2100 & Chemical Process Principles & Fall 2001 & 87 & 6.9\% &
10.34\% & 17.24\% \\ \cline{3-7}

 & & Spring 2002 & 31 & 0\% & 25.81\% & 25.81\% \\ \cline{3-7}

 & & Fall 2002 & 90 & 8.89\% & 16.67\% & 25.56\% \\ \hline

ECE2025 & Introduction to Signal Processing & Fall 2001 & 304 &
8.88\% & 9.54\% & 18.42\% \\ \cline{3-7}
 & & Fall 2002 & 272 & 6.25\% & 9.19\% & 15.44\% \\ \hline

Chem1311 & Inorganic Chemistry I & Fall 2002 & 136 & 10.29\% &
25.74\% & 36.03\% \\ \hline

Math1501 & Calculus I & Fall 2001 & 1421 & 3.31\% & 17.8\% &
21.11\% \\ \cline{3-7}
 & & Spring 2002 & 225 & 7.11\% & 42.22\% & 49.33\% \\ \hline

PHYS2211 & Introductory Physics I & Fall 2001 & 634 & 9.31\% &
29.02\% & 38.33\% \\ \hline\hline
\end{tabular}
\caption{A selection of Withdraw, D, or F rates for lower-division
undergraduate courses at Georgia Tech \label{tbl:wdfrates}}
\end{center}
\end{table}

We believe that collaboration impacts retention through at least
two direct variables that CSCL has the opportunity to impact.

\begin{itemize}
\item \textbf{Interest}: Collaboration can encourage and inculcate
student interest. We see (e.g., in earlier quotes) that students
are inspired by the work that they see other students doing. We
know that student discussions do inspire students to explore
issues more deeply \cite{no-dialog}.

\item \textbf{Motivation to succeed in the course: }Collaboration
can help convince students that they can succeed (as well as
potentially help them learn so that they can succeed).
Collaboration can help in answering questions, in receiving
support in the course, and in reducing students' sense of
anonymity (unless they want it!). In general, collaboration can
make the class seem more \textit{doable}---that it's something
that students can be successful at.
\end{itemize}

We know from the literature that there are many other factors that
influence retention \cite{pintrich}.  Some of the more significant
ones that we plan on studying: \mycomment{Allison, anything else
that we should mention here?}
\begin{itemize}
\item Number of students in a class or section. Obviously, the
number of students doesn't \emph{directly} lead to students'
failure to succeed at a course. However, more students in a class
leads to a greater sense of anonymity which can influence
students' decision to withdraw or skip class \cite{donald}.
Greater numbers may also inhibit's students' comfort with asking
questions, which has been highly correlated with success in
introductory computer science courses \cite{comfort-cs1}.

\item The instructor plays a very significant role in retention in
a course.  We've found that instructors' attitudes and modelling
of behavior influences students perceptions about the course
\cite{coweb-fie, collab-no-work}.

\item Student self-perceptions about efficacy in the class
\cite{code-warriors}.  If students enter a class convinced that
they won't be successful, they probably won't be.  Student
perception that the situation is manageable improves motivation to
succeed \cite{paris-motivation}.  To what students attribute their
efficacy is also correlated with success in the course.  Students
who do not succeed in introductory computer science courses tend
to attribute their performance to external factors such as
``luck'' as opposed to internal factors such as ``hard work''
\cite{comfort-cs1}.
\end{itemize}

Our plan is to conduct surveys and interviews among teachers and
students (both those who complete the course and those who do
not). Our study plan is described in Table~\ref{tbl:studyplans}.
The idea is to study the courses at Georgia Tech that can inform
us about retention and about the role that CSCL is and might play
with a focus on the variables that are identified in the
literature and in our own work that are significantly related to
retention.  Where possible, we plan to use and adapt existing
measures for these factors, such as the MSLQ for measuring
motivation that has been used successfully with undergraduate
students \cite{mslq}.

\begin{table}[htb]
\begin{center}
\begin{tabular}{||p{0.5in}|p{1.5 in}|p{0.5 in}|p{1 in}||}\hline\hline
Class of participants & Factors to be studied & Frequency & Form
of measurement \\ \hline\hline

Faculty & Attitudes toward collaboration, attitudes toward student
efficacy and achievement, policies toward collaboration,
collaborative activities in the class & Before class starts &
Surveys and interviews in particularly low-retention classes \\
\hline

All students & Interest in the course, motivation to succeed in
course, comfort asking questions, perceptions about course & First
week of course & Surveys \\ \hline

Students who drop the course & Interest in the course, motivation
to succeed in course, reasons for dropping the course, perceptions
about course & After drop day & Interviews (including phone and
email) \\ \hline

Students who complete the course & Interest in the course,
motivation to succeed in course, expected grade in course, cause
of grade, perceptions about course & End of class &
Surveys and interviews with a sample of volunteer students \\
\hline\hline
\end{tabular}
\end{center}
\caption{Study plans in first year in targeted
courses\label{tbl:studyplans}}
\end{table}

\subsection{Year 2: Testing the Theory and Defining the Intervention}
\label{subsec:mylabel4}

Based on our study in Year 1, we will develop a theory that will
allow us to predict retention rates.  During Year Two, we will
apply that theory to develop \emph{a priori} predictions about
retention in the courses.  We will then replicate the study of
Year 1 to gather data to refine the theory.  At the end of each
term, we will establish the reliability of our theory.  By the end
of Year 2, we expect to have reasonable accuracy at predicting
retention rates.

Even from the start of Year 2, though, we will have a good idea of
what factors are most significantly related to retention in our
target classes, and which variables might be influenced by CSCL.
During Year 2, we will develop our intervention
plans---determining what factors we might influence via CSCL, and
then developing materials and activities for the target classes
which will focus on those factors.

Obviously, not all factors, not even all significant factors, will
be amenable to intervention using CSCL.  However, we do believe
that CSCL will be able to play a role in \emph{reducing} failure
rates in targeted courses.  We will begin with our hypotheses
about the roles that CSCL can play in influencing retention: a
\textit{source for models} for both process and product, a
\textit{source for lessons-learned}, and a \textit{source for
feedback} well as a place to get questions answered.

To prepare for intervention, we will create:
\begin{itemize}
\item Activity materials for teachers, like our successful \emph{CoWeb
Catalog} \cite{coweb-catalog} that describes activities that
teachers have invented with the CoWeb \cite{coweb-jls}.  The new
materials will focus on activities that will directly impact the
factors that we decide are most amenable for change.  The idea is
to show teachers ways that they can incorporate the CoWeb to
improve retention in their courses.
\item Course suggestion materials which will describe how to
change grading policies or influence course perception in ways to
improve retention.
\item In some instances, modifications to our CoWeb software to
make it easier to implement these activities and suggestions.
\end{itemize}

\subsection{Year 3: Applying the Theory and Interventions}
\label{subsec:mylabel5}

Year 3 is about directly applying the theory and materials that
were developed and verified in Years 1 and 2.  There are two parts
to our goals for Year 3:
\begin{itemize}
\item We need to identify target courses for improving retention.
\item We need to implement the interventions.
\end{itemize}

\subsubsection{Identifying Target Classes}
\label{subsubsec:mylabel4}

We plan to develop a process for selecting courses for
intervention.  The goal is to find courses that are most amenable
to intervention and improvement, and to systematize the process so
that such targeting and intervention can become a regular practice
in the institution.  The process we are currently planning
includes:
\begin{enumerate}
\item Identifying potential target courses based on historically
low retention, e.g., WDF over 20\%.
\item Survey of teachers for those targeted courses in Year 3, to
determine which ones would have attitudes and policies most
amenable to intervention.
\item Approaching these teachers with our materials and asking
them if they would be willing to change their activities and
policies.
\end{enumerate}

We have the support of Robert McMath, Georgia Tech's Vice-Provost
for Undergraduate Education, that we can use in legitimatizing our
effort.  We will also be working with Donna C. Llewellyn, the
Director of Georgia Tech's Center for Enhancement of Teaching and
Learning.  Dr. Llewllyn has a great deal of experience in working
with faculty and encouraging changes in classrooms.  We will be
seeking her advice on how to handle Step 3 in the most effective
manner.

\subsubsection{Implementing the intervention}
\label{subsubsec:mylabel5}

Once we have identified a willing teacher, we plan to work with
the teacher through regular meetings to help him or her implement
the activities and changes that our theory will recommend.  We do
\emph{not} plan to be cautious and objective observers.  Our hope
is to cause a change in retention rates.  We hope to see an
institutional change such that the intervention becomes permanent,
but that's beyond the scope of this project.  Our goal here is to
cause a change through targeted intervention.  While we recognize
that we will be susceptible to a Hawthorne effect, we agree with
the late Ann Brown that that's a desirable outcome as part of a
process leading to deeper and persistent change
\cite{brown-design}.

We will record the amount of effort used in the interventions, as
we have in prior studies \cite{coweb-english}.  Our belief is that
CSCL will allow for a relatively easy to implement change that
will impact retention.  We plan to measure how easy that is.  Our
current plan is to probe the teachers on a regular basis to get
estimates of how much time the intervention is requiring.

During the intervention, we will continue to gather data as
described in Table~\ref{tbl:studyplans} so that we can match
progress to our theory.  In this way, we hope that we can learn
about both the process of change of retention rates in the
classes.

\subsection{Project Roles and Involvement of Students}
\label{subsec:mylabel6}

Mark Guzdial will be the PI of the project.  He will be
responsible for overall management of the project and the
achievement of project and dissemination goals.

Allison Elliot Tew will be a Research Scientist on the project.
For four years, Allison has been the Director of Student Services.
She's familiar with the systems for gathering data on classes, and
with the task of encouraging and cajoling faculty into behavior
for the students' best interests.  She also has prior experience
in project management and software engineering.  She'll handle the
day-to-day management of the project, and will be the main person
developing the measurement instruments used in the project,
reviewing the data and results, developing the theory, evaluating
the theory, developing the interventions, and working with faculty
to implement the interventions.

We will hire two Graduate Student Research Assistants (GSRAs) on
the project.  Their responsibilities will include:
\begin{itemize}
\item Data collection and analysis, including interviews,
\item Developing and maintaining software associated with the
project,
\item Developing materials for the interventions,
\item Providing support to the teachers as they begin the
interventions.
\end{itemize}

In addition, we anticipate working with individual students who
will join us looking for research opportunities.  We have been
very fortunate in finding talented students who are interested in
the goals of our projects.  For example, on the \emph{Media
Computation} course project, all the interviews were undertaken by
a talented undergraduate student, Lauren Rich, who wanted to study
gender issues in the course.  In this way, we hope to be able to
explore particular angles and offshoots of the project as the
arise, while involving additional students.

\subsection{Dissemination Plans}
\label{subsec:mylabel7}

Primarily, we hope to disseminate our results to two audiences:
\begin{itemize}
\item Learning Scientists (e.g., \emph{International Conference of
the Learning Sciences}, \emph{Computer-Supported Collaborative
Learning Conference}, and \emph{Journal of the Learning Sciences})
who will be interested in our theory of retention related to CSCL,
our intervention plans, and our results, as well as our process
for targeting and intervening in low-retention classes.
\item SMET teachers and administrators who will be interested in
mechanisms for improving the poor retention rates in these
classes.  We anticipate reaching Engineering and Computer Science
educators through the IEEE/ASEE \emph{Frontiers in Education
(FIE)}, ACM \emph{SIGCSE} (SIG Computer Science Education), and
\emph{Innovation and Technology in CS Education} (ITiCSE)
conferences.  We also hope to publish in \emph{IEEE Transactions
on Education} and \emph{Journal of Engineering Education} as high
quality and visibility forums for this research.
\end{itemize}

As we target courses in other disciplines (e.g., Mathematics and
Physics), we plan to reach out to those audiences as we have with
our CoWeb work (e.g., \cite{coweb-math}).

We anticipate to have two conference papers per year, and at least
two journal papers over the course of the three year project.  In
addition, we will make all of our intervention materials and our
software available on our websites.

Our websites have been an effective mechanism of intervention for
us. Our CoWeb/Swiki software and associated materials are posted
there, indexed by Google.  A search today for ``Swiki'' on
\url{http://www.google.com} returns 126,000 hits. A review of the
first couple hundred of these hits shows that the vast majority of
hits are sites using our software (available at
\url{http://minnow.cc.gatech.edu/swiki}). American schools using
CoWebs to support teaching and research including University of
Colorado at Boulder, Allegheny College, Lakota University,
University of Kansas, and the National Center for Atmospheric
Research.  Uses of the CoWeb outside of the United States can be
seen in domains .sk, .be, ch., .jp, .se, .fr, and .nz and include
significant U.S. research collaborators such as ETH.

\section{Conclusion}
\label{sec:mylabel3}

The overall focus of our project is to conduct research targeted
at a significant and stubborn problem in SMET courses.  We don't
expect that we will be able to make 90\% of students succeed in
every SMET class at Georgia Tech. But if can lower some of the
20--30\% WDF rate classes into the 10--20\% range, we will have
accomplished something significant, since we will have carefully
engineered and documented our process of intervention.  In
addition, our process of choosing a research question in terms of
practice and then addressing that question will serve as a model
for future joint practice-research agendas.


\subsection{Budget Justification}
\label{subsec:mylabel8}

\emph{Personal Services}. We request 1 month of summer salary for
Mark Guzdial per year, and 12 months of Allison Tew per year, for
all three years of the grant.  In addition, we request funding and
tuition for the two GSRA's on the project.  We also request
funding for 1/2 of a shared post-doc for maintenance of laboratory
equipment in the GVU Center.  We are requesting fringe and
computing charges for these personnel.

\emph{Equipment}: None is requested.

\emph{Travel}: We request \$2,000/year for travel expenses for
attending conferences where we are presenting results from this
work.

\emph{Materials and Supplies}: We ask for \$2,000/year for
miscellaneous expenses, like toner cartridges and software
upgrades.

\section{Facilities}

The College of Computing maintains a variety of computer systems
in support of academic and research activities.  These include
more than 50 Sun, Silicon Graphics, and Intel systems used as file
and compute servers, many of which are quad-processor machines.
In addition, there are more than 1,000 workstation class machines
from Sun, Silicon Graphics, Intel, and Apple especially for
student use.  A number of specialized facilities augment these
general-purpose computing capabilities.  The hardware that will be
purchased for this project will be of similar quality to what the
students use, for testing purposes, but will be set up to
facilitate development.

The Graphics, Visualization, and Usability (GVU) Center houses a
variety of graphics and multimedia equipment, including
high-performance systems from Silicon Graphics, Sun, Intel, and
Apple.  The affiliated Multimedia, Computer Animation, Audio/Video
Production, Usability/Human Computer Interface, Virtual
Reality/Environments, Electronic Learning Communities,
Computational Perception, Software Visualization, Biomedical
Imaging, Collaborative Software, and Future Computing Environments
labs provide shared facilities targeting specific research areas.
These laboratories' equipments will be of use in developing our
multimedia projects.

PI Guzdial is the Director of the Collaborative Software Lab,
affiliated with GVU.  The Collaborative Software Lab has a bank of
ten servers supporting our experimental software for studying
computer-supported collaborative learning.  In addition, we have
three Linux workstations, two NT workstations, and two Apple
workstations used for development.  The focus of the Collaborative
Software Lab is on facilitating multimedia collaboration, so
multimedia facilities available include a high-end Alesis
keyboard, projection facilities, a Canon digital video camera, and
a Sony Mavica digital camera.

All of the College's facilities are linked via local area networks
which provide a choice of communications capabilities from 10 to
1000 Mbps.  The College's network employs a high-performance OC12C
(622 Mbps) ATM and GigabitEthernet (1000 Mbps) backbone, with
connectivity to the campus ATM network provided via OC12C.  The
primary campus Internet connection is provided by a direct 100
Mbps link to the service provider's Atlanta switching center,
augmented by OC3C ATM and OC12C connections, respectively, to the
NSF vBNS (very high performance Backbone Network Service) and
Abilene research networks.  Georgia Tech is also leading southern
regional gigabit network efforts (SoX.net, the Southern
Crossroads) as part of Internet2.

Additional computing facilities are provided to the Georgia Tech
campus by the Institute's Office of Information Technology (OIT),
including five public-access clusters of Sun, Apple, and Dell
workstations, a collection of Sun multi-processors which are
treated as a single computational resource via login load sharing,
and various mainframes.

\newpage


\bibliographystyle{plain}
\bibliography{proposal,finalreport}

\end{document}