Christopher Hundhausen

Abstract: Decades of research have led to notable improvements in the representations used to aid human comprehension of computer programs. Much of this research has focused on visual representations, which leaves open the question of how best to design auditory representations of computer programs. While this question has particular relevance for visually impaired programmers, sighted programmers might also benefit from enhanced auditory representations of their programs. In order to investigate this question empirically, first, we introduce artifact encoding, a novel approach to rigorously measuring the comprehensibility of auditory representations of computer programs. Using this approach as a foundation, we present an experimental study that compared the comprehensibility of two alternative auditory program representations: one with lexical scoping cues that convey the nesting level of program statements, and another without such scoping cues. The results of our first experiment validate both artifact encoding and the scoping cues we used. To see whether auditory cues validated through our paradigm can aid program comprehension in a realistic task scenario, we experimentally compared programmers' ability to debug programs using three alternative environments: (1) an auditory execution environment with our empirically derived auditory cues; (2) an auditory execution environment with the current state-of-the-art auditory cues generated by a screen reader running on top of Microsoft Visual Studio; and (3) a visual version of the execution environment. The results of our second experiment showed that our comprehensible auditory cues are significantly better than the state-of-the-art, affording human performance approaching the effectiveness of visual representations within the statistical margin of error. This research contributes a novel methodology and foundational empirical data that can guide the design of effective auditory representations of computer programs.

Abstract: Background Introductory gateway engineering courses are notorious for their high attrition rates. Deficiencies in students� problem-solving processes may contribute to their failure in these courses. In an empirical study of student problem solving, we observed that students struggle because of misconceptions regarding the basic syntax and semantics of disciplinary diagrams and corresponding mathematical equations. Purpose (Hypothesis) We hypothesize that a scaffolded software environment that provides dynamically-generated feedback on the syntactic and semantic correctness of students� evolving disciplinary diagrams and mathematical equations can improve engineering students� problem-solving abilities. Design/Method We iteratively developed ChemProV, a software environment to assist chemical engineering students in solving material balance problems. To evaluate ChemProV�s effectiveness, we performed two between-subjects experimental studies. The first study compared a preliminary version of the ChemProV to pen-and-paper. The second study compared a redesigned version of ChemProV with dynamic feedback to the same version of ChemProV without dynamic feedback. Results While it did not uncover any significant differences, the first study provided insight into how to improve ChemProV's dynamic feedback mechanism. The second study found that the �feedback� version of ChemProV promoted a statistically-significant advantage in problem-solving accuracy, significantly more time-on-task, and a transfer-of-training to an unscaffolded problem-solving situation. Conclusions A scaffolded software environment like ChemProV can serve as a valuable aid in helping students learn engineering problem-solving skills. Its software design approach can be used as a model for designing educationally-effective software environments for other engineering disciplines.

Abstract: Studio-based learning (SBL), a pedagogical technique that promotes learning through the iterative construction and review of problem solutions, is becoming increasingly popular in computer science education. This paper reviews the use of SBL in computing education with respect to the ways in which it has been implemented, its impact on the curriculum, and the educational outcomes it has promoted.

Abstract: Novices face many barriers when learning to program a computer, including the need to learn both a new syntax and a model of computation. By constraining syntax and providing concrete visual representations on which to operate, direct manipulation programming environments can potentially lower these barriers. However, what if the ultimate learning goal of the novice is to be able to program in conventional textual languages, as is the case for introductory computer science students? Can direct manipulation programming environments lower the initial barriers to programming, and, at the same time, facilitate positive transfer to textual programming? To address this question, we designed a new direct manipulation programming interface for novices, and conducted an experimental study to compare the programming processes and outcomes promoted by the direct manipulation interface against those promoted by a textual programming interface. We found that the direct manipulation interface promoted significantly better initial programming outcomes, positive transfer to the textual interface, and significant differences in programming processes. Our results show that direct manipulation interfaces can provide novices with a �way in� to traditional textual programming.

Abstract: Within the context of an introductory CS1 unit on algorithmic problem-solving, we are exploring the pedagogical value of a novel active learning activity-the ''studio experience''-that actively engages learners with algorithm visualization technology. In a studio experience, student pairs are tasked with (a) developing a solution to an algorithm design problem, (b) constructing an accompanying visualization with a storyline, and finally (c) presenting that visualization for feedback and discussion in a session modeled after an architectural ''design crit.'' Is a studio experience educationally valuable? What kind of technology can best support it? To explore these questions, we conducted an empirical study of two alternative CS1 studio experiences in which students used one of two different kinds of algorithm development and visualization technology: (a) a text editor coupled with art supplies, or (b) ALVIS Live!, a computer-based algorithm development and visualization tool. We found that the students who used ALVIS Live! developed algorithms with significantly fewer semantic errors. Moreover, discussions mediated by ALVIS Live! had significantly more student audience contributions, and retained a sharper focus on the specific details of algorithm behavior, leading to the collaborative identification and repair of semantic errors. In addition, discussions mediated by both ALVIS Live! and art supplies contained substantial evidence of higher order thinking. Based on our results, we make recommendations for educators interested in exploring studio-based approaches, and we propose an agenda for future research into studio-based learning in computer science education.

Abstract: Pedagogical algorithm visualization (AV) systems produce graphical representations that aim to assist learners in understanding the dynamic behavior of computer algorithms. In order to foster active learning, computer science educators have developed AV systems that empower learners to construct their own visualizations of algorithms under study. Notably, these systems support a similar development model in which coding an algorithm is temporally distinct from viewing and interacting with the resulting visualization. Given that they are known to have problems both with formulating syntactically correct code, and with understanding how code executes, novice learners would appear likely to benefit from a more ''live'' development model that narrows the gap between coding an algorithm and viewing its visualization. In order to explore this possibility, we have implemented ''What You See Is What You Code,'' an algorithm development and visualization model geared toward novices first learning to program under the imperative paradigm. In the model, the line of algorithm code currently being edited is reevaluated on every edit, leading to immediate syntactic feedback, along with immediate semantic feedback in the form of an AV. Analysis of usability and field studies involving introductory computer science students suggests that the immediacy of the model's feedback can help novices to quickly identify and correct programming errors, and ultimately to develop semantically correct code.

Abstract: Prior empirical studies of programming have shown that novice programmers tend to program by exploration, relying on frequent compilation and execution of their code in order to make progress. One way visual and end-user programming environments have attempted to facilitate this exploratory programming process is through their support of ''live'' editing models, in which immediate visual feedback on a program's execution is provided automatically at edit time. Notice that the notion of ''liveness'' actually encompasses two distinct dimensions: (a) the amount of time a programmer must wait between editing a program and receiving visual feedback (feedback delay); and (b) whether such feedback is provided automatically, or whether the programmer must explicitly request it (feedback self-selection). While a few prior empirical studies of ''live'' editing do exist, none has specifically evaluated the impact of these dimensions of ''live'' editing within the context of the imperative programming paradigm commonly taught in first-semester computer science courses. As a preliminary step toward that end, we conducted an experimental study that investigated the impact of feedback self-selection on novice imperative programming. Our within-subjects design compared the impact of three different levels of feedback self-selection on syntactic and semantic correctness: (a) no visual feedback at all (the No Feedback treatment); (b) visual feedback, in the form of a visualization of the program's execution state, provided on request when a ''run'' button is hit (the Self-Select treatment); and (c) visual feedback, in the form of a visualization of the program's execution state, updated on every keystroke (the Automatic treatment). Participants in the Automatic and Self-Select treatments produced programs that had significantly fewer syntactic and semantic errors than those of the No Feedback treatment; however, no significant differences were found between the Automatic and Self-Select treatments. These results suggest that, at least in the case of novice imperative programming environments, the benefits of delivering a continuously updated visual representation of a program's execution may fail to justify the substantial costs of implementing such feedback. We recommend that programming environment designers instead direct their efforts toward carefully considering when programmers will be ready to take advantage of the feedback that is coming toward them, along with what content will be of most benefit to them.

Abstract: An end-user visualization environment aims to empower end users to create graphical representations of phenomena within a scientific domain of interest. Research into end-user visualization environments has traditionally focused on developing the human-computer interaction necessary to enable the quick and easy construction of domain-specific visualizations. That traditional focus has left open the question of how such environments might support human-human interaction. Especially in situations in which end-user visualization environments are enlisted to facilitate learning and to build design consensus, we hypothesize that a key benefit is their ability to mediate conversations about a scientific domain of interest. In what ways might end-user visualization environments support human communication, and what design features make them well-suited to do so? Drawing both on a theory of communication, and on empirical studies in which end-user environments were enlisted to support human communication, we propose a provisional framework of six 'Communicative Dimensions' of end-user visualization environments: programming salience, provisionality, story content, modifiability, controllability, and referencability. To illustrate the value of these dimensions as an analytic and design tool, we use them to map a sample of publicly available end-user visualization environments into the 'Communicative' design space. By characterizing those aspects of end-user visualization environments that impact social interaction, our framework provides an important extension to Green and Petre's (J. Visual Lang. Comput. 7 (1996) 131-174) 'Cognitive Dimensions'.

Abstract: This paper reports an empirical study comparing the role of discourse and knowledge representations (graphical evidence mapping) in face-to-face versus synchronous online collaborative learning. Prior work in face-to-face collaborative learning situations has shown that the features of representational notations can influence the focus of learners' discourse and collaborative activities. Two hypotheses were considered in the present study: (1) The influence of knowledge representations in the online condition could be weaker because of the lack of shared awareness and meaning that results from working together in front of a physically shared display, and because of the greater difficulty of utilizing the representations as a resource for conversation through gestural deixis, and (2) The influence of knowledge representations in the online study could be stronger because participants must rely more on them to compensate for the absence of face-to-face modes of communication. Quantitative results largely support the second hypothesis. There was greater consideration of certain coding categories supported by the knowledge representation software. However, essay quality and other observations provide indirect support for the first hypothesis. Explanations for these results and implications for the design of online collaborative learning environments are provided.

Abstract: The importance of both social processes and of representational aids for learning is well-established, yet few experimental studies have addressed the combination of these factors. The research reported in this article evaluates the influence of tools for constructing representations of evidential models on collaborative learning processes and outcomes. Pairs of participants worked with 1 of 3 representations (Graph, Matrix, Text) while investigating complex science and public health problems. Dependent measures included (a) the content of participants' utterances and representational actions and the timing of these utterances and actions with respect to the availability of information; (b) a multiple choice test of the ability to recall the data, hypotheses, and evidential relations explored; and (c) the contents of a written essay. The results show that representational notations can have significant effects on learners' interactions, and may differ in their influence on subsequent collaborative use of the knowledge being manipulated. For example, Graph and Matrix users elaborated on previously represented information more than Text users. Representation and discussion of evidential relations was quantitatively greatest for Matrix users as predicted, yet this came at the cost of excessive consideration and revision of unimportant relations. Graph users may have been more focused in their consideration of evidence, and the work done in the Graph representation had the greatest impact on the contents of the essays. Although limited to initial use of representations in a laboratory setting, the work demonstrates that representational guidance of collaborative learning is worthy of study and suggests several lines of further investigation.

Abstract: Algorithm visualization (AV) software graphically illustrates how algorithms work. Traditionally, computer science instructors have used the software as a visual aid in lectures, or as the basis for interactive laboratories. An alternative approach, inspired by Social Constructivist learning theory, is to have students construct and present their own visualizations. Notice that, in this alternative approach, rather than acting as a knowledge conveyer that transfers an expert's mental model of an algorithm to students, AV software grants students access to forms of expert course participation typically reserved only for instructors. To explore this alternative approach, I conducted a pair of ethnographic field studies in a junior-level algorithms course that included AV construction and presentation assignments. Through a broad range of field techniques, including participant observation, interviewing, videotaping, and diary collecting, I gained crucial insights into the approach. My most significant finding was that, when used within the context of AV construction and presentation exercises, conventional AV software can actually distract students from concentrating on activities and concepts relevant to an undergraduate algorithms course. When supported by an alternative, "low tech" version of AV technology, however, AV construction and presentation appear not only to focus students on relevant activities and concepts, but also to enable them to participate more extensively the course, thus contributing to their learning.

Abstract: Algorithm visualization (AV) technology graphically illustrates how algorithms work. Despite the intuitive appeal of the technology, it has failed to catch on in mainstream computer science education. Some have attributed this failure to the mixed results of experimental studies designed to substantiate AV technology's educational effectiveness. However, while several integrative reviews of AV technology have appeared, none has focused specifically on the software's effectiveness by analyzing this body of experimental studies as a whole. In order to better understand the effectiveness of AV technology, we present a systematic meta-study of 24 experimental studies. We pursue two separate analyses: an analysis of independent variables , in which we tie each study to a particular guiding learning theory in an attempt to determine which guiding theory has had the most predictive success; and an analysis of dependent variables , which enables us to determine which measurement techniques have been most sensitive to the learning benefits of AV technology. Our most significant finding is that how students use AV technology has a greater impact on effectiveness than what AV technology shows them. Based on our findings, we formulate an agenda for future research into AV effectiveness.

Abstract: Computer science educators have traditionally used algorithm visualization (AV) software to create graphical representations of algorithms for use as visual aids in lectures, or as the basis for interactive labs. Typically, such visualizations are high fidelity in the sense that (a) they depict the target algorithm for arbitrary input, and (b) they tend to have the polished look of textbook figures. In contrast, low fidelity visualizations illustrate the target algorithm for a few, carefully chosen input data sets, and tend to have a sketched, unpolished appearance. Drawing on ethnographic field studies of a junior-level algorithms course, we motivate the use of low fidelity AV technology as the basis for an alternative learning paradigm in which students construct their own visualizations, and then present those visualizations to their instructor and peers for feedback and discussion. To explore the design space of low fidelity AV technology, we present SALSA (Spatial ALgorithmic Language for StoryboArding) and ALVIS (ALgorithm VIsualization Storyboarder), a prototype end-user language and system firmly rooted in empirical studies in which students constructed and presented visualizations made out of simple art supplies. Our prototype end-user language and system pioneer a novel technique for programming of visualizations based on spatial relations, and a novel presentation interface that supports human discussions about algorithms by enabling reverse execution and dynamic mark-up and modification. Moreover, the prototype provides an ideal foundation for what we see as the algorithms classroom of the future: the interactive "algorithms studio."

Abstract:

Abstract: Computer science educators have traditionally used algorithm visualization (AV) software to create graphical representations of algorithms that are later used as visual aids in lectures, or as the basis for interactive labs. Typically, such visualizations are high fidelity in the sense that (a) they depict the target algorithm for arbitrary input, and (b) they tend to have the polished look of textbook figures. In contrast, low fidelity visualizations illustrate the target algorithm for a few, carefully chosen input data sets, and tend to have a sketched, unpolished appearance. Drawing on the findings of ethnographic studies we conducted in a junior-level algorithms course, we motivate the use of low fidelity AV technology as the basis for an alternative learning paradigm in which students construct and present their own visualizations. To explore the design space of low fidelity AV technology, we present a prototype language and system derived from empirical studies in which students constructed and presented visualizations made out of simple art supplies. Our prototype language and system pioneer a novel technique for programming visualizations based on spatial relations, and a novel presentation interface that supports reverse execution and dynamic mark-up and modification

Abstract: For over a century, studio-based instruction has served as an effective pedagogical model in architecture and fine arts education. Because of its design orientation, human-computer interaction (HCI) education is an excellent venue for studio-based instruction. In an HCI course, we have been exploring a studio-based learning activity called the prototype walkthrough, in which a student project team simulates its evolving user interface prototype while a student audience member acts as a test user. The audience is encouraged to ask questions and provide feedback. We have observed that prototype walkthroughs create excellent conditions for learning about user interface design. In order to better understand the educational value of the activity, we performed a content analysis of a video corpus of 16 prototype walkthroughs held in two undergraduate/graduate HCI courses. We found that the prototype walkthrough discussions were dominated by relevant design issues. Moreover, mirroring the justification behavior of the expert instructor, students justified over 80 percent of their design statements and critiques, with nearly one-quarter of those justifications having a theoretical or empirical basis. These results suggest that prototype walkthroughs can be useful not only in helping to teach HCI design, but also in helping to gauge students� evolving design knowledge

Abstract: The blind and visually impaired community is signicantly underrepresented in computer science. Students who wish to enter the discipline must overcome signicant technological and educational barriers to succeed. In an attempt to help this population, we are engaged in a three-year research project to build an educational infrastructure for blind and visually impaired middle and high school students. Our primary research goal is to begin forging a multi-sensory educational infrastructure for the blind across the United States. We present here two preliminary results from this research: 1) a new auditory programming environment called Sodbeans, a programming language called Hop, and a multi-sensory (sound and touch) curriculum, and 2) an empirical study of our rst summer workshop with the blind students. Results show that students reported a signicant increase in programming self-ecacy after participating in our camp.

Abstract: Given the increased importance of communication, teamwork, and critical thinking skills in the computing profession, we have been exploring studio-based instructional methods, in which students develop solutions and iteratively refine them through critical review by their peers and instructor. We have developed an adaptation of studio-based instruction for computing education called the pedagogical code review (PCR), which is modeled after the code inspection process used in the software industry. Unfortunately, PCRs are time-intensive, making them difficult to implement within a typical computing course. To address this issue, we have developed an online environment that allows PCRs to take place asynchronously outside of class. We conducted an empirical study that compared a CS 1 course with online PCRs against a CS 1 course with face-to-face PCRs. Our study had three key results: (a) in the course with face-to-face PCRs, student attitudes with respect to self-efficacy and peer learning were significantly higher; (b) in the course with face-to-face PCRs, students identified more substantive issues in their reviews; and (c) in the course with face-to-face PCRs, students were generally more positive about the value of PCRs. In light of our findings, we recommend specific ways online PCRs can be better designed.

Abstract: Development of autonomic chess-playing robots creates several interesting computer vision problems, including plane calibration and object recognition. Various solutions have been attempted, but most either require a modified chess set or place unreasonable constraints on board conditions and camera angles. A more general solution uses computer vision to automatically determine arbitrary chessboard location and identify chessmen on a standard, unmodified chess set. Although much work has been devoted to probabilistic image recognition in general, this paper presents a novel solution to the specific chessboard location problem that is accurate, less restrictive, and relatively time efficient.

Abstract: Advances in modern novice programming environments (NPEs) have made computer programming more accessible to an increasingly large audience. Unfortunately, 35 percent of the top 125 U.S. universities use C or C++, both of which are widely considered to be challenging for novices because they require one to have a firm understanding of computer memory. To make C and C++ more accessible to introductory programming students at these universities, we are developing TransparenC, a NPE that integrates two key advances in modern NPEs�direct manipulation programming and data visualization.

Abstract: The iterative design and evaluation of user interface prototypes is central to the user-centered design process. Designers potentially use a variety of prototyping tools and technologies, ranging from simple art supplies to custom prototyping languages and environments. While we know anecdotally that a range of prototyping technologies are used, there exists little empirical research into what technologies designers actually use in practice, and how and why those technologies are used. To explore these issues, we developed an on-line survey, which was completed by 87 user interface researchers and professionals who attended the CHI 2008 conference. Our results not only provide insight into the current state of user interface prototyping in practice; they also suggest preliminary requirements for the next generation of user interface prototyping software.

Abstract: Modern day computing jobs are increasingly emphasizing on soft skills such as design, collaboration and communication skills in addition to programming skills. Having students participate in peer reviews is one possible way to develop such soft skills. However, we have observed that low student participation and low review quality can impede the educational value of the peer review process. To address this issue and improve student learning in computer programming courses, we propose to implement and evaluate social comparisons� a method suggested by socio-psychological research.

Abstract: Inspired by the formal code inspection process commonly used in the software industry, we have been exploring the use of pedagogical code reviews (PCRs), in which a team of three to four students, led by a trained moderator, (a) walk through segments of each other's programming assignments, (b) check the code against a list of best coding practices, and (c) discuss and log issues that arise. We have found that PCRs not only improve the quality of students' code, but also positively impact students' communication and sense of community. However, implementing PCRs also presents a key logistical challenge: how to make code solutions and review results accessible to team members before, during, and after the team reviews? To address this challenge, we are developing an online environment specifically tailored to support PCRs. Our environment enables students to submit their code solutions; to review team members' code solutions on-line prior to PCRs; to carry out PCRs; and to resubmit their solutions based upon the results of the PCRs. In an empirical evaluation of the environment in a CS1 course, we found that it not only eased the logistics of implementing PCRs, but also improved the organization and efficiency of the PCR process.

Abstract: Given the increasing importance of communication, teamwork, and critical thinking skills in the computing profession, we believe there is good reason to provide students with increased opportunities to learn and practice those skills in undergraduate computing courses. Toward that end, we have been exploring studio-based instructional methods, which have been successfully employed in architecture and fine arts education for over a century. We have developed an adaptation of studio-based instruction for computing education called the pedagogical code review, which is modeled after the code inspection process used in the software industry. To evaluate its effectiveness, we carried out a quasi-experimental comparison of a "studio-based" CS 1 course with pedagogical code reviews and an identical "traditional" CS 1 course without pedagogical code reviews. We found no learning outcome differences between the two courses; however, we did observe two interesting attitudinal trends: (a) self-efficacy decreased more in the traditional course than in the studio-based course; and (b) peer learning decreased in the traditional course, but increased in the studio-based course. Additional questionnaire and interview data provide further evidence of the positive impact of studio-based instruction.

Abstract: Formal code inspections are employed by teams of professional software engineers to identify software defects and improve the quality of software. After reviewing a piece of code individually, members of an inspection team come together to log the issues they have found, and to find new ones. Within the scope of a multi-institutional research project to adapt, refine, and evaluate studio-based learning methods in computing education, we are developing an adaptation of the formal code inspection called the pedagogical code review for use in lower-division computer science courses. In a pedagogical code review, a group of three to four students, led by a trained moderator, (a) walk through segments of each other's programming assignments, (b) check the code against a list of best coding practices, and (c) discuss and log issues that arise. We implemented pedagogical code inspections in three lab sessions of a CS 1 course. Through an analysis of inspection logs and exit surveys, we compiled evidence that the reviews improved the quality of students' code, stimulated increasingly sophisticated discussions of programming issues and practices, and promoted a sense of community.

Abstract:

Abstract: Learning computer science is no longer simply a matter of learning computer programming. Indeed, modern day computing jobs demand design, communication and collaborative skills as well. In order to address this need and make computing education more engaging, motivating, and community-oriented, we have been exploring a �studio based� approach in which students (a) construct computational solutions to problems that have many possible solution strategies, and then (b) present their solutions to their peers and instructors for feedback and discussion. In the fall of 2007, we implemented this approach in a pre-CS1course by requiring students to present their solutions to five course programming projects in face-to-face review sessions. Interview and observational data collected in this course pointed out several practical and logistical problems surrounding the face-to-face review sessions that diminished their educational effectiveness. To address these problems, we describe the preliminary design of a novel asynchronous web-based project review system to support and augment face-to-face studio-based review sessions.

Abstract:

Abstract: While the demand for college graduates with computing skills continues to rise, such skills no longer equate to mere programming skills. Modern day computing jobs demand design, communication, and collaborative work skills as well. Since traditional instructional methods in computing education tend to focus on programming skills, we believe that a fundamental rethinking of computing education is in order. We are exploring a new "studio-based" pedagogy that actively engages undergraduate students in collaborative, design-oriented learning. Adapted from architectural education, the studio-based instructional model emphasizes learning activities in which students (a) construct personalized solutions to assigned computing problems, and (b) present solutions to their instructors and peers for feedback and discussion within the context of "design crits." We describe and motivate the studio-based approach, review previous efforts to apply it to computer science education, and propose an agenda for multi-institutional research into the design and impact of studio-based instructional models. We invite educators to participate in a community of research and practice to advance studio-based learning in computing education.

Abstract: Because they are easy to create and modify, low fidelity prototypes are commonly used in early evaluations of user interface designs. Designers typically use either pen-and-paper or various computer-based tools to create and test low fidelity prototypes. However, our informal analyses of these technologies indicate that they do not optimally support the two key, complementary tasks of (a) prototype creation and (b) wizard-of-oz testing. To address this problem, we have developed WOZ PRO, a pen-based software environment for the quick and easy creation and testing of low fidelity user interface prototypes. When designing interface screens in WOZ PRO, designers can easily (a) propagate a design change to other related screens, and (b) specify the set of screens that are reachable from a given screen. In a wizard-of-oz test, WOZ PRO aims to reduce the cognitive load on the wizard by allowing navigation only to those next screens that are valid. An experimental comparison of WOZ PRO and pen-and-paper provides evidence of WOZ PRO�s potential, and insight into how its design might be improved.

Abstract:

Abstract: Because they are easy to create and modify, low fidelity prototypes are commonly used in early evaluations of user interface designs. Designers typically use either pen-and-paper or various computer-based tools to create and test low fidelity prototypes; however, our informal analyses of these existing technologies indicate that they do not optimally support the two key, complementary tasks of (a) prototype creation and (b) wizard of oz testing. To address this problem, we have been developing WOZ Pro (Wizard of Oz Prototyper), a pen-based software environment for the quick and easy creation and testing of low fidelity user interface prototypes. We are designing WOZ Pro to be as easy to use as pen-and-paper, but to hold key advantages over pen-and-paper and existing computer-based tools. When designing interface screens in WOZ Pro, designers can easily (a) propagate a design change to other related screens, and (b) specify the set of screens that are reachable from a given screen. In a wizard of oz test, WOZ Pro reduces the cognitive load on the wizard by allowing navigation only to those next screens that are valid. We are planning a controlled experiment to compare WOZ Pro against paper-and-pencil along several measures in a set of prototype creation and evaluation tasks.

Abstract: Novices face many barriers when learning to program, including the need to learn both a new syntax and a model of computation. By constraining syntax and providing concrete visual representations on which to operate, direct manipulation programming environments can potentially lower these barriers. However, what if the learning goal of the novice is to be able ultimately to program in conventional textual languages, as is the case for introductory computer science students? Can direct manipulation programming environments lower the barriers to programming, and, at the same time, promote positive transfer to textual programming? To address this question, we designed a new direct manipulation programming interface for ALVIS Live!, a novice programming environment. We then conducted an experimental study that compared the programming outcomes promoted by the new direct manipulation interface to those promoted by ALVIS Live!'s textual programming interface. We found that the direct manipulation interface not only led to significantly better initial programming outcomes, but also to significant positive transfer to the textual interface. Our results show that direct manipulation interfaces can provide novices with a "way in� to traditional textual programming.

Abstract: Empirical studies of novice programming typically rely on code solutions or test responses as the basis of their analyses. While such data can provide insight into novice programming knowledge, they say little about the programming processes in which novices engage. For those interested in improving novice programming environments, a key research question arises: How can we collect and analyze data on novice programming that will enable us (a) to analyze and compare the programming processes promoted by alternative novice programming environments, and (b) ultimately to build better novice programming environments? To address this question, we have collected a large video corpus of novices as they construct code solutions in various versions of ALVIS Live! [17], a novice programming environment. Through detailed post-hoc analyses of our video corpus, we have developed a methodology for compiling the moment-by-moment evolution of novice code solutions. Based on an analysis of a model code solution's key semantic components, our methodology enables researchers to document, on a second-by-second basis, (a) what part of a code solution a programmer is focusing on, and (b) where the semantic feedback provided by the programming environment is helping. Although it is time and labor intensive, our methodology provides researchers with a standard set of data and representations for comparing the programming processes promoted by alternative programming environments.

Abstract: Since the late 1990s, we have been developing ALVIS, a new breed of algorithm visualization software that supports a novel, "studio based" approach to teaching introductory programming. In this approach, students, working in pairs, use the ALVIS software to construct their own algorithmic solutions to appropriate algorithm design problems. ALVIS automatically generates accompanying algorithm visualizations, which students can augment with their own visual and storytelling elements prior to presenting them to their peers and instructor for feedback and discussion. We believe the pedagogical value of this approach rests in students' active engagement in constructing and subsequently discussing their algorithmic solutions. Previous versions of the ALVIS software have enabled students to create only single-procedure, array iterative algorithms. We describe here our preliminary work toward expanding ALVIS so that it supports parameterized procedures and pointers. Our ultimate aim is to explore the pedagogical value of using ALVIS as the basis of an entire "studio based" introductory programming course for students without prior programming experience.

Abstract: Pedagogical algorithm visualization technology aims to assist learners in understanding the dynamic behavior of computer algorithms. A key trend in past experimental studies is that learners benefit most when they are actively engaged with algorithm visualization technology. Inspired by this trend, we are exploring the pedagogical value of a novel active learning activity-the Studio Experience-within the context of an introductory CS1 unit on algorithmic problem-solving. In a Studio Experience, student pairs are given algorithm design problems, e.g., "design two alternative algorithms that reverse the values in a list." They are tasked both with constructing algorithmic solutions and accompanying visualizations, and with presenting their visualizations for feedback and discussion in a session modeled after an architectural "design crit." Through an observational study of studio experience sessions in which students used two alternative forms of visualization technology-art supplies and a computer-based tool-we gained insight into (a) the processes by which students construct visual presentations of algorithms, (b) the characteristics of their visual presentations; (c) the nature of conversations mediated by visual algorithmic solutions; and (d) the kind of visualization technology that best supports these activities. Based on our results, we suggest improvements to the approach, and propose an agenda for future empirical studies.

Abstract: Pedagogical algorithm visualization systems produce graphical representations that aim to assist learners in understanding the dynamic behavior of computer algorithms. In order to foster active learning, educators have explored algorithm visualization systems that empower learners to construct their own visualizations of algorithms under study. Notably, these systems support a similar development model in which coding the algorithm is temporally distinct from viewing and interacting with the resulting visualization. To explore the benefits of narrowing the gap between coding an algorithm and viewing its visualization, we have implemented "What You See Is What You Code," a novel, "radically dynamic" development model to facilitate learner-constructed algorithm visualizations. In this model, the line of algorithm code currently being edited is reevaluated on every edit, leading to the dynamic update of an accompanying visualization of the algorithm. Analysis of usability studies involving introductory computer science students suggests that the immediacy of the modelýs feedback can help novices to quickly identify and correct programming errors, and ultimately to understand their codeýs execution.

Abstract: Algorithm visualizations graphically illustrate how algorithms work. In prior ethnographic studies of a computer science course in which students were required to construct and present their own algorithm visualizations, we observed that visualizations based on storylines tended to stimulate increased audience interest and involvement. This observation, coupled with the empirical research that substantiates the value of stories as mnemonic devices, raises an interesting research question: Do visualizations with storylines actually help students remember the procedural behavior of an algorithm better than visualizations that do not involve storylines? To investigate this question, we conducted an experimental study that compared the memorability of algorithm descriptions involving differing degrees of spatial and verbal embellishment. The study failed to detect significant differences. We reflect on our lack of significant results, and suggest two alternative paths for future research into the value of story-based algorithm visualization.

Abstract: The ALgorithm VIsualization Storyboarder (ALVIS) enables beginning and intermediate computer science students both to construct "low fidelity" (rough, unpolished) visualizations of the algorithms under study, and to present those visualizations to their peers and instructor for feedback and discussion. The original ALVIS software was a frail research prototype. We are using an iterative, user-centered design process to develop a public domain version of the software suitable for use in a "studio-based" computer science course. Through a series of empirical studies of both low and high fidelity prototypes of ALVIS, we have reevaluated the original design of ALVIS, and are presently in the process of making three key designchanges: (a) the underlying scripting language is being redesigned for simplicity and ease of learning; (b) the interface for creating and placing graphical variables and spatial structures is being simplified and streamlined for ease of use; and (c) new features are being added that explicitly support storytelling.

Abstract: The paper reports an empirical study comparing the role of discourse and knowledge representations in face to face versus distance collaborative learning. The study reported in this paper compares Proximal (face to face) with Distal (synchronous collaboration via networked software) conditions. Two competing hypotheses were evaluated: (1) The influence of representations in the Distal condition could be weaker because of the lack ofimplicit "taken as shared" that results from working together in front of a physically shared display, and because of the greater difficulty of utilizing the representations as a resource for conversation through gesturing. (2) The influence of representations in the Distal study could be stronger because participants may rely more on them for their communication in the absence of face-to-face communication. The quantitative data supported the second hypothesis, while other observations supported the first hypothesis. A synthesis is offered.

Abstract: One of the reputed advantages of end-user programming languages is that they support a given problem domain with a set of programming abstractions that are �just right � for the end-users who need to program within that domain. Green and Petre�s [1] Cognitive Dimensions Framework accounts for this advantage in terms of the �closeness of mapping � dimension: the closer the programming domain is to the problem domain, the easier the programming task. This suggests that programming might be conceptualized as a process of translation, with �close � translations being more efficient and error-free than �distant � ones. While there appears to be a strong intuitive basis for this view, there presently exists little direct empirical evidence. To that end, we present an experimental framework for systematically exploring the �closeness of mapping � dimension within the programming domain of college-level textbook algorithms and data structures. A pilot study we conducted within that framework provides preliminary evidence in support of one of our hypotheses. 1.

Abstract:

Abstract: This paper reports an empirical study comparing the roles of representations in supporting face to face versus distance collaborative learning. Findings indicate that while face to face collaborators primarily use knowledge representations to summarize verbal reasoning, distributed collaborators also use knowledge representations to propose ideas before they arediscussed, particularly ideas supported by the categorical affordances of the knowledge representation medium. The work suggests that designers of online learning environments should support fluid crosstalk between multiple representations, because online discussions will be distributed across all available representations

Abstract: The "Algorithms Studio" project explores a novel, studio-based approach to teaching an undergraduate course on computer algorithms. Inspired by the design studio commonly used in architecture education, the approach emphasizes conceptual design activities in whichstudents use sketch-based algorithm visualization technology to create their own visual representations of the algorithms under study. They then discuss their representations with their peers and instructor within the context of regularly scheduled critique sessions. A diverse program of planned empirical studies will investigate the value and role of visualization technology in learning algorithms at the cognitive, social, and cultural levels.

Abstract: This interactive session brings together researchers and educators interested in using external representations to facilitate and assess learning. The session will juxtapose four systems, each of which takes a different design approach. The representations include concept maps, metaphorical textual descriptions or visualizations for helping students learn in complex domains such as science or programming.

Abstract: In order to better understand how software design choices may influence students' collaborative learning, we conducted a study of the influence of tools for constructing representations of evidential models on collaborative learning processes and outcomes. Pairs of participants worked with one of three representations (matrix, graph, text) while investigating a complex public health problem. Focusing on students' collaborative investigative processes and post-hoc essays, we present several analyses that assess the impact of representation type on students' elaborations of their emerging knowledge. Our analyses indicate significant impacts on the extent to which students revisit knowledge and the likelihood that they will use that knowledge later.

Abstract: Visualization technology can be used to graphically illustrate various concepts in computer science. We argue that such technology, no matter how well it is designed, is of little educational value unless it engages learners in an active learning activity. Drawing on a review of experimental studies of visualization effectiveness, we motivate this position against the backdrop of current attitudes and best practices with respect to visualization use. We suggest a new taxonomy of learner engagement with visualization technology. Grounded in Bloom's well-recognized taxonomy of understanding, we suggest metrics for assessing the learning outcomes to which such engagement may lead. Based on these taxonomies of engagement and effectiveness metrics, we present a framework for experimental studies of visualization effectiveness. Interested computer science educators are invited to collaborate with us by carrying out studies within this framework.

Abstract:

Abstract: In exploring how to make programming easier for non-programmers,research into end-user environments hastraditionally been concerned with designing betterhuman-computer interaction. That traditional focus hasleft open the question of how end-user environmentsmight support human-human interaction. Especially insituations in which end-user environments are enlisted tofacilitate learning, we hypothesize that a key benefit maybe their ability to mediate conversations about a domainof interest. In what ways might end-user environmentssupport human communication, and what design featuresmake them well-suited to do so? Drawing onethnographic studies of an undergraduate algorithmscourse in which students constructed and presentedalgorithm visualizations, we develop a provisionalframework of six communicative dimensions of end-userenvironments: programming salience, typeset fidelity,story content, modifiability, controllability, andreferencability. To illustrate the design implications ofthese dimensions, we juxtapose conventional algorithmvisualization technology with a prototype end-userenvironment specifically designed to facilitatecommunication about algorithms. By characterizing thoseaspects of end-user environments that impact socialinteraction, our framework provides an importantextension to Green and Petre's [1] cognitive dimensions.

Abstract: Given the explosive growth in the use of computer media for learning and the wide range of choices available to designers of online learning tools, it is crucial to understand how these design choices may influence learning. This study evaluated the influence of tools for constructing representations of evidential models on collaborative learning processes and outcomes. Pairs of participants worked with one of three representations while investigating complex science and public health problems. Dependent variables included quantity of discourse about evidential relations (�for� and �against�) and two learning outcome measures. Significant effects of tools on learning processes were found, although there appears to have been insufficient time for these process differences to influence learning outcomes.

Abstract: Algorithm visualization software graphically illustrates how computer algorithms work. Past experiments designed to substantiate the software's pedagogical value have yielded mixed results. A review of these studies suggests that the more actively involved learners are in the visualization process, the better they perform. Given this trend, and inspired by ethnographic fieldwork we conducted in an undergraduate algorithms course, we hypothesize that students who use simple art supplies to construct their own visualizations will learn an algorithm better than students who interact with computer-based visualizations constructed by an expert. We conducted an experiment to test this hypothesis, and found no significant differences between the two pedagogical approaches. Thus, students who use low tech materials to construct their own visualizations may learn algorithms just as well as students who study conventional high tech visualizations constructed by an expert. This result motivates a markedly different kind of algorithm visualization software: one that enables learners to construct their own low tech visualizations.

Abstract: Computer science educators have traditionally used algorithm visualization (AV) software to create graphical representations of algorithms that are later used as visual aids in lectures, or as the basis for interactive labs. Based on ethnographic field studies we have conducted in an undergraduate algorithm course, we advocate an alternative teaching approach in which students use simple art supplies to construct and present their own visualizations to their peers and instructor for feedback and discussion. To support this approach, we have built SALSA and ALVIS, a prototype language and system that enable students to (a) quickly construct rough, unpolished (low fidelity) visualizations in much the same way they would do so with simple art supplies, and (b) interactively present those visualizations to an audience. Our prototype pioneers a novel technique for programming visualizations based on spatial relations, and a novel presentation interface that supports reverse execution and dynamic mark-up and modification.

Abstract: Traditional algorithm visualization software supports the creation of "high fidelity" visualizations, which depict the target algorithm for arbitrary input, and have the polished look of textbook figures. Drawing on the findings of ethnographic studies we conducted in an undergraduate algorithms course, we have developed SALSA/ALVIS, a markedly different kind of algorithm visualization software that enables students to construct and present their own "low fidelity" visualizations. Unlike "high fidelity" visualizations, "low fidelity" visualizations depict the target algorithm for a few, carefully-selected input data sets, and have an unpolished look. In addition, they can be viewed both forwards and backwards, and dynamically marked up and modified, making them well-suited for mediating student-instructor discussions about algorithms.

Abstract: Despite their intuitive appeal, computer-based algorithm visualization (AV) artifacts have failed to enter mainstream computer science education. I argue that past research into the design, evaluation, and pedagogical use of AV artifacts has been guided by an underlying theory of effectiveness that is fundamentally deficient. Inspired by an alternative pedagogy in which students construct their own AVs, and by recent research into the situated nature of communication and learning, my dissertation develops an alternative theory that stresses the value of AV artifacts both in facilitating students' participation in the Community of Algorithmaticians, and in providing students and instructors with resources for negotiating a shared understanding of algorithms. KEYWORDS: Algorithm visualization, Situated Action Theory, Situated Learning Theory PROBLEMS Computer algorithms are notoriously difficult to learn. In an effort to enhance algorithms curricula, computer science educators have explored the potential for computer-based algorithm visualization (AV) artifacts, which enable instructors and students to create and explore graphical representations (AVs) of algorithms under study.

Abstract: Many educators have used Algorithm Visualization (AV) to teach students of computer science about how computer algorithms work. Our study sheds light on two important questions: (a) How do people conceptualize algorithm animations in the first place; and (b) To what extent do such visualizations accord with AV software. In the first half of this study, pairs of graduate students in computer science were asked to construct animations for a simple sort (bubble sort) using ordinary art materials. In the second half, they implemented a bubble sort visualization using an interactive AV program called LENS [1], which allows one to construct and view an animation of any C program. The way in which pairs visualized the same sort differed tremendously from each other and did not accord completely with the animation language provided by LENS. This paper analyzes those differences by a detailed examination of the semantics of the human visualizations, the algorithm code, and the LENS AV language.

Abstract: Underlying any single-user software visualization (SV) system is a visualization language onto which its users must map the computations they would like to visualize with the system. We hypothesize that the usability of such systems turns on their ability to provide an underlying visualization language that accords with the ways in which their users conceptualize the computations to be visualized. To explore the question of how to design visualization languages grounded in human conceptualization, we present an empirical study that made use of a research method called visualization storyboarding to investigate the human conceptualization of the bubblesort algorithm. Using an analytical framework based on entities, attributes, and transformations, we derive a semantic-level visualization language for bubblesort, in terms of which all visualizations observed in our study can be expressed. Our empirically-based visualization language provides a means for predicting the usability of the visualization language defined by Lens (Mukherjea and Stasko, 1994), a prototypical single-user SV system. We draw from a follow-up usability study of Lens to substantiate our predictions.

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract: Nearly every community of practice develops its own representations to support the practices of the community. For example, the community of electric circuit designers has developed circuit diagrams to represent electric circuits. Members of that community rely on such diagrams to discuss, communicate, and document what they do. Past research has established that members of a given community not only tend to interpret the community's representations in a similar way, but also tend to construct similar representations for a given problem or concept. I argue that the way in which an individual interprets and constructs the representations of a given community serves as a useful gauge of the individual's membership in the community. Drawing on cultural consensus theory, a framework developed within cognitive anthropology, I introduce an empirical method that uses the way in which individuals read and construct representations as a basis for substantiating the existence of a community, and as a basis for assessing an individual's level of membership in a community. To illustrate how the method can be applied, I present an example drawn from my own research into the use of graphical representations of algorithms (�algorithm visualizations�) in computer science education. If one accepts the social constructivist premise that learning amounts to becoming a fuller member of a community of practice, then the method I introduce constitutes a richer and more valid means of measuring learning than traditional knowledge testing.

Abstract:

Abstract: Inspired by the design studio commonly used in architecture education, this project will explore a "studio-based" approach to teaching an undergraduate course on human-computer interaction design. In addition to a weekly lecture, the course will revolve around a weekly, 150-minute "design studio" that emphasizes conceptual user interface design activities. Using wireless HP tablet PCs, pairs of students will interact with custom, sketch-based software to construct low fidelity design prototypes that meet the requirements of given design problems. Each design problem will (a) focus on the development of an interface to a specific electronic device (e.g., a VCR) or software system and (b) require students to apply the design processes and principles currently being explored in class. Via a wireless network, student pairs will hand in their design prototypes to the instructor. In "design crits" held at the end of each studio, select pairs of students will use the instructor's presentation equipment to present their designs to the rest of the studio for feedback and discussion. In addition, via the wireless network, students will post their designs to a web-based environment that supports artifact-centered threaded discussions, so that students can view and comment upon each other's designs outside of class.

Abstract: The overriding goal of software visualization (SV) technology is to be effective�that is, to provide functionality that people actually want to use (usefulness), and to provide non-problematic access to that functionality (usability). Nonetheless, the eight extant taxonomies of SV focus almost exclusively on technology expressiveness: the kinds of visualizations that can be produced with a system, as well as the methods prescribed by the system for generating and interacting with those visualizations. However, while expressiveness is necessary for effectiveness, it certainly is not sufficient. To enhance the navigational markings of the expressiveness-laden map charted by past SV taxonomies, this meta-study critically assesses the state-of-the-art with respect to SV effectiveness. The meta-study�s focus on effectiveness implies the need to abandon both the SV system as its unit of analysis, and the SV systems literature as its principal data. Accordingly, SV is reconceptualized as a collection of human tasks; the empirical studies that have considered those tasks form the study�s data. The meta-study�s data differ along three key dimensions: their theories of effectiveness, their research techniques, and their research foci. Different choices along any of these dimensions lead to accounts of effectiveness that differ markedly�both qualitatively and quantitatively. To draw out the differences to which alternative choices lead, the meta-study pursues in-depth analyses of divisions of labor (indicating theories of effectiveness); units of analysis, data collection/analysis methods, and desired results (indicating research techniques); and research questions, SV artifacts, target programs, participants, and SV tasks (research foci). In light of the SV effectiveness studies� overriding interest in pertaining to SV in practice, a provisional analysis of their ecological validity is additionally offered. A synthesis of the analysis both proposes theories of effectiveness, research techniques, and research foci that appear ripe for future research, and recommends that future research be firmly grounded in ethnographic descriptions of SV in practice.

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract: Algorithm visualization (AV) systems, which provide graphic depictions of the dynamic state of an executing algorithm, can prove invaluable to anyone interested in gaining insight into the dynamic behavior of computer programs. Although past researchers have taken a multitude of different approaches to designing and implementing such systems (cf. Myers 1984, Brown 1988, Stasko 1990, Naps & Hundhausen 1991, Tuchman, Jablonowski, & Cybenko 1991, Mukherjea & Stasko 1993), they have all had to address, in some way, the same four issues underlying AV system design: (1) portability, (2) time of visualization, (3) instrumentation, and (4) program object-to-graphics mapping. Targeting the specific domain of breakpointbased program monitoring and visualization�a style of AV in which one can set breakpoints and examine the graphical representation of program�s objects (variables, data structures, and other abstractions of program state) of interest at those breakpoints�this paper both identifies the requirements of a targetindependent AV system architecture, and presents ObjectView, an architectural framework that we have developed to meet those requirements. Through its use of a distributed model, the object-oriented programming paradigm, and a portable, device-independent graphics language, ObjectView defines a framework for AV design that supports (1) a natural and non-intrusive method for identifying the program objects of interest, and for preparing those objects to be visualized; (2) a device- and program-independent technique for defining mappings between program objects and the graphics displays to which they should give rise; and (3) a run time source-level visualization environment for controlling the execution of a program, and for viewing, on demand, dynamic graphical representations of program objects.

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract:

Abstract: