Agile Research Studios: Orchestrating Communities of Practice to Advance Research Training at ScaleCSCW 2017
Agile Research Studios (ARS) is a socio-technical system that greatly expands research training opportunities by supporting research communities of practice without increasing faculty mentoring resources.
Studying the Effects of Task Notification Policies on Participation and Outcomes in On-the-go CrowdsourcingHCOMP 2016
Telescope: Fine-Tuned Discovery of Interactive Web UI Feature ImplementationUIST 2016
Telescope traces public website runtimes to create interactive low-barrier learning material for developers. It features composite code views and visual links to help developers overcome barriers in unfamiliar code.
Habitsourcing: Sensing the Environment through Immersive, Habit-Building ExperiencesUIST 2016
We introduce habitsourcing, which harnesses habit-building practices to collect environmental data. We present interaction techniques for acquiring sensing data, and ExperienceKit, an iOS library that helps build habitsourcing applications.
Designing for Inclusion: Supporting Gender Diversity in Independent Innovation TeamsGROUP 2016
We study how independent innovation teams develop an environment of inclusion to support gender diversity.
Designing crowdcritique systems for formative feedback.International Journal of Artificial Intelligence in Education.
Describes 3 design-based implementations, creating a hybrid crowdsourcing systems that combine the intelligence of multiple novices in face-to-face settings might provide an alternate approach for providing intelligent formative feedback.
Enabling Physical Crowdsourcing On-the-Go with Context-Sensitive NotificationsHCOMP Poster 2015
This paper introduces the idea of presenting physical tasks into people’s existing routine so that people can contribute small tasks on-the-go with minimal effort. We present two on-the-go crowdsourcing systems: Libero for package delivery, and CrowdFound for finding lost items.
Using Anonymity and Communal Efforts to Improve Quality of Crowdsourced FeedbackHCOMP 2015
An experiment to test the effect of online anonymity on feedback quality and quantity. We also test a communal feedback method—distributing between teams feedback providers from the class’s collective social networks.
Unravel: Rapid Web Application Reverse Engineering via Interaction Recording, Source Tracing, and Library DetectionUIST 2015
CrowdFound: A Mobile Crowdsourcing System to Find Lost Items On-the-GoCHI Extended Abstract 2015
We present CrowdFound, a mobile crowdsourcing system to find lost items. CrowdFound allows users to input lost item descriptions on a map and then sends notifications to users passing near tagged areas. To assess the system’s efficacy, we conducted interviews and user testing on CrowdFound.
GAZE: Using Mobile Devices to Promote Discovery and Data CollectionCHI SRC 2015
We developed Gaze, an approach for people to gather information about physical environments while on the go. Gaze uses mobile devices to promote data collection through low-effort interactions.
Libero: On-the-go Crowdsourcing for Package DeliveryCHI SRC 2015
We introduce a system called Libero, which utilizes people’s existing routine for package delivery by incorporating just-in-time notifications to reduce task distance and having a community support itself.
Remote Paper Prototype Testing CHI 2015
We develop a method and tool that supports remote wizard-of-oz prototype testing; allows mobile app designers to test paper prototypes in realistic scenarios.
Crowdfunding Science: Sharing Research with an Extended AudienceCSCW 2015
A qualitative study of scientists using crowdfunding, motivations for use, and struggle they face. We offer guidelines for the future design of interfaces.
Building Support Tools to Connect Novice Designers with Professional CoachesCreativity and Cognition
Creativity support tools help learners undertake creative work, such as facilitating coaching by creative professionals. How might we design creativity support tools that in-crease learners’ access to coaching by creative professionals?
CrowdCheer: Situational Crowdsourcing of Motivation for RunnersGrace Hopper SRC 2015
We studied the effects of crowdsourced and targeted motivation for runners in an attempt to understand how to use existing crowds to deliver targeted encouragement and thus improve runners’ performances.
Frenzy: Collaborative Data Organization for Creating Conference SessionsCHI 2014
best paper honorable mention
We present a collaborative web application called Frenzy to draw on the efforts and knowledge of an entire program committee to collectively categorize accepted papers and create sessions.
Learning to Fail: Experiencing Public Failure Online Through CrowdfundingCHI 2014
A qualitative and quantitative study of failed projects on Kicsktarter. We look at which projects relaunch, what project creators change upon relaunch and what project creatorsl learn through the process of failing.
Understanding the Role of Community in Crowdfunding WorkCSCW 2014
best paper honorable mention
A qualitative study to understand community efforts to support crowdfunding work, such as providing mentorship to novices and giving feedback on campaign presentation.
Pair Research: Matching People for Collaboration, Learning, and ProductivityCSCW 2014
To increase productivity, informal learning, and collaborations within and across research groups, we have been experimenting with a new kind of interaction that we call pair research.
Computer Supported Collective Actioninteractions 2014
We investigate the design challenges of computer supported collective action (CSCA) systems that connects crowds and communities of participants, lowers the cost of communication and deliberation, and helps to coordinate action, thereby enabling new forms of collaboration unimaginable a decade ago.
Crowdfunding: Motivations and Deterrents for ParticipationTOCHI 2014
A grounded theory of why people are motivated and deterred to participate in crowdfunding. We present guidelines for the design of interfaces for crowdfunding to enhance performance and engagement.
WeDo: End-To-End Computer Supported Collective Action.ICWSM
We introduce WeDo, a system aimed at promoting simple forms of participatory, end-to-end collective action. Pilot deployments of WeDo illustrate that sociotechnical systems can support automated transitions through different phases of end-to-end collective action, but that challenges, such as the elicitation of leadership and the accommodation of existing group norms, remain.
Computer supported novice group critiqueDIS 2014
Groups of novice critiquers can sometimes provide feedback that is as good as a single expert. Unfortunately, we do not know how to create such systems for novice group critique in design education. The study provides improved principles for developing computer-supported novice group critique systems in design.
Design-Based research process: Problems, phases, and applicationsICLS 2014
Since the first descriptions of design-based research (DBR), there have been continued calls to better define DBR and increase its rigor. Here we address four uncertainties about DBR: (a) the phases of the DBR process, (b) what distinguishes DBR from other forms of research, (c) what distinguishes DBR from design, and (d) the characteristics of DBR that make it effective for answering certain types of questions. By better articulating the process of DBR, this definition helps us to better craft, improve, communicate, and teach design-based research.
Community clustering: Leveraging an academic crowd to form coherent conference sessionsHCOMP 2013
To help conference organizers create sessions, we tap into the interest and expertise of committee members and authors. At CHI 2013 we evaluate alternative approaches and collect 10,000+ affinity pairs from community members.
Cobi: A Community-Informed Conference Scheduling ToolUIST 2013
Cobi engages an entire academic community in planning a large-scale conference. The mixed-initiative scheduling tool helped CHI organizers resolve 100+ newly surfaced scheduling conflicts in just hours.
Game penalties decrease learning and interestAIED 2013
Educational game designers can improve learning and interest by embedding cognitive tutors in games and removing penalties such as loss of progress.
Formative feedback in digital lofts: Learning environments for real world innovationAIED workshop, 2013
Digital Lofts for teaching design process (in engineering, science in law) can support learning through 3 interelated feedback loops: (a) a crowd critique loop in which learners receive formative feedback on their innovation work from a broader community, (b) a case development loop in which examples of student work are semi-automatically created to provide instruction, and (c) a learner-driven instructional loop, in which self-assessments determine which group instruction is provided.
Automated Workflow SynthesisAAAI 2013
We introduce a method aimed at ideally harnessing human efforts in solving a problem by learning about a crowd’s performance on tasks and synthesizing an optimal workflow for solving a problem.
Cobi: Communitysourcing Large-Scale Conference SchedulingCHI Interactivity 2013
Cobi presents an alternative approach to conference scheduling that engages the entire community to take active roles in the planning process. This paper provides an overview of the Cobi system.
A Pilot Study of Using Crowds in the ClassroomCHI 2013
We explore if and how online crowds can support student learning in the classroom and how scalable, diverse, immediate (and often ambiguous and conflicting) input from online crowds affects student learning and motivation for project-based innovation work. We share early evidence and discuss implications for creating a socio-technical infrastructure to more effectively use crowdsourcing in education.
The Future of Crowd WorkCSCW 2013
Drawing on theory from organizational behavior and distributed computing, as well as direct feedback from workers, we outline a framework that will enable crowd work that is complex, collaborative, and sustainable. The framework lays out research challenges in twelve major areas: workflow, task assignment, hierarchy, real-time response, synchronous collaboration, quality control, crowds guiding AIs, AIs guiding crowds, platforms, job design, reputation, and motivation.
Perspective taking, political ideologies and digital gamesInternational symposium on digital ethics, 2013
Students struggle to predict whether different ideologies support or oppose policy interventions. Some ideologies are more difficult to reason about than others. Experience in domains such as political science do not seem to be correlated with improved reasoning ability.
Affect and Creative Performance on Crowdsourcing PlatformsACII 2013
We examine two affective techniques to boost creativity on crowdsourcing platforms – affective priming and affective pre-screening. We present guidelines for the design of human-computer interaction scenarios that involve affect and creative performance.
TurkServer: Enabling Synchronous and Longitudinal Online ExperimentsHCOMP 2012
We introduce TurkServer, a platform supporting synchronous and longitudinal experiments involving real-time interaction betwen participants on Amazon Mechanical Turk.
Hallucination: a Mixed-Initiative Approach for Efficient Document ReconstructionHCOMP 2012
We introduce a mixed-initiative approach for document reconstruction that can significantly reduce the amount of time and effort required to reassemble a document from shredded pieces or an artifact from broken fragments.
Task Routing for Prediction TasksAAMAS 2012
We propose methods for routing a prediction task on a network where each participant can contribute information and route the task onwards. We introduce routing scoring rules that induce effective solving and routing behavior in equilibrium.
Human Computation Tasks with Global ConstraintsCHI 2012
best paper honorable mention
Mobi is a system for crowdsourcing itinerary planning. We introduce and illustrate the crowdware design pattern for tackling complex tasks with global constraints by using a shared, collaborative workspace.
Economics of BitTorrent CommunitiesWWW 2012
We report on a half-year-long measurement study of DIME – a community for sharing live concert recordings – that sheds light on the economic forces affecting users in private fire-sharing communities.
Supporting Creativity in Web-Based ServicesInternational Journal of Design 2012
This paper encourages service designers to view the action of customization as a creative process and provides nine theoretically grounded design principles to support customer creativity within Web-based customization self-services.
Extracurricular Design-Based Learning: Preparing Students for Careers in Innovation, International Journal of Engineering EducationInternational Journal of Engineering Education 2012
The Psychological Experience of PrototypingDesign Studies 2012
How Managers Use Multiple Media: Discrepant Events, Power, and Timing in Redundant CommunicationOrganization Science 2012
Priming for Better Performance in Micro-Task Crowdsourcing EnvironmentsIEEE Internet Computing 2012
Crowdsourcing General ComputationMSR Technical Report 2011
We present a direction of research on principles and methods that can enable general problem solving via human computation systems.
An Iterative Dual Pathway Structure for Speech-to-Text TranscriptionHCOMP 2011
We develop a human computation algorithm for speech-to-text transcription. We introduce an iterative dual pathway structure that eliminates the need to check produced transcripts in a separate process.
PlateMate: Crowdsourcing Nutrition Analysis from Food PhotographsUIST 2011
PlateMate allows users to take photos of their meals and receive estimates of food intake and composition from the crowd. PlateMate is nearly as accurate as a trained dietitian and easier to use than traditional self-reporting.
Using tutors to improve educational gamesAIED 2011
Cognitive games that embed tutors in educational games increase both learning and interest.
Towards Large-Scale Collaborative Planning: Answering High-Level Search Queries Using Human ComputationAAAI 2011
Behind every search query is a high-level mission that the user wants to accomplish. We introduce a human computation system that generates simple plans containing diverse, actionable search results and a roadmap for accomplishing the mission.
Incentive Design for Adaptive AgentsAAMAS 2011
To explore principles for designing incentives to influence the behavior of learning agents, we consider a setting in which a principle seeks to induce an adaptive agent to select a target action by providing incentives.
Toward Automatic Task Design: A Progress ReportHCOMP 2010
We provide a method for automating the design of a task to induce good output from crowd workers. We construct and learn models for predicting the rate and quality of work and use them to optimize designs.
An Intelligent Debater for Teaching ArgumentationITS 2010
people’s choice for best young researchers track poster
Toward a framework for the analysis and design of educational gamesDIGITEL 2010
The EDGE framework assists educational game design by helping designers (a) align goals, assessment and instruction, (b) considering mechanics, dynamics and aesthetics, and (c) tightlying learning and gameplay.
Learning to Waste – Wasting to Learn? How to Use Cradle to Cradle Principles to Improve the Teaching of DesignInternational Journal of Engineering Education 2010
Engineers of the future expected to be knowledgeable about the principles and practices needed to develop eco-effective products and manufacturing processes, yet it is challenging to integrate these principles and practices into undergraduate engineering design education. We present and evaluate an approach that exposing students to cradle to cradle design and help them make progress toward developing adaptive expertise in this area.
A General Approach to Environment Design with One AgentIJCAI 2009
Environment design is the problem of influencing an agent’s decisions by making changes to the agent’s environment. We present a general framework for the formulation and solution of environment design problems with one agent.
Constructing causal diagrams to learn deliberationIJAIED 2009
We present a cognitive framework for policy argumentation and show that causal diagram improve reasoning. Having a correct diagram improves reasoning and so does practice constructing a diagram (in cases where correct diagrams are unavailable).
Will Google destroy western democracy? Bias in policy problem solvingAIED 2009
Confirmation bias in analyzing policy evidence may not arise because of biased search or biased evaluation, but because changes in beliefs after reading contradictory evidence have a very small effect on overall beliefs and because people recall the mass of evidence as supporting their current beliefs. This suggests that a better way to alievate confirmation bias might be to provide external representations of evidence.
Policy Teaching Through Reward Function Learning EC 2009
We study the learning and optimization problem of using rewards to induce an agent to follow a particular policy. We prove that the problem is tractable and that effective rewards can be discovered after few interactions.
Using Improvisation To Enhance the Effectiveness of BrainstormingCHI 2009
We explore how designers use techniques from theatrical improvisation to adhere to the rules of brainstorming thereby enhancing group interactions while collaborating. The usefulness of improvisation for brainstorming stems from the similarity of the goals of improvisation and brainstorming, the similarity of the recurrent problems that actors and designers encounter when collaborating, and the distinctness of the ways each have devised to resolve the problems that block the group’s performance.
Design requirements of argument mapping software for teaching deliberationOnline deliberation: Design, research, and practice 2009
For argument mapping software to be usuable in education they must: (a) provide appropriate visual represntations, (b) allow flexible consruction, (c) support visualogic, (d) automate nonsemantic operations, (e) allow presentation of multiple diagrams, and (f) allow cross platform campatibility.
Strong Activity Rules for Iterative Combinatorial AuctionsComputers & Operations Research 2009
We propose strong activity rules that allow for simple, consistent bidding strategies in complex, iterative auctions, for example for spectrum or landing slots at congested airports.
Value-based Policy Teaching with Active Indirect ElicitationAAAI 2008
We study the learning and optimization problem of using rewards to induce an agent making sequential decisions to follow a policy that maximizes the total expected value of an interested party.
‘Tis better to construct than to receive? The effects of diagram tools on causal reasoningAIED 2007
We show that causal diagrams improve policy reasoning. Having a correct diagram improves reasoning and so does practice constructing a diagram (in cases where correct diagrams are unavailable).
The logic of Babel: Causal reasoning from conflicting sourcesAIED Workshop, 2007
Improvisation Principles and Techniques for DesignCHI 2007
The paper argues that improvisation can build perspectives and skills that are critical for designers, such as creative collaboration, fostering innovation, supporting spontaneity, learning through error, and presenting ideas. The applications are analyzed to demonstrate individual and group level outcomes in design work.