Start Submission Become a Reviewer

Reading: Conjectures, Cycles and Contexts: A Systematic Review of Design-based Research in Engineerin...

Download

A- A+
Alt. Display

Literature Reviews

Conjectures, Cycles and Contexts: A Systematic Review of Design-based Research in Engineering Education

Authors:

Margret A. Hjalmarson ,

School of Education, George Mason University, US
X close

Allison Ward Parsons

School of Education, George Mason University, US
X close

Abstract

Background: Design-based research frameworks have become more widely embraced by education researchers since 2003. Ann Brown (1992) first suggested design-based research as a means to move research about learning from the laboratory to the classroom setting. Design-based research does not imply particular methods, but rather is an epistemological approach to carrying out research projects focused on the design of an innovation. This shift in epistemological outcome shifts how researchers consider evidence gathering and analysis. So how are engineering education researchers using and reporting on design-based research?

Purpose: The design of educational innovations should be situated in context and balance both theory and practice. In this paper, we summarize research in engineering education that has used a design-based research approach. We first describe design-based research from a theoretical and epistemological perspective and then present its use thus far in engineering education settings. Our goal is to suggest it as a useful model for undertaking engineering education research that advances knowledge about the design of learning environments.

Scope: Our analysis is situated in significant features of design-based research (Kelly, 2004; Sandoval, 2014). Our review process began with systematically searching for articles and then coding them for the features of design-based research. We have coded a set of engineering education and design-based research articles to summarize the contexts, conjectures, and design cycles.

Discussion/Conclusions: We found articles across K–12 and higher education settings. The articles focused on designing models and tools for engineering education including curriculum, tasks, and frameworks. First, design-based research in engineering education includes conjectures about engineering disciplinary practices, problem-based learning, and affective aspects of students’ learning. Second, cycles of design inform improvement and theory development. Design-based research is still emerging in engineering education but has potential for building knowledge and developing resources grounded in practice.

How to Cite: Hjalmarson, M. A., & Parsons, A. W. (2021). Conjectures, Cycles and Contexts: A Systematic Review of Design-based Research in Engineering Education. Studies in Engineering Education, 1(2), 142–155. DOI: http://doi.org/10.21061/see.35
85
Views
23
Downloads
2
Twitter
  Published on 31 Mar 2021
 Accepted on 15 Feb 2021            Submitted on 16 Jan 2020

Design-based research as a methodological framework for research now has a long history in education dating back to the initial article by Ann Brown (1992) that explored how research about learning needed to move from the laboratory to the classroom in order to examine the intersection of theory and practice. The increased use of design-based research continued in the early 2000s with the publication of special issues in Educational Researcher and the Journal of the Learning Sciences plus handbook volumes (e.g., Kelly, Baek & Lesh, 2008; van den Akker et al., 2006). During this same period, Journal of Engineering Education created a new mission focused explicitly on research and scholarship in engineering education (Lohmann, 2003) and the field of engineering education research experienced growth and development. However, engineering education research has been slower to adopt design-based research frameworks than other education disciplines (e.g., science, math, and literacy). For instance, a review of design-based research found only 10 papers in engineering and technological sciences (including computer science) published between 2003–2013 and 61 in natural science (including mathematics) during the same period (Zheng, 2015). Anderson and Shattuck’s review (2012) found that half of the available articles focused on science, and the remainder on math, literacy, or other disciplines.

We present this synthesis to explore how design-based research has been used in engineering education research and present a rationale for why a design-based epistemology for research is particularly meaningful for engineering education. We begin by explaining salient distinctions between design-based research and other framings of education research. Then, we present a systematic review of existing papers in engineering education that use a design-based research methodological framing to guide the projects. We ultimately identified 17 papers to include in our analysis and synthesize how they approach characteristic features of design-based research. Our goal is to answer: how are engineering education researchers using and reporting on design-based research?

Epistemology and Features of Design-Based Research

Design as an Educational Research Endeavor

Design-based research frameworks have become more widely embraced by education researchers since 2003. Ann Brown (1992) first suggested design-based research as a means to move research about learning from the laboratory to the classroom setting. More recently, design-based research methods have been adopted to develop innovations in teaching and learning while also researching questions about teaching and learning (e.g., Hermes et al., 2012; Hira & Hynes, 2019; Kieran et al., 2013; Weber et al., 2014). Design-based research does not imply a particular set of methods, but rather is an epistemological approach to carrying out a research project focused on the design of an innovation. This shift in epistemological underpinning influences how researchers consider evidence gathering and analysis. For instance, if we approach the study of educational innovations as a design process rather than an experimental process, how does that shape the types of evidence that are used and the design of studies? If we shift from the testing of hypotheses, to the development of design principles and conjectures, how does that shape the research?

Design-based research is concerned not only with the question “Did it work?” but also the question “How did it work?” Design-based research is also concerned with how to make it better over more time, more settings, and with more people (Collins, Joseph & Bielaczyc, 2004). Epistemologically, this is a distinct way of thinking in terms of research design and the kinds of evidence that would support claims to make about what is developed. There is a distinction between testing hypotheses and designing a framework or a model (The Design-based Research Collective, 2003). The design-based researcher is interested both in what went well and what didn’t go well in order to refine the design over time (e.g., Newstetter, 2005). Variation between settings is not something to be controlled but rather something natural in the context in order to understand the generalizability of the design from setting to setting (The Design-based Research Collective, 2003; Kelly, 2006; Wang & Hannafin, 2010). Design-based research is also concerned with clarity about in what settings a design might work – not all designs are suitable for all settings. So, the design-based researcher should be clear about what is being designed and for what purpose.

A signature feature of design-based research is the development of working theory alongside the design of learning environments, tools, strategies, models, or resources. The design-based research process was originally conceived to understand learning in authentic contexts (e.g., Brown, 1992; Collins et al., 2004). Later authors explain the pragmatic purposes for design-based research as meeting the needs of the users of educational resources (e.g., Anderson & Shattuck, 2012; Sandoval & Bell, 2004; Wang & Hannafin, 2005). So what is developed is grounded in theory and practice and informs theory and practice. Design-based research explores the notion that researchers and educators can systematically gather evidence about learning and learning environments to develop and test theory at the same time that resources are designed (Cobb et al., 2003). Within this process is exploring how the resources are usable and used in multiple settings and contexts.

Stages of Design: Cycles and Iterations

“By studying a design with an eye toward progressive refinement, it is possible to develop more robust designs over time.” (Collins, Joseph, & Bielaczyc, 2004, p. 19).

“Nevertheless, the force of design research resides in an idea that unifies all of engineering—the concept of failure.” (Sloane & Gorard, 2003, p. 30).

Understanding what needs to be improved is as important as understanding what worked. In design-based research, because the objective is developing a resource, model, or framework that is usable in practice and guided by theory, the process of design is examined and documented as closely as the final product. In addition, a cyclic process brings in different forms of evidence about the context and the learners (e.g., cognitive, affective, social; Collins, Joseph & Bielaczyc, 2004). Progressive refinement means that design-based researchers report on variations and revisions as designs are improved. Along the way, they may also develop theory about education (e.g., Langman et al., 2019). For engineering knowledge, many concepts have not been studied at length so understanding and documenting students’ development of knowledge needs to occur and tasks need to be designed to elicit that knowledge. For instance, students’ learning in interdisciplinary settings needs further exploration for assessment and task design (Van den Beemt et al., 2020). Gainsburg (2015) explored the epistemological perspectives of engineering students about mathematics and similar studies could be done about science or computer science related to engineering. So how do engineering students view the role of computational thinking in engineering or develop understanding of computational thinking in engineering courses? Some stages of design-based research might be used to gather evidence about students’ conceptions of the content in addition to how the learning environment is designed. Evidence takes multiple forms depending on the questions of the designers, researchers or other stakeholders at different cycles of the research.

The iterative and cyclic process of design is a hallmark of design-based research. For example, Bannan-Ritland (2003) explains the Integrative Learning Design Framework that describes cycles of a design-based research process in education as grounded in design in other fields (e.g., technology, user-centered design). There are early stages of pilot testing and refinement that precede later stages that analyze the impact and implementation of a design. Examples in engineering education have explained such design cycles (e.g., Dasgupta, 2019; Hira & Hynes, 2019) as part of the research process. As a result, the sources of evidence at various stages vary depending on the research questions and the stage of the design process. For instance, in early stages, the research team needs to understand if the design works in a particular setting and then in later stages will shift to other settings to understand possible variations. The question for the project may not be “Did they learn?” but “What are they learning?” because the knowledge developed may need to be analyzed particularly for complex skills like learning about the engineering design process.

Design Principles & Conjectures

Another signature feature of design-based research is the creation of design principles (Wang & Hannafin, 2005) or design conjectures (Sandoval, 2014). In this structure, the research team makes explicit the rationale for how and why a design was developed and what it is intended to accomplish. These principles can be grounded in theory about learning. Principles are then embodied in the designs that are created. The iterative testing of the conjectures or principles is the goal of the study (e.g., Dasgupta, 2019; Hira & Hynes, 2019). Such conjectures may be refined as the cycles of the study proceed. The goal then is to not just produce one tool or resource but the principles that would guide the design of other tools and resources. There is an emphasis in design-based research on applicable, authentic contexts for research (Bannan-Ritland, 2003; Cobb, Confrey, diSessa, Lehrer & Schauble, 2003; Edelson, 2002; Wang & Hanafin, 2005). This is important for engineering education because the contexts of engineering problems are constantly changing. In engineering education, there is a priority placed on real-world applicable experiences, so limiting research to a particular task is not as productive for the field as understanding a space of tasks that might be defined by a set of underlying principles for their design. It is the role of the research team to explain how and when such principles might be meaningful. For instance, do the same principles for designing tasks apply across K–12 and undergraduate education or do conjectures for design tasks vary across these settings?

Sandoval (2014) describes the fundamental tension in educational design research as researchers grapple with the dual commitment of better understanding of learning processes as well as improving educational practices (p. 20). Theory, then, must be the initial step in conceptualizing design research: researchers must consider how, when, and why learning may happen as a result of the design in a learning environment. Sandoval refers to this theoretical conceptualization as a conjecture (2004; 2014). Bakker (2018) describes conjectures as hypotheses that “follow from some emergent theory that still needs to be tested empirically” (p. 48). Design research requires that researchers begin with a working theory that is embodied in a design, which is then iteratively tested and modified to achieve the desired learning outcome (pedagogical goal). The process is as interesting as the outcome as researchers learn what works in their particular environment. Sandoval (2014) explains that conjecture mapping codifies explicit conjectures and their expected performance to achieve learning outcomes, thereby making the abstract theory concrete and testable.

Conjecture mapping offers a way to visualize abstract thinking through use of active terms and connections between theory and key features of the design. A conjecture map can help coordinate theory with design decisions as researchers plan intervention components to address an identified learning problem. A conjecture map begins with a predictive hypothesis about how a potential learning support will work in the desired context. The conjecture is then embodied in a design, which should lead to mediating processes that support desired outcomes (Bakker, 2018). The embodiment of the conjecture includes the tools, measures, structures, and discursive practices, which can also be considered the characteristics of the design (Bakker, 2018). In turn, design conjectures connect the embodiment to mediating processes, which are the mechanisms of the study that connect the conjectures to the desired outcomes. For example, in Hira and Hynes (2019), one of the final principles for their learning experiences was “be human-centered” (p. 114) and they explain two mediating processes (or mechanisms) that support that principle and achieving the learning outcomes as “include authentic clients (real or fictional)” and “Prompt students to frame problems by defining the people, what those people want to be improved, and the nature of the improvement” (p. 114). As another example, Newstetter’s work for problem-based learning in biomedical engineering included the principle “scaffolding to support reasoning” (p. 209). The mediating process was a means for students to organize a white board to focus on different aspects of the problem that started with a diagnosis model from medicine using facts and hypothesis but evolved to pieces from engineering reasoning incorporating problem recognition, modeling, and validation (Newstetter, 2005). The mediating processes, in turn, are connected by theoretical conjectures to the study outcomes. In every step, then, theory must be actively considered and applied to the research in order to test and understand how (if?) it works in the intervention context (Bakker, 2018; Sandoval, 2014). However, Bakker reminds us that feedback loops between design and conjecture and also between design components must also be considered as the design and theory evolve during an experiment (2018, p. 57).

Conjecture mapping is a useful technique in engineering educational design research as it lends focus to the working theory and the possible designs that enable the theory to influence and impact the learning environment. As Cobb and colleagues explain, the working theory is “held accountable to the activity of design. The theory must do real work” (Cobb et al., 2003, p. 10). Therefore, in this synthesis, we consider the affordances of conjecture mapping on the corpus of engineering education design research studies. We consider the evidence of theory presented to support the design and intervention, look for conjectures to connect the theory with the embodied design, describe the design cycles, and examine the relevant details presented within and across studies to understand the overall state of design-based research in engineering education.

Systematic Review Process

Systematic reviews are useful when existing work needs to both be synthesized and when that synthesis can support forward movement on research in the discipline (Cooper & Hedges, 2009; Petticrew & Roberts, 2006). We acknowledge the nascent state of design-based research in engineering education, yet there is a growing corpus of literature and enough research to provide some early recommendations for future work. The systematic review process provides guidelines and organization to the process of synthesizing published articles in a consistent way to identify patterns in the research and gaps that might be filled in the future. Systematic reviews that are focused on reports of original research provide a standardized approach to review, ensuring more thoughtful analysis that leads to reliable and valid findings. In order to understand the evolution of design-based research as an emergent methodology in engineering education research, this paper presents a systematic review of the literature from 2003–2019. We selected 2003 as the starting point for this review due to the influential nature of the Educational Researcher special issue on design-based research published that year. We have situated the analysis in guiding concepts from design-based research (Kelly, 2004, 2014; Sandoval, 2014). To structure our work, we follow procedures outlined by Cooper (2010) which includes (a) formulating the problem, (b) searching the literature, (c) gathering information from studies, (d) evaluating the quality of the studies, (e) interpreting the evidence, and (f) presenting the results. We have identified the problem as a need to understand the corpus of engineering education research that has used design-based research methodology in order to determine how and when the methodology is used. By synthesizing the existing design-based research publications, we can formulate a clear picture of how design-based research is used in engineering education. In turn, this invites a discussion of future work possibilities.

Searching the Literature

We began the synthesis process by searching in the Education Research Complete database using the terms design-based research, design research, and developmental research plus education to locate general education-based design studies. The search parameters included publication in English, in a peer-reviewed journal, and a publication date of 2003 or later. While we recognize that publication bias exists, we adhere to the notion that the peer review process to vet research is desirable to gain an overview of design-based research in engineering education, and we therefore chose to exclude grey literature from this synthesis. Further, adhering to peer-reviewed journal publications offers a more consistent approach to reporting and the ability of researchers to extract information from those reports (Cooper, 2007; Rothstein & Hopewell. 2009). Our focus was on understanding research studies using design-based research frameworks so we wanted articles with as much information about the research reporting as possible. Therefore, this synthesis may be considered purposively sampled from the extant literature on design research in engineering education.

Initial searching returned a broad set of 7,900 articles. We next added the limiting search term engineering education, which reduced the set to 149 articles. We then examined titles and abstracts, and eliminated all that were not actually reports of design research, which further reduced our results to 35 potential articles for preliminary review. Criteria for the initial review included 1) ensuring the article referenced design-based research (or variations) as a research methodology, theoretical framework, or conceptual framework and 2) excluding projects about design education, or design projects as structures for students’ learning. After a preliminary review of each article, we determined that 23 were not focused on engineering education, or did not employ design-based research methods in their research, and so those articles were excluded from further review. To ensure we hadn’t missed articles specifically in engineering education publications, the first author next performed a hand search for design-based research and design-based implementation research1 of the Journal of Pre-College Engineering Research (J-PEER) and the Journal of Engineering Education (JEE), yielding 13 additional articles for review. Of those 13, five were discarded for editorial stance (2), other research methods (2), and research review (1). At this point we had 20 articles found to be design-based research studies. Three articles summarized long-term design-based projects as overviews of their work (Bernhard, 2010; Chiu & Linn, 2011; Marshall & Berland, 2012), but these articles did not include descriptions of the research methods or findings that would allow synthesis with the other articles in terms of research design. Instead, these three articles were more expository and editorial in nature rather than reports of specific research findings. We did not follow up with additional data gathering because these works were overviews of multiple projects rather than reporting on a study. Our final corpus of 17 studies featuring design-based research in engineering education is analyzed below.

Gathering the Information & Evaluating Quality from the Studies

Each of the 17 articles were coded for (1) general study elements and (2) specific design-based research elements to summarize the types of evidence that were gathered, the nature of what was designed, and general outcomes. We began by coding for the study elements, including evidence of conceptual or theoretical frameworks underpinning the research, educational focus in K–12 or university settings, specific research questions and goals, stakeholder and participant information, data collection, and data analysis using a spreadsheet format. Following that initial reading by both authors of all the articles, we created a coding structure for the design-based research elements including design principles, conjectures, embodiment or description of what was designed, and general outcomes. Each author created a separate coding table for the design-based research elements for half the articles.

Many systematic review studies include an in-depth assessment of the quality of the studies for the purposes of eliminating flawed studies with potentially questionable findings (Cooper, 2010). Our research seeks instead to understand how investigators are using and reporting design-based research. Our goal is not to make inferences about the findings of their studies, but rather to understand how design-based research is being operationalized, framed, and used by investigators. For reporting, given the large scope of many design-based research projects, we expected that most papers would report on only some components of the project that may be defined by a specific time period, the phase of the design process, a participant group, or another phenomenon of interest. In addition, there was large variation in methods used across qualitative, quantitative, and mixed methods research and evaluation so we also did not want to make comparisons about quality in a diverse but small sample of articles. In each article, we found selected information about their questions, data collection, and data analysis procedures even if incomplete in some ways. Based on our findings, we also provide suggestions about areas we see for improvement in design-based research and reporting that would not have been possible without in-depth analysis of the full set. The results of our analysis are presented in the following sections.

Findings: Interpreting the Evidence

Our findings are organized by aspects of design-based research: (1) research questions, (2) context, (3) cycles of design, and (4) conjectures, principles, and embodiment of designs. After the initial review, we selected these features to both synthesize general aspects of the projects (e.g., research questions, context) and analyze specific features of design-based research projects (e.g., cycles of design, conjectures, principles, embodiments of designs). The findings are reflective of the variation in contexts and interests in engineering education research and the focus on design of learning environments and innovations. Twelve of the publications were found in engineering education outlets (Journal of Engineering Education, Journal of Pre-college Engineering, European Journal of Engineering Education, and International Journal of Engineering Education). Five articles were from other areas in education research (Educational Technology Research & Development, Higher Education, Teacher Education Quarterly, and Journal of Cooperative Education & Internships). This is reflective of the multidisciplinary and interdisciplinary nature of engineering education as a whole and design-based research specifically.

Synthesis of Research Questions

The research questions were extracted from each article to learn what the project sought to understand or to design. For K–12 education, there were eight papers. For higher education, there were nine papers. Table 1 describes how the papers were organized by focus on the learning experience or the instructor/teacher level. Two themes emerged in the coding of the research questions. The first theme was that many projects focused not only on students’ or teachers’ knowledge of engineering, but also focused on affective variables like interest, motivation, or dispositions for engineering. These kinds of affective variables are complex and difficult to measure, but they are important for considering the design of learning environments. The second theme focused on designing meaningful or authentic engineering learning experiences for students. These experiences needed to be a representation of the discipline of engineering, possible work environments, or connected to students’ interests that might influence the selection of task contexts/situations. The development of disciplinary knowledge was intertwined with the affective considerations. Projects attempted to balance both these areas. These two themes are linked in that the goal of a meaningful and authentic task is connected to supporting and developing students’ interest in engineering. The pedagogical strategies and structures selected (e.g., problem-based learning, design tasks/challenges, modeling, cooperative education) are conducive to developing disciplinary knowledge in meaningful contexts that are authentic to engineering.

Table 1

Study Context and Focus.

Note: Articles listed in italics are focused on affect (e.g., interest, motivation) of students or teachers. Gale et al. (2019) is listed in both learning experience and teacher focus because it included data about the perspectives of both students and teachers.

Synthesis of Study Focus

Of the studies examined in this systematic review, eight explored aspects of engineering education in K–12 settings (Blanchard, 2015; Dasgupta, 2019; Gale et al., 2019; Hardre et al., 2010; Hira & Hynes, 2019; Langman et al., 2019; Moore et al., 2014; Tang, 2013). Moore and colleagues (2014) sought to create a standards framework to inform quality K–12 engineering education. Hardre and colleagues (2010) examined residential professional development to build teacher effectiveness in engineering education. The remaining five examined designs to build student interest and engagement in engineering through problem-solving activities and clubs. Hira and Hynes (2019) were the only researchers who included upper elementary students in their study. The other researchers focused their work in middle and high school settings. Langman and colleagues (2019) explored model-eliciting activities in both high school and college settings, so it is categorized in both K–12 and higher education.

Nine studies examined design-based research in university and community college settings (Diefes-Dux et al., 2010; Freidrichsen et al., 2017; Guloy, 2017; Langman et al., 2019; Newstetter, 2005; Spelt et al., 2015; Todd et al., 2011; Weber et al., 2014; Yueh et al., 2014). Of those, Diefes-Dux et al. and Friederichsen et al. focused on the instructors through development of tools to aid reliable and valid student evaluation, and a network for sharing concepts, respectively. Guloy and colleagues examined paired learning supports for first-year engineering and computer science students in order to increase student success. Yueh and colleagues examined a framework for assessing e-learning design for virtual nanotechnology labs.2 Similar to the embodied designs in the K–12 studies described above, the remaining six university-based studies (Langman et al., 2019; Newstetter, 2005; Spelt et al., 2015; Todd et al., 2011; Weber et al., 2014) explored elements and applied models of problem solving along with other learning mechanisms to engage and support engineering students in undergraduate and graduate programs.

The final study was not oriented towards specific student or instructor experiences in specific educational settings. Finelli and colleagues (2015) designed tools to support engineering education researchers through taxonomies of terms/keywords. Their stated goal was to make engineering education research more easily searchable through use of shared keywords.

Overall, we noted that students are the key stakeholders in the corpus of studies. Most of the K–12 studies (Blanchard et al., 2015; Dasgupta, 2019; Gale et al., 2019; Hira & Hynes, 2019; Langman et al., 2019; Tang, 2013) sought to increase opportunities for middle and high school students (and their teachers, as in Hardre et al., 2010) to explore engineering in engaging formats in hopes of stimulating interest in future STEM careers. Most of the university studies explored coursework and student engagement in practical problem-solving to model the pragmatic, real-world nature of applied engineering across subfields (Diefes-Dux et al., 2010; Newstetter, 2005; Spelt et al., 2015; Todd et al., 2011; Weber et al., 2014; Yueh et al., 2014). Guloy and colleagues (2017) were somewhat unique in that their work focused on supporting engineering students outside of class in order to keep them engaged and motivated in their coursework. Together, these studies reflect the desirability of design-based research to address problems of practice and reveal the nuances of specific designs in real educational contexts (Anderson & Shattuck, 2012).

Design Cycles and Iterations

As discussed previously, the iterative and cyclic nature of design-based research is a common feature of the methodology that distinguishes it from other types of education research. Since it is a design process, having such cycles and iterations is a key component of describing how and why the object of design was developed as well as the findings and outcomes. We have summarized how the papers diverged in terms of explicitly reporting on multiple cycles or not. We are intentional here about describing this as reporting to indicate that a design-based research study often includes more than what is any individual paper (e.g., more cycles, additional data types or findings). It is not uncommon for individual articles to only report on one aspect or component of the design-based research project because of the complexity of such projects. This does not disqualify the studies from being design-based research, rather it is a choice on the part of the researcher(s) about what aspects to publish and at what points. For example, the publication of an impact study conducted as part of the design process (e.g., Blanchard et al., 2015) or a comparison of two research sites (e.g., Gale et al., 2019) does not mean the study as a whole is not design-based research. It only means that the researchers have selected to publish findings about an impact study. Table 2 presents the organization of the articles by the cycles reports and the criteria describing each category.

Table 2

Articles by Design-Based Research Cycles Reported.

Cycles reported Studies Paper features

Multiple, iterative design cycles Dasgupta, 2019
Diefes-Dux et al., 2010
Finelli et al., 2015
Hira & Hynes, 2019
Langman et al., 2019
Moore et al., 2014
Newstetter, 2005
Todd et al., 2011
  • Explains and justifies revisions over time to principles/conjectures or the object of design (e.g., curriculum, tools, frameworks)
  • Multiple sources of evidence that may vary over phases/cycles
  • Multiple participants/reviewers over time
  • Explains variations, revisions, and refinements
One design cycle or one aspect Blanchard et al., 2015–impact study
Friedrichsen et al., 2017
Gale et al., 2019
Guloy et al., 2017
Hardre et al., 2010
Spelt et al., 2015
Tang, 2013
Weber et al., 2014
Yueh et al., 2014
  • Examines/investigates one version of the design
  • Explains design process that led to the version used in the publication
  • Explains research questions and sources of evidence
  • Conclusions/implications provide recommendations for future designs

Within the papers reporting on multiple design cycles or iterations, the tool or curriculum resources being designed may go through multiple iterations (e.g., Diefes-Dux et al., 2010; Finelli et al., 2015; Langman et al., 2019; Moore et al., 2014; Newstetter, 2005). In these types of studies with tasks, activities or pedagogical tools, students’ work on the tasks is a critical source of evidence to support refinement of the design. Designers document this evidence either using artifacts from students (e.g., written responses to tasks) or observations (e.g., videos of classroom work or small groups of students). Researchers may support this evidence with interviews or surveys about students’ experience. However, to understand the design of the task itself, design-based researchers need to document how students completed the task in order to know whether the guiding principles or conjectures were met or not. Alternatively, the conjectures or principles guiding the design may also be refined (e.g., Hira & Hynes, 2019; Newstetter, 2005). Students’ work on tasks and observation of students’ solving tasks informs these revisions as well. For instance, Newstetter states, “In analyzing the results from our initial design experiment, we speculated that the scaffolding system from medicine might be the culprit. We hypothesized that it was failing to guide students towards engineering solutions which utilized diagrams or models as a starting point. We needed a model of engineering reasoning and problem solving, less based on diagnosis” (p. 210). The investigators are revisiting the principles underlying their design and describing a design failure. This explicit description of failure is a significant piece of design-based research as it is important to document the design processes’ successes and failures, strengths and weaknesses.

The papers reporting on single cycles or single aspects of a design-based research study have a different purpose in their reporting. These papers report a single macrocycle or phase of the larger design, with multiple microcycles encompassing smaller data collection, analysis, and revision cycles (Gravemeijer & Cobb, 2006). For instance, Tang (2013) documents media representations of engineering that students’ encounter in informal and formal learning settings. The design-based research used to create the learning environment created a context for gathering artifacts high school students found meaningful and interviewing them about their perspectives of engineering. Friedrichsen and colleagues (2017) have a different purpose in using a diffusion of innovations framework to understand use of their concept warehouse over a three-year period of annual macrocycles. For example, they ask “Do different communication channels correspond to differences in types of demonstrated user knowledge and to the proportion of users who reach the implementation stage in the innovation-decision process?” (p. 179) Gale et al. (2019) explore variations in implementation of the same materials by two different middle school teachers. Blanchard et al. (2015) conducted an impact study of their design as part of their research project. Other authors report on early stages of design. In all cases, the authors are attempting to explore specific aspects, phases, or characteristics of what they are designing. The authors also acknowledge that what is reported is part of a larger design-based research project.

Conjectures, Principles and Embodiment of Designs

As discussed previously, design conjectures or principles are a signature component of design-based research. For each article, we either used (1) a set of principles or conjectures specifically described that way or (2) inferred based on what was being designed and the design process if the authors stated characteristics or guidance that was shaping the design process and the designs that were developed. Table 3 summarizes the different design embodiments in the articles. For instance, Hira and Hynes (2019) describe the design process for an interest-based engineering challenge framework including three principles: “be human-centered,” “have broad themes,” and “involve the making of artefacts” (p. 114), with details shaped by multiple cycles of design. Revision of the operationalization of those principles resulted in guidance for each principle such as “include authentic clients (real or fictional)” and “themes can either be chosen by the student team or be broad enough that students’ teams can choose a topical interest within the theme” (p. 114). The principles were used to make pedagogical choices and design tasks for the learning environment.

Table 3

Embodiment of Designs by Studies.

Embodiment of Design Studies Paper features

Curriculum design and implementation Blanchard et al., 2015
Gale et al., 2019
Hardre et al., 2010
Hira & Hynes, 2019
Langman et al., 2019
Newstetter, 2005
Spelt et al., 2015
Tang, 2013
Todd et al., 2011
Weber et al., 2011
Yueh et al., 2014
  • Describes specific courses, modules, or tasks
  • May include multiple cycles of revisions of the materials in the article
Pedagogical tools, strategies, or supports Dasgupta, 2019
Diefes-Dux et al., 2010
Friedrichsen et al., 2017
Guloy et al., 2017
  • May be independent of a particular class or content topic
  • May support implementation problem-based or inquiry-based learning
Frameworks Finelli et al., 2015
Moore et al., 2014
  • Larger scope
  • Beyond a specific pedagogical approach or a course

One group of articles focused on curriculum design and implementation in the form of modules, tasks, or whole courses (Blanchard et al., 2015; Gale et al., 2019; Hardre et al., 2010; Hira & Hynes, 2019; Langman et al., 2019; Newstetter, 2005; Spelt et al., 2015; Tang, 2013; Todd et al., 2011; Weber et al., 2011; Yueh et al., 2014). Guiding principles for the modules often included supporting students’ engagement in meaningful or relevant engineering contexts (e.g., Langman et al., 2019; Newstetter, 2005). These curricular designs also reflected engineering practices (e.g., problem solving, design-based learning, modeling, multidisciplinary teams, collaboration). For instance, Blanchard et al. (2015) designed an after-school robotics club for middle school called Beyond Blackboards, which was “a project rooted in the design principles of contextualized, inquiry-oriented instruction; socially relevant design problems and collaborative problem solving” (p. 2). Other projects also emphasized the need to increase students’ understanding of engineering not only in terms of specific content but also as a career and a discipline. This served both increasing students’ interest in wanting to pursue engineering as a career (e.g., Blanchard et al., 2015) but also areas of engineering that might be less familiar (e.g., Newstetter, 2005, in biomedical engineering; Weber et al., 2014, on environmental science; Yueh et al., 2014, on nanotechnology). Hardre et al. (2010) designed a professional development program for teachers to help them understand and implement science and engineering in their classrooms in meaningful, authentic ways by engaging them in research with university faculty.

A second group of articles focused on pedagogical tools, frameworks, or learning supports that might go beyond a particular course or content area (e.g., Dasgupta, 2019; Diefes-Dux et al., 2010; Friedrichsen et al., 2017; Guloy et al., 2017). The resources were often designed to support problem-based learning, inquiry-based instruction, or other related models where students might need more support, or where aspects of instruction might be novel for students and instructors. For instance, Diefes-Dux et al. (2010) developed assessment tools for students working on modeling tasks and teaching assistants who needed a reliable and valid way to evaluate student work. Friedrichsen et al.’s (2017) concept warehouse included assessment tools and a resource repository for supporting concept-based instruction in chemical engineering. In terms of learning supports, Guloy et al. (2017) designed a paired learning support for undergraduate students and Dasgupta (2019) designed improvable models to scaffold students solving engineering challenges. Similar to the curriculum resources described previously, conjectures and principles underlying these designs also focused on engineering disciplinary practice, student engagement or interest, and supporting students’ learning of complex content and practices (e.g., collaboration). The tools also needed to be usable by the intended users (e.g., teaching assistants, course instructors). Conjectures and principles sometimes included practical elements such as “…to maintain fidelity to characteristics of high performance as described by engineering experts and to obtain reliable scoring by graduate teaching assistants…” (Diefes-Dux et al., 2010, p. 807).

A third set of articles focused on the design of frameworks to inform the engineering education, teaching, learning, or practice at a larger scale (e.g., Finelli et al., 2015; Moore et al., 2014). In Finelli et al. (2015), the design is a taxonomy of keywords for engineering education research. Moore et al. (2014) developed the Framework for Quality K–12 Engineering Education. These two resources go beyond specific content areas, courses, or pedagogical practices. Rather they were developed to inform the wider community. For Finelli et al., that community is engineering education researchers and practitioners. For Moore et al. (2014), that community includes engineering educators and researchers but also education policymakers, teachers, and leaders at the K–12 level. Both tools have specific purposes and goals. The underlying principles for both included aspects of usability, meaningfulness, and being reflective of engineering and engineering education as disciplines.

Methodologically, design-based research supports the design of resources and tools alongside the analysis of underlying theoretical and practical conjectures (Sandoval, 2014). The papers we analyzed exemplified this feature by stating these underlying principles or conjectures. Then, investigators analyzed the use or implementation of the resulting tools (e.g., Gale et al., 2019). Investigators might also revisit the conjectures themselves in the publication (e.g., Hira & Hynes, 2019; Newstetter, 2005). The final design and the principles should be improved by the end of the project. The design process then informs the creation of future resources and the understanding of the theories that underlie the design.

Discussion & Implications

All researchers framed their focal problem, research goals, and/or research questions within their educational settings. All researchers offered justification and grounding for their study choices through conjectures and working theory of how their embodied designs may address the problem. All 17 studies provided details about the designs-in-action and needed adaptations to improve designs to address working theory and project goals. In seven studies, the researchers referenced specific theories to support the development of their working theory/conjectures and embodied designs (e.g., Friedrichsen et al., 2017; Dasgupta, 2019; Hira & Hynes, 2019; Newstetter, 2005; Spelt et al., 2015; Tang, 2013; Todd et al., 2011). The working theories may be strengthened by additional support from an established theoretical base. Many of the papers described the iterations and cycles of design they used. The authors also addressed a variety of designs across K–12, higher education, and engineering education research in general, although we detected a lack of research involving K–3 learners, and just one study including grades 4–5 learners (e.g., Hira & Hynes, 2019). Further, the papers represent a variety of research methods across quantitative, qualitative, and mixed data collection and analysis which is reflective of the different stages of design and implementation they were investigating.

However, while all papers included identified themselves as design-based research, they varied in terms of explaining some typical aspects of design-based research. Data collection and design cycles sometimes included a range of stakeholder perspectives including teachers or instructors, expert review, or analysis (e.g., Finelli et al., 2015; Moore et al., 2014), engineering content experts (Diefes-Dux et al., 2010), or other students (e.g., Hira & Hynes, 2019; Langman et al., 2019). Particularly for articles that described multiple cycles of design or were explaining early-stage work, the data collection included explicit attention to review by other experts or pilot testing with students/teachers. This kind of review and data collection is a critical point of understanding possible points of failure and improvement before a design is used more broadly.

Theoretical frameworks offer a starting place for planning and initial study design. For instance, theoretical frameworks about learning guide the creation of curriculum materials or pedagogical approaches. Dasgupta (2019) presents clear implications for how theory informed the design and how students’ interaction with the curriculum contributes to theory about students’ engagement. However, what is less clear in some of the papers is how findings then contribute to theoretical frameworks or offer revisions to theory. For engineering education research, there is also a parallel growth happening between the use of design-based research frameworks and the growth of an engineering education research more broadly. Only one paper was published before 2010 (Newstetter, 2005). As engineering education has needed more research about teaching and learning alongside the design of resources to support new pedagogies, design-based research will hopefully continue to support the design of frameworks and guiding principles that can be used in many settings. Given a focus on meaningful and relevant contexts for learning, there is a need not just for tasks and curriculum but also mechanisms for local design and creation of such resources to fit current needs. Projects focused in some cases are areas where students might have less knowledge (e.g., biomedical engineering, nanotechnology, environmental science) but that represent areas that would broaden their understanding of the practice and discipline of engineering in new ways thereby helping more students view engineering as a possible career path.

Projects also reflected attention to increasing interest and engagement in engineering by working with more diverse sets of learners. However, topics of equity were limited to a few articles (e.g., Blanchard et al., 2015, worked in a “Hispanic-majority school district”). The theoretical frameworks represented did not generally include critical theories or other frameworks reflected in studies of diversity in learners’ knowledge and perspectives. Design-based research projects in other education research disciplines have explored these issues (e.g., Gutiérrez & Jurow, 2016; Hermes et al., 2012; Stromholt & Bell, 2018). Hira and Hynes (2019) engage a humanist perspective on learning that is aligned with these perspectives; however, this is an area where design-based research could be used to understand different contexts and varied learner experiences, knowledge, and perspectives. This would move beyond only seeking to increase interest in engineering to also valuing and engaging diverse perspectives on engineering.

There are a few areas where design-based research could contribute to engineering education research. Broadly, there are existing studies in curriculum design or other studies of pedagogical strategies that could use design-based research as a framework for conducting the research. Then, other researchers or instructors would have better understanding of the design principles and how those might generalize to other contexts. For instance, there is a need for developing K–12 engineering teaching and for helping teachers understand engineering learning (National Academies of Science, 2020). Our review included a few studies focused on in-service teachers (those who are already teaching K–12), but design-based research would support the development of courses and curriculum for pre-service teachers. Design-based research could explore the kinds of school-based experiences in engineering learning that pre-service teachers need or the development of courses for elementary engineering teaching and learning. Design-based research frameworks could engage the design process, the development of learning progressions for elementary engineering teaching, or the theories that would support the classroom practices pre-service teachers need to learn for integrating engineering and other disciplines. The design of learning progressions generally would be an area for expansion in engineering education research. Another area for growth is the study of interdisciplinarity in engineering and students’ learning in interdisciplinary contexts as described in a recent review of the research in this area (Van den Beemt et al., 2020). For example, their review raises questions about assessment design for interdisciplinary projects: The kinds of scaffolding and supports for interdisciplinary learning could be addressed by design-based research that would result in principles that could be used across multiple contexts or settings.

The design-based research projects included in this systematic review could also be improved either in the reporting or the conduct of the study. We did not conduct an evaluation of the quality of the research methodologies, but most studies included qualitative and quantitative data which is consistent with design-based research (Bakker, 2018). Common challenges for researchers are deciding (a) what to report from a project and (b) how much data to include. For the purposes of a systematic review, we were cautious about focusing on what was reported because we recognize from our own experience and the reporting of other projects that there may be data and questions that are beyond the scope of an individual paper. At the same time, given that design-based researchers are often simultaneously designing and investigating the design, they need to be especially attentive to bias, reliability, and validity when reporting on their work. In describing a design process, it is important to report on failures as well as successes. For instance, Newstetter (2005) provides an account of the pieces of the scaffolding for students’ learning that didn’t work and the pieces that did work. This requires an epistemological shift from looking only for what works to also understanding what worked in what contexts and reporting changes in the design over time. Such changes are not research design flaws, but a natural part of the design-based research process that need to be documented for the benefit of those attempting similar approaches.

Limitations

A clear limitation of any systematic literature review is that authors are limited by the space available in their articles. In our case, we only included peer-reviewed journal articles. Some projects were grounded in dissertations (e.g., Dasgupta, 2019), which may provide further information about the study. Some projects were one phase of a large study (e.g., the impact study by Blanchard et al., 2015) or one aspect of a larger study (e.g., Tang, 2013). The three overview papers we did not include in our detailed analysis provide descriptions of long-term projects that evolved over many years (e.g., Bernhard, 2010; Chiu & Linn, 2011; Marshall & Berland, 2012). Such overviews are helpful for understanding overarching frameworks, revision over time, and the larger context of a design-based research project.

For design-based researchers, there are often many complex layers and cycles in any study. Sometimes this is considered an overwhelming amount of data (Wang & Hannafin, 2005; Sandoval, 2014), but it also means that collections of papers might be published by an interdisciplinary team of researchers. While journals are important for a peer review process, and indeed were targeted for this systematic review, we recognize that other types of publications are valuable for presenting the larger scope of a project. Alternatively, teams should attend to what disciplines may be interested in different aspects as they investigate not only engineering education but math and science education, educational psychology, learning sciences, teacher education, higher education, or other disciplines.

There may also be projects that share some features of design-based research but do not claim to be design-based research, and therefore were not picked up in our keyword search of the databases or included in our analysis. For instance, Cunningham et al. (2020) frame design principles for creating elementary engineering curriculum and have conducted studies about that curriculum. However, to our knowledge, they do not describe their work as design-based research in the papers they have published. There are other researchers engaged in curriculum design, studies of pedagogical tools, studies of educational technology and other resources for engineering education that do not characterize their work as design-based research. One of our goals in this paper is to encourage such researchers to incorporate the tools of design-based research as an epistemological framework that might be useful for organizing and carrying out such work. It would be worth exploring how researchers in engineering education are conceptualizing the design of such resources, but that is beyond the scope of our work here.

Conclusions

In this systematic review we sought to understand how design-based research has been used in engineering education. We found 17 relevant studies published in peer-reviewed journals and analyzed them for elements of design-based research. All 17 studies rooted their work in problems of practice noted within the field. Half described a single macrocycle, or phase of data collection and analysis of a design in action. The other half described multiple cycles of data collection and analysis in a larger design-based research project. These findings suggest that design-based research is useful to describe various aspects of research, particularly in single-macrocycle reports where the focus is on specific elements that can impact thinking in the field. The level of detail and amount of data collected in design-based research projects tends to be vast, and in order to provide deep and rich description of particular project aspects, papers that report a slice of a larger project are useful.

Our second goal for this systematic review was to explore ways in which design-based epistemology for research is particularly meaningful for engineering education. Overwhelmingly, these 17 studies demonstrate that design-based research is important in understanding how interventions and designs worked in specific engineering contexts. The breadth of the studies shows that engineering education is concerned with a wide array of topics and problems of practice. As designs are further developed and need study in new contexts, design-based implementation research as a variation of design-based research may be helpful (e.g., Penuel et al., 2011). We only found one article that described itself as design-based implementation research (Gale et al., 2019); however, this is a framework that is expanding in disciplines such as mathematics and science education for investigations at the district-level or at larger scale (e.g., Cobb et al., 2017; Penuel & Fishman, 2012). The studies in our review were largely situated in one school or a few classrooms so questions of scale represent a future frontier for design-based research in engineering education.

Given that engineering blends theoretical and empirical research to create pragmatic value (Reinking & Bradley, 2008), design-based research is well matched for engineering education research. Our findings echo those of Anderson and Shattuck (2012), who noted that design-based research is germane to education research, as it “seeks to increase the impact, transfer, and translation of education research into improved practice” (p. 16). The assembled studies in this review examined pragmatic problems of effective labs and coursework, increased student engagement and interest in STEM careers, plus frameworks, taxonomies, and databases to guide instruction and research. Underlying each article reviewed herein were sound conjectures (Sandoval, 2014) and principles (Anderson & Shattuck, 2012) of design-based research.

Notes

1Design-based implementation research was added as a search term for the two engineering journals based on the suggestion of a reviewer. Only one article in J-PEER (Gale et al., 2019) was found using this search term. 

2For Yueh, the article was not specific about the grade levels of the learners; however, based on references to common constructs for higher education (e.g., laboratory assistants) and review of another article on the same topic by the lead author, we have inferred that they intend a higher education context (Yueh & Sheen, 2009). 

Disclaimer & Acknowledgement

This material is based upon work while the first author (Hjalmarson) was serving as a Program Officer at the National Science Foundation. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Competing Interests

The authors have no competing interests to declare.

References

    References marked with an asterisk* indicate that the article was part of the review.

  1. Anderson, T., & Shattuck, J. (2012). Design-based research: A decade of progress in education research? Educational Researcher, 41(1), 16–25. DOI: https://doi.org/10.3102/0013189X11428813 

  2. Bakker, A. (2018). Design research in education: A practical guide for early career researchers. Routledge. DOI: https://doi.org/10.4324/9780203701010 

  3. Bannan-Ritland, B. (2003). The role of design in research: The integrative learning design framework. Educational Researcher, 32(1), 21–24. DOI: https://doi.org/10.3102/0013189X032001021 

  4. *Bernhard, J. (2010). Insightful learning in the laboratory: Some experiences from 10 years of designing and using conceptual labs. European Journal of Engineering Education, 35(3), 271–287. DOI: https://doi.org/10.1080/03043791003739759 

  5. *Blanchard, S., Judy, J., Muller, C., Crawford, R., Petrosino, A., White, C., Lin, F.-A., & Wood, K. (2015). Beyond blackboards: Engaging underserved middle school students in engineering. Journal of Pre-College Engineering Education Research (J-PEER), 5(1), Article 2. DOI: https://doi.org/10.7771/2157-9288.1084 

  6. Brown, A. L. (1992). Design experiments: Theoretical and methodological challenges in creating complex interventions. Journal of the Learning Sciences, 2, 141–178. DOI: https://doi.org/10.1207/s15327809jls0202_2 

  7. *Chiu, J., & Linn, M. (2011). Knowledge integration and wise engineering. Journal of Pre-College Engineering Education Research (J-PEER), 1(1), Article 2. DOI: https://doi.org/10.7771/2157-9288.1026 

  8. Cobb, P., Confrey, J., diSessa, A., Lehrer, R., & Schauble, L. (2003). Design experiments in educational research. Educational Researcher, 32(1), 9–13. DOI: https://doi.org/10.3102/0013189X032001009 

  9. Cobb, P., Jackson, K., & Dunlap Sharpe, C. (2017). Conducting design studies to investigate and support mathematics students’ and teachers’ learning. In Cai, J. (Ed.), Compendium for research in mathematics education, 208–236. National Council of Teachers of Mathematics. 

  10. Collins, A., Joseph, D., & Bielaczyc, K. (2004). Design research: Theoretical and methodological issues. Journal of the Learning Sciences, 13, 15–42. DOI: https://doi.org/10.1207/s15327809jls1301_2 

  11. Cooper, H. (2007). Evaluating and interpreting research syntheses in adult learning and literacy. National College Transition Network, New England Literacy Resource Center/World Education, Inc. DOI: https://doi.org/10.1037/e549792010-001 

  12. Cooper, H. (2010). Research synthesis and meta-analysis: A step-by-step approach (4th ed.). Sage. 

  13. Cooper, H., & Hedges, L. V. (2009). Research synthesis as a scientific process. In Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). The handbook of research synthesis and meta-analysis. 3–16. Sage. DOI: https://doi.org/10.7758/9781610448864.4 

  14. Cunningham, C. M., Lachapelle, C. P., Brennan, R. T., Kelly, G. J., Tunis, C. S. A., & Gentry, C. A. (2020). The impact of engineering curriculum design principles on elementary students’ engineering and science learning. Journal of Research in Science Teaching, 57(3), 423–453. DOI: https://doi.org/10.1002/tea.21601 

  15. *Dasgupta, C. (2019). Improvable models as scaffolds for promoting productive disciplinary engagement in an engineering design activity. Journal of Engineering Education, 108(3), 394–417. DOI: https://doi.org/10.1002/jee.20282 

  16. *Diefes-Dux, H. A., Zawojewski, J. S., & Hjalmarson, M. A. (2010). Using educational research in the design of evaluation tools for open-ended problems. International Journal of Engineering Education, 26(4), 807–819. 

  17. Edelson, D. (2002). Design research: What we learn when we engage in design. Journal of the Learning Sciences, 11(1), 105–121. DOI: https://doi.org/10.1207/S15327809JLS1101_4 

  18. *Finelli, C. J., Borrego, M., & Rasoulifar, G. (2015). Development of a taxonomy of keywords for engineering education research. Journal of Engineering Education, 104(4), 365–387. DOI: https://doi.org/10.1002/jee.20101 

  19. *Friedrichsen, D., Smith, C., & Koretsky, M. (2017). Propagation from the start: The spread of a concept-based instructional tool. Educational Technology Research & Development, 65(1), 177–202. DOI: https://doi.org/10.1007/s11423-016-9473-2 

  20. Gainsburg, J. (2015). Engineering students’ epistemological views on mathematical methods in engineering. Journal of Engineering Education, 104(2), 139–166. DOI: https://doi.org/10.1002/jee.20073 

  21. *Gale, J., Koval, J., Ryan, M., Usselman, M., & Wind, S. (2019). Implementing NGSS engineering disciplinary core ideas in middle school science classrooms: Results from the field. Journal of Pre-College Engineering Education Research (J-PEER), 9(1), Article 2. DOI: https://doi.org/10.7771/2157-9288.1185 

  22. Gravemeijer, K., & Cobb, P. (2006). Design research from a learning design perspective. In van Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.), Educational Research Design, 17–51. Routledge. 

  23. *Guloy, S., Salimi, F., Cukierman, D., & McGee Thompson, D. (2017). Insights on supporting learning during computing science and engineering students’ transition to university: A design-oriented, mixed methods exploration of instructor and student perspectives. Higher Education, 73(3), 479–497. DOI: https://doi.org/10.1007/s10734-016-0097-6 

  24. *Hardré, P. L., Nanny, M., Refai, H., Ling, C., & Slater, J. (2010). Engineering a dynamic science learning environment for K–12 teachers. Teacher Education Quarterly, 37(2), 157–178. https://www.jstor.org/stable/23479594. 

  25. Hermes, M., Bang, M., & Marin, A. (2012). Designing indigenous language revitalization. Harvard Educational Review, 82(3), 381–402. DOI: https://doi.org/10.17763/haer.82.3.q8117w861241871j 

  26. *Hira, A., & Hynes, M. M. (2019). Design-based research to broaden participation in pre-college engineering: Research and practice of an interest-based engineering challenges framework. European Journal of Engineering Education, 44(1/2), 103–122. DOI: https://doi.org/10.1080/03043797.2017.1405243 

  27. Kelly, A. E. (2004). Design research in education: Yes, but is it methodological? The Journal of the Learning Sciences, 13(1), 115–128. DOI: https://doi.org/10.1207/s15327809jls1301_6 

  28. Kelly, A. E. (2006). Quality criteria for design research: Evidence and commitments. In van Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (Eds.), Educational design research, 107–118. Routledge. 

  29. Kelly, A. E. (2014). Design-based research in engineering education: Current state and next steps. In Johri, A., & Olds, B. M. (Eds.), Cambridge handbook of engineering education research, 497–517. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781139013451.032 

  30. Kelly, A., Lesh, R., & Baek, J. (Eds.). (2008). Handbook of design research methods in education: Innovations in science, technology, engineering, and mathematics learning and teaching. Routledge. 

  31. Kieran, C., Boileau, A., Tanguay, D., & Drijvers, P. (2013). Design researchers’ documentational genesis in a study on equivalence of algebraic expressions. ZDM, 45(7), 1045–1056. DOI: https://doi.org/10.1007/s11858-013-0516-4 

  32. *Langman, C., Zawojewski, J., McNicholas, P., Cinar, A., Brey, E., Bilgic, M., & Mehdizadeh, H. (2019). Disciplinary learning from an authentic engineering context. Journal of Pre-College Engineering Education Research (J-PEER), 9(1), Article 5. DOI: https://doi.org/10.7771/2157-9288.1178 

  33. Lohmann, J. R. (2003). The editor’s page. Journal of Engineering Education, 92(1), 1. DOI: https://doi.org/10.1002/j.2168-9830.2003.tb00732.x 

  34. Marshall, J., & Berland, L. (2012). Developing a vision of pre-college engineering education. Journal of Pre-College Engineering Education Research (J-PEER), 2(2), Article 5. DOI: https://doi.org/10.5703/1288284314869 

  35. *Moore, T., Glancy, A., Tank, K., Kersten, J., Smith, K., & Stohlmann, M. (2014). A framework for quality K–12 engineering education: Research and development. Journal of Pre-College Engineering Education Research (J-PEER), 4(1), Article 2. DOI: https://doi.org/10.7771/2157-9288.1069 

  36. National Academies of Sciences, E. (2020). Building Capacity for Teaching Engineering in K–12 Education. DOI: https://doi.org/10.17226/25612 

  37. *Newstetter, W. C. (2005). Designing cognitive apprenticeships for biomedical engineering. Journal of Engineering Education, 94, 207–213. DOI: https://doi.org/10.1002/j.2168-9830.2005.tb00841.x 

  38. Penuel, W. R., & Fishman, B. J. (2012). Large-scale science education intervention research we can use. Journal of Research in Science Teaching, 49(3), 281–304. DOI: https://doi.org/10.1002/tea.21001 

  39. Penuel, W. R., Fishman, B. J., Cheng, B. H., & Sabelli, N. (2011). Organizing research and development at the intersection of learning, implementation, and design. Educational Researcher, 40, 331–337. DOI: https://doi.org/10.3102/0013189X11421826 

  40. Petticrew, M., & Roberts, H. (2006). Systematic reviews in the social sciences: A practical guide. Blackwell Publishing. DOI: https://doi.org/10.1002/9780470754887 

  41. Reinking, D., & Bradley, B. A. (2008). On formative and design experiments: Approaches to language and literacy research. Teachers College Press. 

  42. Rothstein, H. R., & Hopewell, S. (2009). Grey Literature. In Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). The handbook of research synthesis and meta-analysis, 103–126. Sage. 

  43. Sandoval, W. A. (2004). Developing learning theory by refining conjectures embodied in educational designs. Educational Psychologist, 39, 213–223. DOI: https://doi.org/10.1207/s15326985ep3904_3 

  44. Sandoval, W. A. (2014). Conjecture mapping: An approach to systematic educational design research. The Journal of the Learning Sciences, 23, 18–36. DOI: https://doi.org/10.1080/10508406.2013.778204 

  45. Sandoval, W. A., & Bell, P. (2004). Design-based research methods for studying learning in context: Introduction. Educational Psychologist, 39(4), 199–201. DOI: https://doi.org/10.1207/s15326985ep3904_1 

  46. Sloane, F. C., & Gorard, S. (2003). Exploring modeling aspects of design experiments. Educational Researcher, 32(1), 29–31. DOI: https://doi.org/10.3102/0013189X032001029 

  47. *Spelt, E. J. H., Luning, P. A., van Boekel, M. A. J. S., & Mulder, M. (2015). Constructively aligned teaching and learning in higher education in engineering: What do students perceive as contributing to the learning of interdisciplinary thinking? European Journal of Engineering Education, 40(5), 459–475. DOI: https://doi.org/10.1080/03043797.2014.987647 

  48. Stromholt, S., & Bell, P. (2018). Designing for expansive science learning and identification across settings. Cultural Studies of Science Education, 13(4), 1015–1047. DOI: https://doi.org/10.1007/s11422-017-9813-5 

  49. *Tang, K. (2013). Out-of-school media representations of science and technology and their relevance for engineering learning. Journal of Engineering Education, 102(1), 51–76. DOI: https://doi.org/10.1002/jee.20007 

  50. The Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5–8. Retrieved from http://www.jstor.org/stable/3699927. DOI: https://doi.org/10.3102/0013189X032001005 

  51. *Todd, A. M., Mannheimer-Zydney, J., & Keller, J. (2011). Developing an online learning community for engineering, cooperative education students: A design-based research study. Journal of Cooperative Education & Internships, 45(1), 67–79. https://www.ceiainc.org/wp-content/uploads/2017/08/JCEIA_Vol45_Issue01_100311.pdf. 

  52. Van den Beemt, A., MacLeod, M., Van der Veen, J., Van de Ven, A., van Baalen, S., Klaassen, R., & Boon, M. (2020). Interdisciplinary engineering education: A review of vision, teaching, and support. Journal of Engineering Education, 109(3), 508–555. DOI: https://doi.org/10.1002/jee.20347 

  53. Van den Akker, J., Gravemeijer, K., McKenney, S., & Nieveen, N. (2006). Educational design research. Routledge. DOI: https://doi.org/10.4324/9780203088364 

  54. Wang, F., & Hannafin, M. J. (2005). Design-based research and technology-enhanced learning environments. Educational Technology Research, and Development, 53(4), 5–23. DOI: https://doi.org/10.1007/BF02504682 

  55. *Weber, N. R., Strobel, J., Dyehouse, M. A., Harris, C., David, R., Fang, J., & Hua, I. (2014). First-year students’ environmental awareness and understanding of environmental sustainability through a life cycle assessment module. Journal of Engineering Education, 103(1), 154–181. DOI: https://doi.org/10.1002/jee.20032 

  56. *Yueh H. P., Chen, T., Lin, W., & Sheen, H. (2014). Developing digital courseware for a virtual nano-biotechnology laboratory: A design-based research approach. Journal of Educational Technology & Society, 17(2), 158–168. https://www.j-ets.net/collection/published-issues/17_2 

  57. Yueh, H.-P., & Sheen, H.-J. (2009). Developing experiential learning with a cohort-blended laboratory training in nano-bio engineering education. International Journal for Engineering Education, 25(4), 712–722. 

  58. Zheng, L. (2015). A systematic literature review of design-based research from 2004 to 2013. Journal of Computers in Education, 2(4), 399–420. DOI: https://doi.org/10.1007/s40692-015-0036-z 

comments powered by Disqus