Topic

Appendix 2: Design-Based Research (DBR) Dissertations

The core of the DBR dissertation is the research, development and testing of a product or process to solve an educational problem. Design-based research protocols require intensive and long-term collaboration involving researchers and practitioners. Design-based research integrates the development of solutions to practical problems in learning environments with the identification of reusable design principles. Barab and Squire (2004) defined design-based research as “a series of approaches, with the intent of producing new theories, artifacts, and practices that account for and potentially impact learning and teaching in naturalistic settings” (p. 2). They described it as a methodology that requires: addressing complex problems in real contexts in collaboration with practitioners integrating known and hypothetical design principles with technological affordances to render plausible solutions to these complex problems; and conducting rigorous and reflective inquiry to test and refine innovative learning environments as well as to define new design principles. A research proposal for a doctoral study using a design-based approach must include a practitioner-oriented focus as well as degrees of collaboration that are not necessarily required for more traditional predictive research designs. 

A design-based research proposal consists of the following:

Phase 1: Problem Identification (Chapter 1)

For design-based research in education, the identification and exploration of a significant educational problem is a crucial first step. It is this problem that creates a purpose for the research, and it is the creation and evaluation of a potential solution to this problem that will form the focus of the entire study. Many research students begin by thinking of a solution—such as a technology-based intervention, an educational game, or a technology tool—before they consider the educational problem it could solve. Problems then arise when the solution is revealed to be a project of interest or ‘pet’ project, rather than a genuine attempt to solve an educational problem. The statement of the problem in design-based research should identify an issue or an opportunity, explore its history or background, and provide a convincing and persuasive argument that this problem is significant and worth researching. This includes articulating both the practical and scientific relevance of the study. In line with the exploratory nature of design research, driving questions should therefore be open in nature. The assumptions that direct DBR are derived from the definition of the research problem in close collaboration with practitioners.

Literature Review (Chapter 2)

The research is fine tuned through the literature review that serves to (a) help flesh out what is already known about the problem and (b) to guide the development of potential solutions. In such instances, the inquiry that forms the basis of DBR serves the researcher to help understand the underpinning processes and variables and how they impact on the learning and learning outcomes. A well-described theoretical framework provides a sound basis for the proposed solution, because theory can inform practical design guidelines. Even though they are largely based on the literature, it is unlikely that draft principles will be complete at the time the proposal is presented. At the very least, the process of deriving them should be described and examples given.

Description of proposed intervention (Chapter 3)

The proposed solution to the educational problem is developed from consideration of relevant literature, consultation and collaboration with researchers and practitioners, and as an instantiation of the principles derived from these sources.  It is important to describe in the prospectus or proposal the process of how the intervention will be conceptualized and developed.

Methodology (Chapter 4)

The methodology serves to describe the Iterative cycles of testing and refinement of solutions in practice. Both qualitative and quantitative methods may be used in DBR. A research proposal would include details of the methodology of the implementation and evaluation of the proposed solution, as it largely constitutes the data collection and analysis stages of the study. The proposal should also acknowledge the likelihood, even the desirability in some cases, of significant modifications being required in the data collection and analysis phases of the ongoing study.

Implementation of intervention (First iteration – Alpha stage – prototyping)

The iterative nature of design-based research means that a single implementation is rarely sufficient to gather enough evidence about the success of the intervention and its effect on the problem situation. A typical design-based research study would have two or more cycles, where after the first implementation and evaluation, changes are made to the learning environment to further improve its ability to address the problem. In DBR the context of the inquiry must be seen as a means to an end rather than an end in itself. The intention should be to use the setting to gain an understanding which will have meaning beyond the immediate setting.

Participants

In a research proposal, the description of participants and the method of their selection provide important information for reviewers about the potential for bias in the proposed study. Participants are usually individuals who reflect the characteristics of or are influenced by the issues being considered by the investigation. 

Data collection and analysis

The method of data collection in design-based research can involve the collection of qualitative and/or quantitative data, and it may be collected in cycles of several weeks or semesters, or even years. Types of data collected are likely to vary along with the phases. For example, data contributing to contextual understanding are more likely to be emphasized in earlier stages of the study, whereas data on prototype characteristics or user reactions are more likely to be collected later on. This section of the proposal describes (a) data sources such as varying time, location and participants; (b) data collection methods, including varying formats (interviews, observations, etc.); and (c) analysis approaches to be used with each data type. 

Implementation of intervention—second and third iterations (Beta stage testing iteration and Gamma stage final assessment) 

Although it is impossible to describe the nature of the second and subsequent iterations of the intervention, because they are so totally dependent on the findings of the first iteration, it is useful to describe the process that should be undertaken in the proposal. The cyclic nature of the data collection and analysis cannot be described in great detail in the proposal, but the process of data collection, analysis, further refinement, implementation and data collection (and so on) of the learning environment should be explained as a method in the proposal.

After the Proposal

Once the proposal is approved and IRB acquired, the student will begin collecting data for the Alpha stage which is then analyzed.  Based on those findings, the intervention is adapted and then tested again in the Beta phase. After data collection and analysis in the Beta stage, the intervention is again modified and a final test (Gamma phase) with actual users is conducted.

Findings (Chapter 5) 

In this chapter the researcher presents the findings of each iteration as well as describes the modifications made to the intervention based on the findings of each stage. The specific organization of the findings in Chapter 5 should be decided in consultation with the dissertation chair.

Conclusion (Chapter 6) 

Reflection to produce “design principles” and enhance solution implementation

Design principles

The knowledge claim of design-based research, and one that sets it apart from other research approaches, takes the form of design principles. Design principles contain substantive and procedural knowledge with comprehensive and accurate portrayal of the procedures, results and context, such that readers may determine which insights may be relevant to their own specific settings. In the traditional sense, generalization of design-based research findings is rather limited; instead, use of design principles calls for a form of analytical generalization.

Practical outputs: Designed artifact(s)

In design -based research, the product of design is viewed as a major output . Design artifacts in this field may range from software packages to professional development programs, and many more.

Societal outputs: Professional development of participants

The collaboration that is so integral to the process of defining and accomplishing a design -based research project has an additional benefit to the extent that it enhances the professional development of all involved

Theoretical/Conceptual Framework and Literature Connections

The final chapter should also address how the findings from the research study compare and contrast with expectations based on the literature and the theoretical or conceptual framework used by the researcher.

The figure below shows a typical  DBR process.

Other Content

As in the traditional and three-paper dissertation options, the dissertation will include appropriate Appendices and a full Reference list.

Timeline

Design-based research requires frequent and prolonged periods of fieldwork, off-set by periods of review, reflection and re-design. These intervals should be clearly taken into account in any timeline accompanying the research proposal. A major strength of design research lies in its adaptability, the commitment to adjusting a study’s course based on findings after each iteration.

Committee Review

LTEC has adopted the following criteria taken as the basis for evaluation for a DBR dissertation beyond the metrics for a traditional dissertation:

  • Appraise the intellectual merit of the research and the product/process proposed;
  • Review the contribution to new or existing design principles 
  • Assess the quality and  appropriateness of the  practical solutions proposed for real-world educational problems 

In general, the dissertation chair and committee are responsible for determining the appropriateness and quality of the proposed DBR research.

Format

The format of the final dissertation must meet the style guidelines established by the UHM Graduate Division for theses and dissertations. All LTEC dissertations use APA style guidelines.