How to critically review the methods chapter of a dissertation

If we plan to earn a doctoral degree, we usually need to write a dissertation as part of the requirements. A dissertation is a report of large research project we implement with the approval of our dissertation chairperson and committee. Before we can begin collecting data for our project, we must write a proposal in which we describe our plan to complete our project. In the social sciences, such proposals often have three chapters.

In Chapter 1, we state the problem we propose to study, explain the background and purpose of our project, and list the questions we hope to answer with our findings. In Chapter 2, we review what other scholars have studied and try to show the gap in knowledge we hope to fill with our study. In Chapter 3, we describe our plan for accomplishing our study—this is the methods chapter.

What sections are in a methods chapter?

The methods chapter is our research plan, in which we describe in detail how we intend to carry out our project. The methods chapter of a social sciences dissertation follows a predictable format. For example, the for-profit institution I attended required the following sections (no more and no less):

Introduction (in this section, we briefly restate the problem, purpose, and research questions and offer an overview of the chapter)
Research design (what methodology have we chosen: quantitative, qualitative, or both—and why?)
Population and sample (who or what are we studying? How many are we going to study?)
Materials and instrumentation (what questions, what surveys, what protocols, what experimental equipment are we going to use to study our sample?)
Operational definitions of variables (usually for quantitative studies only, what variables are we trying to assess and how are we defining them so we can measure them?)
Procedures (what exactly are we going to do? This is an overview of the entire research plan)
Data collection and analysis (specifically, how will we collect and analyze data?)
Assumptions (what are we assuming is true about the topic, population, and research plan?)
Limitations (what could keep us from being entirely certain that we’ve measured what we planned to measure?)
Delimitations (what parameters have we set to define the scope of our study?)
Ethical assurances (what is our plan for protecting our research subjects and their data? How will keep our own preconceptions and biases from interfering with our conclusions?)
Summary (what were the main points of the chapter?)

Why is the methods chapter important?

The methods chapter is the blueprint for our study. The methods chapter should provide enough detail that another researcher could replicate the study to validate our findings. A major reason why chairs and committees reject dissertation proposals is because the methods chapter fails to provide sufficient detail.

Why should we review methods chapters in other dissertations?

Two reasons: We will learn what works, and we will learn how to improve our methods chapters so we are more likely to earn approval.

For some examples, locate some dissertations recently published in your field that use methods you are considering for your own project. Dissertators often disclose what methods they used in the titles of their dissertations. Keywords include quantitative, qualitative, phenomenological, case study, experiment, ethnographic, and narrative.

If you attend a for-profit university, select some dissertations from your institution as well as some from other for-profits and nonprofits (both public and private) so you can discern differences between documents. If you attend a public or private nonprofit university, select some dissertations from institutions comparable to your own (excluding for-profit universities). After you have read ten or so dissertations on topics similar to yours, using methods similar to yours, you will have a sense of how the elements should appear in the methods chapter.

How do we critically review a methods chapter?

Before you start, create a matrix or spreadsheet to help you keep track as you review the documents. You can assign points or grades as you go through the following checklist. Your subjective ability to evaluate the quality of each methods chapter will evolve and sharpen as you read more dissertations.

In fact, it may please you to see how many dissertators failed to follow the university template. (Don’t you do that, at least not for your proposal). You will see, the hoop we have to jump through is bigger than we might think!

Remember, we are reviewing published dissertations. Can you tell how much of the discussion was from the proposal? (Hint: Watch for future tense when it should be past tense—for example, “The research design selected for this study will be qualitative phenomenology.” Be aware, when it’s your turn: It’s easy to forget to revise future tense to past tense, and editors don’t always catch this problem.)

First, find the document’s table of contents and look for the methodology/methods chapter. It may be entitled Research Design and Methodology. It might be called simply Methods. It often is the third chapter of a proposal but depending on the institution and the field of study, it could be another chapter.

Next, for each section of the methods chapter, follow these steps:

Introduction. Did the dissertator briefly restate the problem and purpose, present the research questions, and preview the sections in the chapter? Many authors exclude this important information. It might seem repetitive, but orienting readers at the start of the chapter can help them follow along as we present our argument.
Research design. The research design sets the stage for the details of the research plan. First, was a discussion of research design present? (If not, whoopsie.) If you find it, or something similarly named, read that section critically, considering the following questions:

How much of the discussion was culled from Creswell and the like? We want some detail about the nature of the research design, but not too much. We want to avoid sounding like a textbook discussion of research design. In our proposals, we should briefly mention what the experts recommend and then explain what we plan to do and why.
Along those lines, how long did it take the dissertator to get around to actually identifying and describing his or her research design? Did you get the feeling there were more words than there needed to be about irrelevant topics? (Were you bored and losing interest within a couple paragraphs?)
Did the dissertator clearly explain why that design was selected? That is, did he or she justify the choice clearly? When I say clearly, I mean explicitly: “I chose this design because…” Readers appreciate clarity. Don’t make them search for clues!

Population and sample. This section covers details about the population from which the sample was drawn and shows how the sample was selected. Consider these questions:

How many paragraphs did it take the dissertator to state what he or she actually did to find and recruit the sample? Or did it seem like the discussion was mostly about what a sampling plan is and what it means to have a sampling plan? (Rather than explaining the actual sampling plan?) Don’t waste valuable space reiterating textbook discussions. Get right in there and describe what you plan to do.
Did the dissertator state the size and location of the overall population? Was the population explicitly defined and described in terms of all relevant characteristics?
How big was the sample? How was the sample chosen? Did the dissertator describe the sampling process in enough detail that you could replicate this study? If not, their blueprint was inadequate.

Materials and instrumentation. This section covers the practical tools and methods the dissertators used to implement their studies.

For quantitative social science research projects, consider these questions:
- Did the dissertator choose an experimental design? If so, what materials or special equipment were used?
- What instruments did the dissertator use to measure the variables? Were they existing instruments (i.e., designed by someone else and used with permission), or were they designed by the dissertator (and were they validated)?
- How did respondents generate data—did they go online and access a Survey Monkey survey? Did they check boxes on a piece of paper with a pencil?
- Were the procedures clearly explained so you could replicate this study?
For qualitative social science research projects, consider these questions:
- Did the dissertator present an interview protocol or focus group discussion guide?
- Did the dissertator use a video camera or an audio recorder?
- Did the dissertator observe behavior or analyze secondary data? What forms, tools, or special equipment were used?
- Were the procedures clearly explained so you could replicate this study?

Operational definitions of variables. This section applies mainly to quantitative studies, although some qualitative researchers might define the constructs they sought to measure. Consider these questions:

Were the operational definitions clear, succinct, and replicable?
For hard-to-measure constructs (e.g., justice, leadership), were the important facets of each variable included in the definition? Did it make sense to you? Did the dissertator leave anything out?
Was this section brief and to the point, maybe presented with a table? Was there too much discussion? Did it seem like a repeat of the instrumentation discussion?

Data collection and analysis. Consider these questions:

Did the dissertator write a lot about the generic type of data analysis processes one could use and not so much about the processes he or she actually did use? That is, does it read like a textbook discussion of data analysis processes?
Are the details of the data collection process clear? We expect some overlap with other sections. Are the details consistent in all the sections? (You’d be amazed at how often dissertators write one thing in the instruments section and something else in the data collection section.)
Is the data analysis plan clear? Could you replicate them? Do they make sense?
For quantitative studies, did they mention the types of data they collected (nominal, ordinal, interval, or ratio?) Did they use appropriate analyses techniques? Did they omit descriptions of the validation techniques they used?
For qualitative studies, did the dissertator describe the process he or she used to code text or images? Did the description make sense? Could you replicate the process or was it a vague claim of coding, categorizing, and magically arriving at themes?

Assumptions, limitations, and delimitations. Consider these questions:

Were any of these three sections missing? Most for-profit university templates require dissertators include all three sections. Many dissertations published by nonprofit institutions omit the assumptions and delimitations sections. Almost all studies I have reviewed include at least a minimal description of limitations.
If you can find these three sections, were the discussions clear? Did they seem logical?

After you review a few methods chapters, you will start to see the rhythm and alignment of the sections. A solid methods chapter will give you just enough information, no more and no less, so that you could replicate the study yourself. The plan will seem like a logical progression toward answering the research questions (which, of course, is our goal). The actions the dissertators took will seem solid to you, even if you don’t totally understand all the techniques they applied. Their justifications for using those techniques will make sense.

Now you are ready to polish your own methods chapter. Have fun! Let me know how it goes.

=====================================

Find my latest books and resources

Applying Theory

Dissertators often struggle to choose and apply a theoretical framework to their research projects. In this helpful guide, I offer suggestions from my own experience. In addition, I reveal how other dissertators have applied theory successfully and earned their doctorates.

Written in a friendly, nonscholarly manner, I demystify the challenges of applying academic theory to a research project. You will learn that theory is nothing to fear—in fact, we all use theory all the time! With the help of this powerful little book, you will learn to master theory and achieve your dream of earning your Ph.D.

Print version $15.99
Kindle version $7.99

Coming in 2019

Aligning the Elements

In the second book of the Desperate Dissertator Series, I dig into a common problem dissertators face when preparing their proposals: aligning the elements of the dissertation. The elements of the dissertation usually include the problem statement, the purpose statement, the research questions, the theoretical framework, and the methodology and methods. Aligning these elements means ensuring they logically flow from one to the next. Lack of alignment is possibly the main reason proposals are not approved. In this book, I offer some tips to help you align the elements and show what other dissertators have done to succeed (it could be easier than you think).

Resubmit! 28 ½ Reasons Why You Can’t Get Your Dissertation Proposal Approved

This comprehensive book is the missing link for dissertators who have struggled to get their proposals approved. This indispensable book bulges with insights, suggestions, examples, diagrams, and practical tips, written especially for the online dissertator who may receive little support during the proposal process.

I present solutions to address twenty-eight potential reasons why you might be struggling to get your proposal approved. For example, you will learn how to write a clear problem statement, devise research questions and hypotheses, and align the elements of the proposal to facilitate speedy approval. I unlock the mysteries of Word and Excel to show you specifically how to use these tools for your proposals. Over 200 tables and figures show you exactly what to do. As a bonus, you will learn how to design a web-based survey and make a plan for fielding and analyzing the data. In this book, I cover it all to help you overcome obstacles and finish your dissertation.

Free templates and worksheets are available here.

Print version $29.99
Kindle version $9.99

November 27, 2018

Got a dissertation question? Ask Dr. Carol

If you’ve never written a dissertation before, the prospect can be daunting. Novice researchers don’t always understand the process at first. Take heart: We aren’t born knowing how to research and write a dissertation, but we can learn. Here are some questions I’ve received from confused dissertators who are trudging toward their goal of earning a Ph.D. or other advanced degree. (See my disclaimer at the end of this post.)

What do I do while I’m waiting for my proposal to be approved?

Dear Dr. Carol: I am working on my proposal for my Doctor of Business Administration research project. While I’m waiting for approval, should I start working on Chapter 4? –Ready to be done in Washington

Dear Ready: I know you are anxious to keep moving forward with your dissertation. However, it is not time yet to start on Chapter 4. For most social science dissertators, the first three chapters comprise the dissertation proposal. Chapter 4 is where we report our findings. Before we can write Chapter 4, we need to collect some data.

For most of us, before we collect data (that means, before we administer surveys, conduct interviews, or observe people), we must apply to the Institutional Review Board (IRB) and any other entities that might be involved (school districts, companies, and the like) for permission to collect data.

After we receive permission to collect data, we implement the next phase of our study according to the plan we presented in Chapter 3.

So, the answer to your question is no, don’t start on Chapter 4. You can’t move forward until you get your proposal approved. For now, focus on that major milestone. While you are waiting for feedback and approval on your proposal, write a draft of your IRB application. Get current on new literature in your field. Take a nap. Enjoy your friends and family, because once you start collecting data, you might not see them for a while.

Help! My proposal lacks “alignment.”

Dear Dr. Carol: I just got feedback on my dissertation proposal. My chairperson said my paper “lacks alignment.” I don’t know what that means. Can you help? –Tearing out my hair in Denver

Dear Tearing: I hear your frustration. Alignment is a critical concept for researchers, but our mentors often don’t know how to define it, let alone explain it. They can tell if alignment is missing, but that’s not much help when we are trying to figure out what they want.

These are the main elements of most social science proposals:

Problem Statement
Purpose Statement
Theoretical Framework
Research Questions/hypotheses
Methodology/methods

Alignment means all these elements logically flow from one to the next. That is, from the problem we identified emerges the purpose statement. The theoretical framework supports the purpose. The research questions flow from the purpose as well. The methodology and methods are logical, given the purpose statement and the theoretical framework.

Here is an example of lack of alignment among the elements:

The problem is kids watch too much television.
Therefore, the purpose of the study is to observe kids’ consumption of junk food.
The theoretical framework will be based on adult learning theory.
The primary research question will be “How do kids’ ratings of Saturday morning cartoons correlate with their consumption of Twinkies?”
The methodology will be qualitative.
The main data collection method will be to observe kids’ napping behavior after eating sugar.

Does that plan make any sense at all to you? The lack of alignment is grossly exaggerated to make my point, but I’ve seen proposals whose elements are almost this misaligned. It is so easy to get off track.

Alignment is important! By far, the most common problem I see in proposals and dissertations is a lack of alignment between the problem, purpose, research questions, and methodology/methods. Alignment among these elements is such an important requirement that I devote an entire chapter to it in my book.

Go back through your main elements and make sure each one follows logically from the others. Use the same terms in all the elements: The problem is [xyz], therefore the purpose is to study [xyz]. Keep it simple. Read what other dissertators have done. Learn from their experience. Focus on nailing down the problem, and then let the other elements unroll logically from there. If something seems out of alignment, it usually means the problem statement is not clear.

Sometimes when all the elements align, I feel awe at the beauty of research. The elements make sense, logically flowing and interacting, supporting each other, justifying each other. The components come together to create the potential for new knowledge. Thus, we are knowledge generators—our research efforts lead to new insights, some big, some small, each valuable as a step toward new knowledge. Practitioners and scholars study, combine, synthesize, adapt, and extend our insights into new practices, new conclusions, new theories. How cool is that!

How do I choose my dissertation topic?

Dear Dr. Carol: I’m having trouble choosing a topic for my research project. Where do I start? –Confused in Florida

Dear Confused: I hear you. The world is full of fascinating topics to study. How do we choose just one?

First, do these four things:

Identify your program of study.
Make a list of problems that interest you.
Identify problems that need addressing (not every problem is worthy of doctoral-level research).
Review existing research to locate a gap in the literature.

This process should help you develop a list of possible topics, broad and narrow, that might work for a doctoral study.

Next, do the following:

Read Ellis and Levy’s (2008) article: Ellis, T. J., & Levy, Y. (2008). Framework of problem-based research: A guide for novice researchers on the development of a research-worthy problem. Informing Science: The International Journal of an Emerging Transdiscipline, 11, 17-33. Retrieved from http://www.inform.nu/Articles/Vol11/ISJv11p017-033Ellis486.pdf
Review your list of ideas in the Thomson Reuters Web of Science/Web of Knowledge (http://wokinfo.com/). You will usually have access to this tool through your university’s online library.
I know you are already doing this, but keep on reading articles in your field. Look at the end of each article for suggestions for future research. Make a list of interesting directions.
Talk to practitioners in your field. What problems do they face? How can you help? Make a list of pressing problems faced by people working in the real world.

In the intersections of these actions, I have no doubt you will identify many interesting topics suitable for a doctoral research study in the social sciences. The hard part now will be choosing just one.

Now think about the two possible research methodologies (qualitative or quantitative). You could do both (mixed methods), but choosing just one methodology is usually the best approach for a novice researcher. What do you prefer, analyzing text or analyzing numbers?

Finally, think about the methods available to you. Do you prefer listening to people, surveying people, observing people, or analyzing secondary data? You can find some tips and suggestions in a free e-guide available on the Love Your Dissertation Resources webpage.

Disclaimer

I hesitated before posting this essay. “Ask Dr. Carol” presumes I’m some kind of expert. At the risk of never editing another dissertation, I confess, I’m not. Even though I’ve edited many proposals and manuscripts, I’ve only written one dissertation.

In the social sciences, we study people’s behaviors, thoughts, attitudes… when we plan our study, we’d like to be certain that our research approach is the best one, but humans are complicated and messy, and so is studying them. Social science research continually evolves as researchers devise and apply new theories and research methods. In other words, research is a process.

We have all read the research gurus; however, if you’ve read successive editions of their books, you know their thinking evolves as they try to hone their explanations of research processes. Those of us who are learning to do research might be confused as the gurus’ thinking changes from year to year, but we need to remember that the research process is flexible, organic, and evolving. That means whether we choose quantitative or qualitative methodology, there is no one correct way to conduct a study.

Are you discouraged? Don’t be. This is good! This abundance of approaches might be confounding for new researchers, but on the bright side, we really can’t fail. As long as we get all the elements logically aligned and solidly justified, our research design will be valid.

I’ve seen many research designs and data collection and analysis methods. I’m sure I’ve seen only a fraction of what is possible. We researchers are a creative bunch. Most of us would like to blaze our own unique trail. For instance, I incorporated rich pictures into my project—I thought it was fun and unique. (You can read all 300+ pages of my massive tome on ProQuest Open Access if you want an example of a novice researcher learning as she goes.)

The point is, we are all on a continuum of research, learning as we go. I’m grateful for the gurus who keep refining their guidance—we who trudge the path behind them benefit from their repeated attempts to clarify and explain complex research methods. We should rejoice that the buffet of research approaches eventually produces a 360° perspective on the topics we study. That is our goal, right? To illuminate and clarify our topics.

As an author, I write to help me understand my topic. If I already understood the topic, what would be the point of writing about it, except to show off how much I know—an exercise in arrogance. If I am already sure of the answers, if I believe no new knowledge can be discovered, then why bother researching anything?

Some people think we already have all the answers, that research is a waste of resources. I don’t subscribe to that belief. Curiosity may have killed the proverbial cat, but it has also given us access to atoms and stars and everything in between. I believe the human historical arc bends toward curiosity. I support your curiosity, in all its myriad expressions.

Got a question?

Got a question about your dissertation proposal, research, or manuscript? Submit your question for Dr. Carol using the contact form.

=====================================

Find my latest books and resources

Applying Theory

Print version $15.99
Kindle version $7.99

Resubmit! 28 ½ Reasons Why You Can’t Get Your Dissertation Proposal Approved

Free templates and worksheets are available here.

Print version $29.99
Kindle version $9.99

October 24, 2018

Get a free review of your dissertation proposal

Have you struggled to get your dissertation proposal approved? Get practical feedback from an expert to help you overcome obstacles so you can achieve your dream of earning a Ph.D.

Free review offer

ARE YOU AN ONLINE LEARNER AT A FOR-PROFIT UNIVERSITY?

Are you satisfied with the attention and feedback you are currently receiving? If most of your academic experience happens online, you may be feeling isolated. Your chairperson and committee may not be providing you with the kind of feedback you need to move forward. Having an outside perspective from a professional editor could help you break through roadblocks. This free review is a good place to start.

This review is free. There is no risk to you. You have nothing to lose and a Ph.D. to gain.

For more information about Dr. Carol M. Booton, click here. For editing services and pricing, click here.