Quantitative research workshop

Quantitative research has always
been a challenging topic in my teaching of research methods. Quite a number of
my students are innately disinclined to deal with numbers and find statistical
analysis uninteresting and even unnerving. So every time I cover quantitative research,
I always start with this quote from Earl Babbie’s The Practice of Social
Research:

““Empirical research is first and
foremost a logical rather than a mathematical operation. Mathematics is merely
a convenient and efficient language for accomplishing the logical operations inherent
in quantitative data analysis. This textbook is not intended to teach you
statistics or torture you with them. Rather, I want to sketch out a logical
context within which you might learn and understand statistics.”

It is important that students understand
statistics are just a means to an end – a tool that we can use to help us accomplish
our research objective by making sense of quantitative data. In my teaching, I
focus primarily on the conceptual understanding of statistics. Students are
expected to master what the frequently used statistical measures are, when to
use them, for what types of variables and what kinds of analytic objectives, and
don’t have to worry about the computational process. They can explore that on
their own – there are many tutorials on YouTube.

At this year’s SCELC Research Day, I gave a
workshop on quantitative data analysis, and this “conceptual understanding”
strategy seemed to have worked well. We had interesting discussions around the
tables and it’s great to see how the librarians planned to use the statistics
covered in the workshop in the analysis of their existing data sets. I just
wish there were more time for us to do some hands on exercises. Nonetheless, it
was a great experience talking about research methods with librarians, as always.

Determining the proper research methods (2)

Once we know whether we know what type of research we will be focusing one (qualitative vs. quantitative), the next step is to figure out the specific method for data collection. For quantitative research, the frequently used methods include survey, quantitative observations, and quantitative content analysis. There are different factors to consider, but the characteristics of each method carry the most weight. For example, we are conducting a study to find out if there’s any relationship between library users’ socio-economic status and their reading behavior. The socio-economic status is defined by annual income, and the reading behavior is indicated by the number of books they check out, the genres of the books, and check-out frequency, etc. We may distribute survey to all library users and ask them to provide information on these variables, but mail survey could be costly, and online survey often gets a very low response rate. Besides, it’s always imposing when we send out survey questionnaires. So, is there any other method that could overcome these limitations and get us the same data? One alternative is quantitative content analysis. If we have access to all library users’ library records, we can anonymize them, and then look at each user’s address (where people live could be an indicator of their level of income, and therefore socio-economic status) and their borrowing record. This method is completely unobtrusive, and will give us all the information we need on the two variables in the study, and maybe more accurately than a survey would because self-reported data is not always 100% precise (there might be memory lapses, incorrect interpretation of the question, etc.)

When it comes to qualitative research, the popular methods include field observations, focus group interview, and qualitative interview. Again, it’s important that we understand the affordance of each method in order to determine which one suits our needs best. For example, a library has a new teen space and we would like to find out how teenagers are using this space. We could go to the teen space and observe what teen library users are doing there (reading books, resting on the comfy couch, meeting in groups, etc.). Being a complete observer helps us maintain our objectivity, but the detachment from the phenomenon we are studying makes it difficult to get an in-depth understanding. So, we could invite teenagers to participate in focus group interviews and talk about how they are using the space and what they think about it. Such interview will surely provide us with a more in-depth view of the use of the teen space, but guiding and moderating a focused conversation is not easy and takes a lot practice and if not done properly, it would result in biased data.

Now we know that understanding each data collection method’s characteristics (advantages, disadvantages, etc.) is the most important factor in determining which method to use in our own study. There are also other factors, such as our access to the study population (how can we sample), the demographic of our study population (if we are studying children, we need to be super careful about surveying/interviewing them), and constraints like budget (do we have the money to provide incentives) and time frame (how much time do we have to complete the study).

(to be continued)

(picture source http://alephunky.deviantart.com/art/Choose-your-heart-190249846)

Determining the proper research methods (1)

In early August, our Gateway PhD students will come to San Jose for a one-week residency, and I was asked to give a talk to them about choosing proper research methods for their studies. To prepare for this talk, I will be sorting out my thoughts on this blog, and hopefully I will find the best way to cover this complex topic in the thirty minutes I’m given.

We all know that there are two types of research – quantitative and qualitative. Quantitative research seeks to describe observations of a phenomenon in quantitative measures, and results of quantitative research are usually numerical represented. Qualitative research, on the other hand, defies quantification and it captures the nuanced details of a phenomenon that cannot be observed by quantitative methods. Which type of research to pursue has everything to do with the nature of one’s research topic and research problem.

For example, we are studying people’s attitudes toward a new library policy, and we may approach it quantitatively. We may administer a survey among library patrons. On the survey, there are five statements representing different attitudes toward the policy, and patrons are asked to select the one they most agree with. Findings of the study can be described via measures like frequency distribution, mode, or even correlational measures (e.g. the relationship between demographic variables and the statement choice). For the same topic, we may also approach it qualitatively. Instead of using the survey instrument, we gather patrons in the library conference room to conduct focus group interviews. This means of inquiry will give us an in-depth view of their attitudes toward the policy, which will be a much fuller view than what the five statements can cover.

Now we are at a dilemma – which type of research should we engage in? Well, we need to go back to our original problem – people’s attitude toward the new library policy. How do we operationally define the variable “attitude toward the policy”? We may ask questions like – has there been any research about the new policy? Do we know enough about this policy to generate an exhaustive list of attributes for the variable (e.g. a list of statements to describe every possible attitude)? Is it our goal to find out how many people have what attitudes, or do we just want to understand how exactly patrons respond to this policy?

As you can see, these questions are helping us decide whether we want to pursue this topic deductively or inductively. The deductive approach allows us to go from general to specific – that is, we have a general theory, and we want to test it out in specific cases. The inductive approach is the other way around – we go from specific to general, and we make observations of specific cases and draw conclusions from that. So, if our answers to the above questions are – yes we do know enough to general an exhaustive list of attributes for the variable “attitude” and we do want to find out how many library patrons have what kinds of attitudes, it means we are approaching the topic deductively and should engage in quantitative research. On the other hand, we may approach the topic inductively and pursue qualitative research to find out what exactly are people’s attitudes toward the library policy.

(to be continued)

Books about Research Methods (3)

This book I’m introducing here, Edward Tufte’s “Visual Explanations: Images and Quantities, Evidence and Narrative”, is actually not about research methods. However, it is still relevant as it talks about how to visually display data and information, which can be results of research studies. Reporting and presenting research is as important as conducting research. After all, one of the goals of doing research is to inform practice via the dissemination of the findings. So, the topic of reporting and presenting of research definitely deserves a place in any research methods class.

I first came to know about Tufte’s book when I was still a doctoral student. My officemate Ron strongly recommended it to me. It was indeed a good and informative read – I enjoyed the thorough discussion of how to use charts and graphs to effectively display numerical information. The most impressive part of this book was the chapter on the Challenger disaster in 1986. The engineers had concerns about the launch, but failed to communicate their worries to NASA due to ill-designed graphics. Tufte reconstructed the data and produced convincing visual display suggesting the launch should have been postponed.

This book is very helpful for people who do a lot of quantitative research. It helps us understand there are creative and yet effective ways to present the seemingly boring numerical data. I have two presentations coming up in August, and both of which are about some quantitative studies I did. So I guess it’s time for a refresher read of this book.