Determining the proper research methods (3)

In the previous two posts, we looked at how we could choose a proper research method for our study needs, specifically, the qualitative or quantitative instruments we can use to gather data. But a research study’s methodological design is more than determining the specific data collection instruments. There is a broader consideration – the overall framework of the design. This framework could be a particular type of research design or a research paradigm that provides guidance for the data collection process.

For example, we are conducting a study to evaluate the impact of a new pedagogical approach on learning outcomes, and we may consider the experimental/semi-experimental design. In our study, we have an independent variable – new pedagogical approach, and a dependent variable – learning outcomes. We can treat the independent variable as the experimental stimulus, and see what happens when it’s present/absent. Given our resources, we may further consider whether a classical experiment is possible, or we should use semi-experimental designs like the time-series design or using non-equivalent control groups. Once we are clear about the design, we can further figure out the specific data gathering methods – methods we can use to collect data that measures the two variables.

Another example – let’s say we are interested in finding out what competencies are essential for reference librarians. Survey could be an appropriate method to use – we can send out surveys to reference librarians and ask them to self-report the competencies they consider essential to their work. But what if we are hoping to generate a consensus from the study, where the list of competencies is agreed upon by the study participants? In that case, we may use the Delphi study design as the design framework to guide the data collection process – the first step is to select a panel of experts (e.g. established authors in journals like RUSQ, leaders of the profession such as the head of RUSA), and then, we administer rounds of surveys among them to reach a consensus on what competencies are essential for reference librarians.

In qualitative research, there are numerous research paradigms that we can use as the framework of methodology –case study, grounded theory, participatory action research, institutional ethnography, ethnomethodology, and naturalism. These different paradigms represent different approaches to qualitative research. There are no specific methods attached to these paradigms. The distinctions are epistemological, having to do with what data mean, regardless of how they are collected.

Now it’s time to conclude the series of posts on determining the proper research methods – from the overall design to specific data collection methods, we need to understand both our own study needs/constraints and the characteristics and affordance of each method/design in order to make the best decision.

Determining the proper research methods (2)

Once we know whether we know what type of research we will be focusing one (qualitative vs. quantitative), the next step is to figure out the specific method for data collection. For quantitative research, the frequently used methods include survey, quantitative observations, and quantitative content analysis. There are different factors to consider, but the characteristics of each method carry the most weight. For example, we are conducting a study to find out if there’s any relationship between library users’ socio-economic status and their reading behavior. The socio-economic status is defined by annual income, and the reading behavior is indicated by the number of books they check out, the genres of the books, and check-out frequency, etc. We may distribute survey to all library users and ask them to provide information on these variables, but mail survey could be costly, and online survey often gets a very low response rate. Besides, it’s always imposing when we send out survey questionnaires. So, is there any other method that could overcome these limitations and get us the same data? One alternative is quantitative content analysis. If we have access to all library users’ library records, we can anonymize them, and then look at each user’s address (where people live could be an indicator of their level of income, and therefore socio-economic status) and their borrowing record. This method is completely unobtrusive, and will give us all the information we need on the two variables in the study, and maybe more accurately than a survey would because self-reported data is not always 100% precise (there might be memory lapses, incorrect interpretation of the question, etc.)

When it comes to qualitative research, the popular methods include field observations, focus group interview, and qualitative interview. Again, it’s important that we understand the affordance of each method in order to determine which one suits our needs best. For example, a library has a new teen space and we would like to find out how teenagers are using this space. We could go to the teen space and observe what teen library users are doing there (reading books, resting on the comfy couch, meeting in groups, etc.). Being a complete observer helps us maintain our objectivity, but the detachment from the phenomenon we are studying makes it difficult to get an in-depth understanding. So, we could invite teenagers to participate in focus group interviews and talk about how they are using the space and what they think about it. Such interview will surely provide us with a more in-depth view of the use of the teen space, but guiding and moderating a focused conversation is not easy and takes a lot practice and if not done properly, it would result in biased data.

Now we know that understanding each data collection method’s characteristics (advantages, disadvantages, etc.) is the most important factor in determining which method to use in our own study. There are also other factors, such as our access to the study population (how can we sample), the demographic of our study population (if we are studying children, we need to be super careful about surveying/interviewing them), and constraints like budget (do we have the money to provide incentives) and time frame (how much time do we have to complete the study).

(to be continued)

(picture source http://alephunky.deviantart.com/art/Choose-your-heart-190249846)

Determining the proper research methods (1)

In early August, our Gateway PhD students will come to San Jose for a one-week residency, and I was asked to give a talk to them about choosing proper research methods for their studies. To prepare for this talk, I will be sorting out my thoughts on this blog, and hopefully I will find the best way to cover this complex topic in the thirty minutes I’m given.

We all know that there are two types of research – quantitative and qualitative. Quantitative research seeks to describe observations of a phenomenon in quantitative measures, and results of quantitative research are usually numerical represented. Qualitative research, on the other hand, defies quantification and it captures the nuanced details of a phenomenon that cannot be observed by quantitative methods. Which type of research to pursue has everything to do with the nature of one’s research topic and research problem.

For example, we are studying people’s attitudes toward a new library policy, and we may approach it quantitatively. We may administer a survey among library patrons. On the survey, there are five statements representing different attitudes toward the policy, and patrons are asked to select the one they most agree with. Findings of the study can be described via measures like frequency distribution, mode, or even correlational measures (e.g. the relationship between demographic variables and the statement choice). For the same topic, we may also approach it qualitatively. Instead of using the survey instrument, we gather patrons in the library conference room to conduct focus group interviews. This means of inquiry will give us an in-depth view of their attitudes toward the policy, which will be a much fuller view than what the five statements can cover.

Now we are at a dilemma – which type of research should we engage in? Well, we need to go back to our original problem – people’s attitude toward the new library policy. How do we operationally define the variable “attitude toward the policy”? We may ask questions like – has there been any research about the new policy? Do we know enough about this policy to generate an exhaustive list of attributes for the variable (e.g. a list of statements to describe every possible attitude)? Is it our goal to find out how many people have what attitudes, or do we just want to understand how exactly patrons respond to this policy?

As you can see, these questions are helping us decide whether we want to pursue this topic deductively or inductively. The deductive approach allows us to go from general to specific – that is, we have a general theory, and we want to test it out in specific cases. The inductive approach is the other way around – we go from specific to general, and we make observations of specific cases and draw conclusions from that. So, if our answers to the above questions are – yes we do know enough to general an exhaustive list of attributes for the variable “attitude” and we do want to find out how many library patrons have what kinds of attitudes, it means we are approaching the topic deductively and should engage in quantitative research. On the other hand, we may approach the topic inductively and pursue qualitative research to find out what exactly are people’s attitudes toward the library policy.

(to be continued)

Books about Research Methods (3)

This book I’m introducing here, Edward Tufte’s “Visual Explanations: Images and Quantities, Evidence and Narrative”, is actually not about research methods. However, it is still relevant as it talks about how to visually display data and information, which can be results of research studies. Reporting and presenting research is as important as conducting research. After all, one of the goals of doing research is to inform practice via the dissemination of the findings. So, the topic of reporting and presenting of research definitely deserves a place in any research methods class.

I first came to know about Tufte’s book when I was still a doctoral student. My officemate Ron strongly recommended it to me. It was indeed a good and informative read – I enjoyed the thorough discussion of how to use charts and graphs to effectively display numerical information. The most impressive part of this book was the chapter on the Challenger disaster in 1986. The engineers had concerns about the launch, but failed to communicate their worries to NASA due to ill-designed graphics. Tufte reconstructed the data and produced convincing visual display suggesting the launch should have been postponed.

This book is very helpful for people who do a lot of quantitative research. It helps us understand there are creative and yet effective ways to present the seemingly boring numerical data. I have two presentations coming up in August, and both of which are about some quantitative studies I did. So I guess it’s time for a refresher read of this book.

Phenomenology and LIS research

Phenomenology is a philosophy that qualitative researchers use to guide their own research. There are three schools of this phenomenology:

Edetic or descriptive phenomenology, guided by the work of Husserl
Hermeneutics, also referred to as interpretive or existential phenomenology, guided by the work of Heidegger and Gadmaer
The Dutch (Utrecht) school of phenomenology, which combines descriptive and interpretive phenomenology and raws on the work of Van Manean and Others.

An article I recently read (Dowling, M. & Cooney, A. (2012). Research approaches related to phenomenology: Negotiating a complex landscape. Nurse Researcher, 20(2), 21-27.) provided a detailed explanation of the three different perspectives and research approaches, and how they should be applied by nurse researchers.

One key difference between Husserl’s descriptive phenomenology and Heidegger’s interpretive phenomenology is that Husserl believed that a phenomenon and its essence can be objectively studied, requiring researchers to bracket out their perceived reality of world, whereas Heidegger rejected the notion of bracketing, claiming that a researcher cannot separate description from his her own interpretation. This difference bears resemblance to what distinguishes positivism from interpretivism. Researchers (at least nurse researchers nowadays) are trying to find the middle ground and focus more on understanding the reality of their experiences to the person as they engage with the phenomenon rather than the more objective reality of the nature of the phenomenon itself. In other words, phenomenology is most useful when the task at hand is to understand an experience as it is understood by those who are having it.

Reading this article made me want to find out how phenomenological research is done in LIS. I did a keyword search in Library Literature & Information Science Fulltext and came up with 32 results. Some were book reviews, some were reflection pieces and some others were empirical studies. Here is the list of citations – it will be very helpful if I ever teach a course on qualitative research and need to put together a list of class readings.

Burns, C., & Bossaller, J. (2012). Communication overload: a phenomenological inquiry into academic reference librarianship. Journal Of Documentation, 68(5), 597-617.
Chen, K., & Huang, I. (2012). Library Use by Medical Students Engaging in Problem-based Learning: A Taiwanese Case Study. Libri: International Journal Of Libraries & Information Services, 62(3), 248-258.
Budd, J. (2012). Phenomenological Critical Realism: A Practical Method for LIS. Journal Of Education For Library & Information Science, 53(1), 69-80.
Klentzin, J. (2010). Collective Success: A Phenomenological Case Study of Ohio Public Libraries. Public Library Quarterly, 29(4), 293-319.
Budd, J. M., Hill, H., & Shannon, B. (2010). Inquiring into the Real: A Realist Phenomenological Approach. Library Quarterly, 80(3), 267-284.
Hultgren, F. (2013). The stranger’s tale: information seeking as an outsider activity. Journal Of Documentation, 69(2), 275-294.
Veletsianos, G., & Kimmons, R. (2013). Scholars and faculty members’ lived experiences in online social networks. Internet & Higher Education, 1643-50.
Pietras, M., & Robinson, L. (2012). Three views of the “musical work”: bibliographical control in the music domain. Library Review, 61(8/9), 551-560.
Stephens, M. (2008). The Pragmatic Biblioblogger: Examining the Motivations and Observations of Early Adopter Librarian Bloggers. Internet Reference Services Quarterly, 13(4), 311-345.
Budd, J. M. (2008). Cognitive Growth, Instruction, and Student Success. College & Research Libraries, 69(4), 319-330.
Templeton, T. (2008). Placing the Library: An Argument for the Phenomenological and Constructivist Approach to the Human Geography of the Library. Library Quarterly, 78(2), 195-209.
Julien, H., & Hoffman, C. (2008). Information Literacy Training in Canada’s Public Libraries. Library Quarterly, 78(1), 19-41.
Antell, K., & Engel, D. (2006). Conduciveness to Scholarship: The Essence of Academic Library as Place. College & Research Libraries, 67(6), 536-560.
Dalbello, M. (2005). A Phenomenological Study of an Emergent National Digital Library, Part I: Theory and Methodological Framework. Library Quarterly, 75(4), 391-420.
Brown, J., & Duke, T. (2005). Librarian and faculty collaborative instruction: A phenomenological self-study. Research Strategies, 20(3), 171-190.
Limberg, L., & Alexandersson, M. (2003). The School Library as a Space for Learning. School Libraries Worldwide, 9(1), 1-15.
Watson, J. (2001). Making sense of the stories of experience: methodology for research and teaching. Journal Of Education For Library & Information Science, 42(2), 137-148.
Horn, J. (1998). Qualitative research literature: a bibliographic essay. Library Trends, 46(4), 602-615.
Saab, D. J., & Riss, U. V. (2011). Information as ontologization. Journal Of The American Society For Information Science & Technology, 62(11), 2236-2246.

Books about Research Methods (2)

One of my favorite professors in my PhD program, Dr. Barbara Wildemuth, published a research methods book in 2009, titled “Applications of Social Research Methods to Questions in Information and Library Science”. This book divides the chapters into different stages of conducting research – formulating the research question, establishing the research and sampling design, collecting data, and analyzing data. Each chapter focuses on a particular design (e.g. case studies, longitudinal studies) or a specific method (e.g. semi-structured interviews, qualitative content analysis). What I like most about this book is that for each chapter, two to three exemplar studies using the research design or method discussed in that chapter were carefully critiqued, which provides a contextual and concrete way to understand how to successfully employ the design/method in a research study.

The book is intended for a wide variety of audience – doctoral/master’s students that are learning to conduct research, practitioners that are interested in carrying out research studies, and experienced researchers that are considering a method they are not familiar with. I use several of its chapters as readings for my own research methods class and students often find them helpful. I use it as a reference book for my own research as well.

Oh, by the way, I co-wrote two of the chapters in this book. Maybe I should’ve opened with that? 🙂

QQML 2013

Last week I was in Rome attending the 2013 International Conference on Qualitative and Quantitative Methods in Libraries (QQML), where 288 papers from all over the world were presented, covering a wide variety of topics in LIS. I had a great time talking to librarians, educators and researchers from different countries and learning about their research. Two of my old friends from Chapel Hill were also there – it’s been six years since I last saw them, and it was just wonderful to see them again.

The papers that interested me most were the ones about library assessment and valuation. Recently I have been thinking about developing a course entirely on library assessment. This topic has been a component in many courses in our program, but I think it is important enough to spend a full course on it. More and more libraries, especially academic libraries are facing increasing pressures to establish their relevance and value, and assessment is critical in this process.

Here are a few papers about library assessment that I enjoyed:

A survey study to examine students’ use of an academic library, its resources and spaces, and their engagement and persistence at UT Austin (by Meredith Taylor).
A project using the Understanding Library Impacts (ULI) protocol to assess library contributions to undergraduate student learning at Barnard College (by Derek Rodriguez and Lisa Norberg).
A study using the MISO survey to assess the quality and effectiveness of library services (by David Consiglio, Katherine Furlong and Gentry Holbert).
The single or joint use of qualitative and quantitative metrics to support library accreditation efforts, funding requests and assess user needs and expectations ( by Michael Maciel).
The best practices of data visualization and examples of how some of them have been applied in libraries (by Rachel Besara).

There are also a couple of papers about health information access that caught my attention. This is an area that I wish to further explore. I just learned this morning, my first paper in this area, “Preparing Public Librarians for Consumer Health Information Service: A Nationwide Study”, co-authored with Dr. Van Ta Park, has been accepted to publish in Library and Information Science Research. Van and I will also discuss ideas of developing grant proposals together to fund our collaborative research pursuit down the path of health information.

Overall, I had a great time at QQML 2013. Now with all the ideas and inspirations grained from the conference, I’m going to have a very busy summer.

Students’ research ideas from LIBR 285 Research Methods in LIS

The spring semester is officially over. There is one thing I always do before diving into the joy of summer – gathering the research ideas my students had in the Research Methods class and putting them into a list. I used to do this in an Excel file, but now, documenting them on this blog seems a better idea. So here we go:

What barriers exist that inhibit parents of elementary school children from utilizing public library materials and services?
What motivates patrons to use tagging features in library catalogs? Do different tagging interfaces make a difference?
What connection, if any, does the age of a school library collection have upon student use of the library, both in terms of visits and circulation? In addition, are there specific genres/subject areas in which currency has a greater impact?
Are public libraries attracting new digital users through mobile library services?
Do academic libraries lend more support to group studying, or to individual studying, or do they support the two types of studying equally?
Do LGBTQ young adults have access to LGBTQ collections and programs in San Francisco Bay Area public libraries?
How does library jargon on reference desk signage affect student recognition of in-person reference services at the CSULB Library?
What are best practices for developing and administering Drupal user permissions and roles for library websites?
What are the challenges users face when browsing and transferring library ebooks into their ereaders?
How does the content and visibility of online Interlibrary Loan signage impact the success of library patrons in using ILL services?
What are key factors in the success of animal-assisted literacy programs offered at public libraries?
How do California public librarians use information gathered from social cataloging sites in their reader’s advisory programs?
How effectively does the Oakland Public Library meet the technology needs of Oakland teens?
What is the level of accuracy in the mapping of interface terminologies from selected fields in electronic health record systems to SNOMED CT?
How do library patrons’ emotions and thought processes affect their choice of mode of reference service, specifically between face-to-face reference service and chat reference service?
What are the nature, size, and accessibility of the University of California and California State campus radio archives?
Does pictogram signage improve patron access to the collection in the elementary school library?
Does allowing food and drink increase the number of people coming to the library?
Do reference librarians yield the same results when given the same research question face-to-face vs. an asynchronous form?
What are attorneys’ perceptions of their law library transitioning to an entirely electronic library?
Do professionally staffed elementary school libraries have an impact on students’ reading achievement?
How does the library staff benefit from a full-time social worker at the SFPL?
What comparisons and contrasts can be drawn between the respective impacts of children’s and teens’ participation in variously structured public library-run summer programs on teen reading and library use habits?
What combination of skills, experience, and academic qualifications are required to get a job as an entry-level special librarian in a hi-tech company?

As always, a wonderful array of research questions seeking to tackle various kinds of important problems in our profession. They each designed a study based on their respective research question, and wrote a research proposal about it. Hopefully some of them will actually carry it out!

(photo from http://www.ksos.kit.edu/img/IDEAS_Design.JPG)

The Research Methods Course – An opportunity to tackle real-world problems

One of the courses I teach regularly is “Research Methods in Library and Information Science”. The section I teach is general-purposed and covers the frequently used research methods in the field of LIS. In the meantime, the course has other sections that focus on special topics, and students are free to choose based on their interest. The specializations include different types of research, such as qualitative research or action research, or different LIS domains, such as research in reference and information services or youth services. Students with different backgrounds and pursuits may choose accordingly to fulfill their needs. For example, students who intend to seek positions as children or young adults’ librarians, may elect the section that introduce them to the theory and methods of planning and evaluating youth services (children and young adults). Those who have an interest in working in archives may elect the section that covers theory and methods of historical research and writing. In addition, the research methods course can be taken twice for two specializations so students can expand their repertoire of research knowledge and skills.

A major assignment in the course is to develop a research proposal, where students need to identify a research problem, formulate a research question, and design a research study to answer the research question. I always position the research methods course as a “problem-solving” course that will equip students with skills/techniques to solve actual problems they encounter at work. And this proposal assignment manifests how the skills/techniques can be put to work in a hypothetical situation. Well, not entirely hypothetical. I encourage students to talk to practitioners and see if they have any problems that could benefit from a research study, and some of them have developed their research proposals based on real problems faced by librarians. For example, after talking to Oakland public librarians, a student is proposing a study to assess Oakland teens’ technology needs and how they are met by the local public library, because this is what the librarians want to find out.

I think the research methods course presents a good opportunity for students to understand the kinds of issues and problems that original research can help tackle. But for now I haven’t done more than encouraging them to talk to practitioners to identify practical problems that they can propose a research study to solve. I’m sure there’s more that I can do, and I just need to figure out what and how. To be continued.