Design Thinking and Research Methods

Lately I have been reading about Design Thinking (DT) and its application in libraries. Tim Brown, CEO of IDEO explains that “Design thinking is a human-centered approach to innovation that draws from the designer’s toolkit to integrate the needs of people, the possibilities of technology, and the requirements for business success.” In essence, it’s a design methodology that provides a solution-based approach to solving problems. The Hasso-Plattner Institute of Design at Stanford (d.school) proposed a five-stage Design Thinking model – Empathise, Define (the problem), Ideate, Prototype, and Test. This model actually shares a lot of characteristics of the research process – I often tell students that our research methods course aim to equip them with practical problem solving skills via a valid and reliable research process, that is, a process of arriving at dependable solutions to problems through the planned and systematic collection, analysis, and interpretation of data.

Here’s how the DT process and the research process overlaps:

The first two steps of DT are emphathize and define (the problem), which focuses on determining the problem that needs solving through observing, engaging and empathizing with the target population people to understand their experiences and motivations, and then analyzing the observations and synthesizing them in order to define the core problems in a human-centred manner. This is similar to the research question formulation stage of the research process, where we determine the objective/problem of our research study (e.g. to provide better library services to international students) and develop the question our research seeks to answer (e.g. what are the library needs of international students).

Then, the third step of DT is ideate, where ideas are being generated to identify solutions to the problem. In the research process, in order to generate ideas to solve the problem (e.g. figure out how to provide better library services to international students), we need to choose a proper research method (e.g. survey), devise a meaningful data collection instrument (e.g. the survey questionnaire), and collect data from the appropriate population (e.g. a representative sample of international students).

The fourth step of DT – prototype, is an experimental phase that seeks to identify the best possible solution. The prototypes are investigated and either accepted, improved and re-examined, or rejected on the basis of the users’ experiences. This sort of mirrors the data analysis stage of the research process, where we analyze the collected data and synthesize it to answer the research question (e.g. describing the library needs of international students), and such an answer will tell us what we should do to solve the problem (e.g. based on the needs of international students, what changes we should make to current library resources/services/programs, and what new elements we should add).

In the final stage of DT – test, rigorous testing is conducted to examine the efficacy of the solution, and alterations and refinements are made to continuously improve the solution. In research, especially applied research, it is not uncommon to conduct evaluation of the actions we take to solve practical problems as a result of a research study (e.g. after we implement changes/additions to library services for international students as informed by our research study, we evaluate the efficacy of these changes/additions and to make further refinements if necessary).

So, looks like I can include some readings on DT in my research methods class now. 🙂

For those interested, here are some useful resources on DT:

IDEO Shopping cart example

A Virtual Crash Course in Design Thinking

Design Thinking for Libraries Toolkit

Promoting Evidence Based Library and Information Practice – OK-ACRL Conference Trip

Last week I attended the 2018 OK-ACRL Annual Conference, and the conference theme was “Elementary, My Dear Data: Evidence-based Library Practice”, seeking to promote evidence-based library and information practice (EBLIP) among OK academic librarians. I was invited to give a talk about the very first step in the EBLIP process – formulating the question or articulating the problem that needs to be answered/resolved through the application of evidence.

EBLIP advocate Denise Koufogiannakis defines EBLIP as a process that involves “methods for resolving daily problems in the profession through the integration of experience and research” and “asking questions, finding information to answer them (or conducting one’s own research) and applying that knowledge to our practice”. The evidence-based movement originates in medicine in the early 1990s. Evidence-based medicine (EBM) focuses on merging what is learned from the research literature with what is observed in daily practice, all to produce a better-informed outcome for patients. Over the years, the movement has migrated to librarianship and EBLIP.

Interestingly, librarians’ conceptualization of what constitutes “evidence” differs quite a bit from medical professionals. In EBM, evidence is strictly rooted in the published research literature, and methods such as meta-analysis or systematic reviews are conducted to appraise the evidence to determine its applicability in informing practice. Yet, my own research and reading about EBLIP has shown that librarians hold a more generous and inclusive view of what may be considered “evidence”. As indicated in the picture below, the sources of evidence in EBLIP span across a wide spectrum.

slide

Still, I would strongly encourage librarians to give priority to research evidence in their evidence-based practice. It is worth noting that currently the quality and quantity of the published literature in librarianship have not reached a level where we can conduct meta-analysis or systematic reviews to meaningfully draw on the research evidence to support decision making. Thus, librarians may consider more actively engaging in producing original research evidence and publish it – on one hand, the original research generates the evidence needed to address practical problems; on the other hand, once the original research gets published, it helps grow and enrich the existing literature, and ultimately moving the profession toward a future where the literature can become a vast and reliable source of quality research evidence.

Anyways, I’m glad that OK-ACRL chose EBLIP as their conference theme – hopefully more academic librarians, and academic librarian organizations will take notice and embrace EBLIP. Although the conference trip was short, I had a wonderful time enjoying the warmth of OK librarians, the scrumptious food and the gorgeous views on the campus of University of Central Oklahoma (UCO). I met librarians who read and appreciated my articles, alum from our SJSU iSchool, and even an MLIS student who has family living in the small CA town where I live. What a memorable trip!

Here are a few photos of the beautiful UCO library:

A few research methods resources from SAGE

SAGE is the well-known publisher about research methods. My institution, San Jose State University, subscribes to a great online resources from SAGE – SAGE Research Methods. It is a comprehensive resource containing articles, books, and video on various research methods topics. It has greatly aided the research methods instruction in our program.

SAGE Research Methods is a subscription database though. Still there are free SAGE resources online that can be of benefit to researchers and research methods instructors:

Methods Map. Aims to help researchers “Explore the research methods terrain, read definitions of key terminology, and discover content relevant to your research methods journey”.
Reading List. Provides reading lists of key research methods and statistics resources created by users.
Which Stats Test. Helps researchers choose the right statistical test to use.
Methodspace. An informative research methods blog hosted by SAGE Publishing.
SAGE LibGuides I know, how can librarians not love SAGE – they have LibGuides page! Here’s an excerpt from the page explaining its purpose “I’ve created this research guide to help you in your exploration of research methods in the social sciences. In this guide are tips to find resources on the SAGE Research Methods platform. It will also guide you in finding the best literature to enable you to choose a methodology and launch into your research project. ”

My new book – “Enhancing Library and Information Research Skills: A Guide for Academic Librarians”

My two wonderful colleagues, Kris
Brancolini and Marie Kennedy, and I wrote a book together based on our
experience with the Institute for Research
Design in Librarianship (IRDL), an IMLS-funded program that provides
professional development opportunities for academic librarians to improve their
research skills. The book is titled “Enhancing Library and Information Research
Skills: A Guide for Academic Librarians”, in which we covered the whole
spectrum of being a practitioner-researcher in the academic library setting.

It was a great experience working with
Kris and Marie on this book. They both are strong advocates for academic
librarians’ engagement in research and use of research evidence to inform
decision making. We truly hope that this book will help academic librarians
around the nation to become more aware of the value of research to academic
librarianship, develop a solid understanding of the research process, and ultimately
improve their confidence and competency in conducting and applying research in
their professional practice.

IRDL 2017 and Interview as a research method

Last week I went to the beautiful campus
of Loyola Marymount University for IRDL 2017. I’m glad that this wonderful
research methods training program received funding to continue for another
three years – more academic librarians will benefit from it and gain important
skills to become more competent and confident practitioner researchers. I had
great conversations with the participants this year. They were all working on
interesting projects – e.g. how ARL libraries design their fundraising page on
the library website, how students understand or misunderstand library jargon,
how students’ interpretation of research differs from faculty’s expectation and
observation of their research behavior, just to name a few. I can’t wait to
read more about these great projects in journal publications.

In the past week, I taught during the
day, and worked on other projects at night (yes, it’s a productive week for
me!). Particularly, I took full advantage of LMU library’s subscription to SAGE
Research Methods, and read a number of articles about the research method –
interview. There are different ways to use interview in gathering qualitative
data. Here’s a summary of them from the SAGE Encyclopedia of Qualitative
Research:

Convergent
interviewing – a technique that aims to collect, analyze, and interpret
people’s experiences, opinions, attitudes, beliefs, and knowledge that converge
around a set of interviews. It was created primarily to address issues in under-researched
areas. It permits in-depth interviewing by promoting a cyclical research
process that requires ongoing analysis as part of the overall strategy. Interviewers
engage in a constant comparative reflexive process that permits detailed rich
content and theoretical sampling as researchers seek to continuously test
emerging interpretations from early interviews in subsequent interviews.
Cognitive
interviewing – it encompasses a variety of approaches for eliciting qualitative
data on how participants interpret and respond to a wide variety of situations.
Cognitive interviewing increasingly is used in the evaluation of technology
interfaces such as websites and tools for informatics. It is used in education
to understand how students think about content and respond to test items and in
marketing to understand how to evaluate products better.
Conversational
interviewing – an approach used by research interviewers to generate verbal data
through talking about specified topics with research participants in an
informal and conversational way. Interviewers and interviewees rely on
taken-for-granted assumptions about how everyday talk occurs and how speakers
make meaning of one another’s utterances. In emphasizing features of mundane conversation,
conversational interviewers strive to facilitate a research environment in
which participants feel free to participate in extended discussions of research
topics in a less hierarchical environment than that convened in structured
interview settings.
Narrative interview
– an interview that is organized to facilitate the development of a text that can
be interpreted through narrative analysis. Narrative analysis is guided by a
theory of narrative, and these theories of narrative vary in the influence of
the reader, the text, and the intent of the author on interpretation. For this
reason, the content and structure of a narrative interview will depend both on
the theory of narrative being used in the analysis and on the research
question.
Interactive interviewing
– an interpretive practice for getting
an in-depth and intimate understanding of people’s experiences with emotionally
charged and sensitive topics such as childbirth, illness, loss, and eating
disorders. Emphasizing the communicative and joint sense-making that occurs in
interviewing, this approach involves the sharing of personal and social
experiences of both respondents and researchers, who tell (and sometimes write)
their stories in the context of a developing relationship.
The critical
incident technique (CIT) – originally, it refers to a set of procedures to
collect direct observations of human behaviors in a way that facilitates their
use in solving practical problems and developing broad psychological
principles. Over the years, it has been increasingly applied to studying psychological
states or experiences, and emphasis has shifted from direct observation by
experts to retrospective self-report in interviews.
Co-constructed
narratives – this refers to stories jointly constructed by relational partners
about epiphanies in their lives. This approach offers a way for participants to
actively construct a version of a relational event that provides insight,
understanding, and an in-depth and complex reflection on what occurred. As
such, this mode of doing research provides an alternative to traditional interviewing,
especially when the topic under consideration is emotionally charged, personal,
and sensitive.

Creative Research Methods for LIS Research II.

In library research, an important source
of data is input from the community of library users. How are users using
library collections/services/facilities? How satisfied are they with their
library experience? What are the inadequacies in meeting their needs? User
input is critical in helping library professionals understand how successful
the library is in serving the user community. Traditionally, librarians gather
user input through survey, in-depth interview and focus group. To make the
research process more engaging to both the researcher and library users, they
may consider more creative alternatives, such as the photovoice method. Through
the utilization of photographs and stories, this method allows community
members to identify and represent issues of importance to them, enabling policy
makers to develop a greater understanding of the issues, and formulate
effective and comprehensive strategies to address them in a way that is also
meaningful for the community.

Photovoice is a qualitative method often
used in community-based
participatory research (CBPR), where community members take photos related
to a particular issue and tell their stories behind the photos in a facilitated
discussion. Community members are believed to be more imaginative and observant
of community issues than the most experienced photographers and photo
journalists. In library and information science (LIS), photovoice is still
relatively new and not widely used. Recently I wrote an article to review this
method and discuss its implications for LIS research. The article is published
in Library Hi Tech, titled “Photovoice:
A Creative Method to Engage Library User Community”.

Since Photovoice is a CBPR method, it is
often applied in action-oriented projects, where a partnership among the key
stakeholders is established and all partners are equitably involved in the
process with the aim of combining knowledge and action for community
improvement . The key stakeholders include researchers, community members and
decision makers, and they participate in the photovoice process collaboratively
to explore community issues and produce an action plan to address them. However,
photovoice projects are known for being time consuming as they usually extend
over several weeks, which can be burdensome for some community participants and
lead to high attrition rates. The cost related to photography and exhibition
can be a concern too. Without community members’ sustained commitment and
adequate funding, it is unlikely for the photovoice project to succeed.

Photovoice provides an engaging way for
libraries to examine user needs, perceptions or behavior in order to provide
better services to them. Combining visuals and narratives, this method may help
capture a nuanced view of topics that are sensitive, vague or difficult to
articulate verbally, and thus may generate data unexpected when using
interviews or focus groups. The collaborative nature of the photovoice method
allows community members take on the role of co-researchers as they participate
in photovoice projects. It provides an empowering mechanism for library users
to become actively engaged in issues pertinent to the user community and
advocate for their concerns, needs and desires. With active user involvement,
library outreach efforts are more likely to be successful.

Creative Research Methods for LIS Research I.

Recently I have been reading “Creative Research Methods in the Social Sciences: A Practical Guide” by Helen Kara. It’s refreshing to read about the various approaches in arts-based research, research using technology, mixed-methods research and trans formative research. It made me think how these methods can be applied in LIS research.

One of the methods is vignettes, which I happened to co-write a paper about with two wonderful librarians, Allison Benedetti and John Jackson. Here’s an excerpt from our paper:

“Vignettes are short
stories about hypothetical characters in hypothetical circumstances, to whose
situation the interviewee is invited to respond. As a
methodological tool, vignettes can be used in focus group interviews, in-depth
interviews, or survey interviews, where the interviewee is invited to draw upon
their own experience, and provide perceptions, opinions, beliefs, attitudes,
and diagnostic predictions about how the fictional character in the vignette
will behave. When
observing or placing an individual in a particular context would not be
possible for logistical or ethical reasons, vignettes are often considered. Approximating a real-world situation, vignettes
allow for features of the context to be specified so that the interviewee can
make normative statements about a set of social circumstances rather than
provide their responses in a vacuum.

Vignettes
are often presented as a written narrative that the interviewee can read. Vignettes
must carry sufficient detail to allow the interviewee to visualize the
hypothetical circumstances as an actual situation. Particularly, the
situational elements of a vignette need to be carefully specified and the main
characters in the vignette are usually given names. Following each vignette,
the researcher may ask an open-ended question or a closed question with a set
of response categories from which the interviewee can choose. Probes are used
when necessary for the interviewee to elaborate on their responses.

In LIS research, vignettes have not been widely used. Given that vignettes
are helpful to depersonalize sensitive topics and encourage respondents to talk
more openly, they could be potentially useful for researching the attitudes and
behaviors of scholars related to publishing, copyright, open access practices,
and practically any area in which libraries are curious about the behaviors and
habits of users. Vignettes could also be used with students to teach about or
demonstrate concepts related to plagiarism and academic integrity. User
experience and web design practices have long been utilizing personas or
archetypes to evaluate designs and functionality of interfaces; there may be a
place for vignettes to augment these practices, perhaps with remote or online
usability studies.”

I would like to continue explore the use of creative methods in LIS research. Writing a blog series about this would help me document my exploration. So this post would be the first in this series. The next method I hope to explore is photovoice. A member of the 2016 IRDL cohort is considering using it. How great is that!

Setting the research agenda

A few weeks ago I gave a talk at SJSU Gateway PhD students’ virtual residency about how to set the research agenda. To prepare for that talk, I looked back in the past 11 years and thought about how I have been planning, conducting and disseminating my research since I was a doctoral student. I was able to summarize a few useful (hopefully) tips from my experience and share them with our PhD students. I’m posting them here too.

1. Set aside blocks of time designated for “research thinking”. I often do my “thinking” while I’m cleaning the house or cooking – such thinking could be about anything related to research (e.g. ideas for the next research project, how to interpret the data in a completed project and what arguments to make, etc.)

2. Reading favorite journals/blogs/Websites to keep up to date with research developments in one’s field.

3. Record/organize all research-related thoughts/ideas for later consideration – I use Evernote, but a traditional notebook will do too.

4. Join/form a journal club with colleagues/students – exchanging critical evaluation of the published literature with my peers is intellectually stimulating.

5. Serve on the editorial board or as a peer reviewer for journals/grants/conferences – this is a great opportunity to see the most recent research in one’s field.

6. When attending conferences/workshops, it is worth paying attention to work-in-progress presentations.

7. If interested in a research topic, it would be helpful to teach a graduate seminar course on that. Teaching and research are mutually beneficial.

8. Be flexible – set long term goals and short term objectives, and conduct elf-examination at regular intervals and adjust accordingly.

9. Develop a writing routine – this is hard and takes much self-discipline.

10. Quantify the publication targets – for example, every year aim for X number of journal articles, X number of conference presentations, etc.

11. Identify 3-5 journals or conferences for research dissemination – be familiar with their topical coverage, editorial style, submission guidelines and everything else that needs to be noted.

12. Collaborate – potential collaborators may be colleagues, students, researchers connected at conferences or even from other disciplines.

13. Enhance research profile via various online venues – social media (e.g. blog/tweet about your research, create a presence on ResearchGate, academia.edu or Google Scholar), institutional repository, publisher’s marketing tools

14. Be active and be part of the ongoing scholarly conversation/debate.

Formulating the research question

Last week, students in our SJSU Gateway PhD program gathered in San Jose for a one-week residency. This year we have 9 incoming students, and it’s great to chat with them about their research interests and share their passion and excitement of embarking on a new journey. I have two new students this year – one of them is interested in study the relevance of information literacy skills, and another one seeks to better understand the role of medical librarians in improving the public’s health literacy. We spent quite a bit of time talking about developing their research questions in the past week.

The formulation of research question, is probably the utmost important step in a research process. It anchors the entire research study. Yet I have seen too many studies that proceeded without a clearly defined research question. I always ask students to think about what they want to achieve with their study , what is the research problem they hope to resolve, and what their research objectives are. Answers to these questions are helpful for us to understand what exactly it is we try to study, and hence formulate a research question to properly reflect that. Once we have the research question, we can then move on to operationally define each concept and variables in the question, and then start thinking about research design and methodology.

In the many research methods textbooks I have read, research question formulation and development is a topic that’s often lacking. One book that addresses it relatively well is Babbie’s “The Practice of Social Research” (I wrote a blog post about it). And that’s one of the reasons I choose it to be the textbook for my research methods class. The new semester is starting in just a couple of weeks, and in the fall, I will be teaching a new research methods course that focuses on survey research. One of the course assignments is to complete a proposal for a research study that’s appropriate for survey research. Thus, students will need to come up with a research question that has to be answered by a survey study. I wonder how that will impact the question development process. Will make it harder or easier? I think it’s the latter. We’ll see.

Books about Research Methods (5)

I have been thinking about developing a research methods course focusing on the survey method. Survey is a very old research technique. According to Earl Babbie’s “The Practice of Social Research”, the use of survey could be traced to the Old Testament – “After the plague the Lord said to Moses and to Eleazar the son of Aaron, the priest, ‘Take a census of all the congregation of the people of Israel, from twenty old and upward’” (Numbers 26: 1-2). Surveys are mainly used in studies that have individual people as the units of analysis, and to collect original data for describing a population too large to observe directly. In LIS research, survey is one of the most frequently used research methods. For example, survey research can be conducted to study user satisfaction with reference service, to assess student learning in information literacy instruction, to examine user awareness of mobile services provided by the library, etc.

To develop a course on survey research, the first thing is to find a textbook. I looked around and decided on “Internet, mail, and mixed-method surveys – the tailored design method”, by Don Dillman, Jolene Smyth and Leah Christian. It’s a comprehensive guide to designing and administering surveys. I particularly like how they talk about constructing survey questions – this is the most critical element in questionnaire design. I like their writing style too – straightforward and easy to understand. The only regret is that they did not include much discussion of analyzing survey responses. It’s understandable though – data analysis in survey research is a huge topic that warrants a book of its own. But I guess this means I will have to find other readings for this topic then. Time to dive back into the literature!