Using ChatGPT in Research Design

In the past two weeks, we completed the research methods training portion of 2023 IRDL Online. IRDL is a continuing education program for academic and research librarians designed to create a growing community of confident librarian-researchers. The research methods training portion was provided in the form of an online summer research workshop, where I was an instructor. It’s incredibly energizing and exciting to talk to a group of research-minded librarians about conducting research to improve professional practice. One theme that arose from our discussions was the use of AI in research design. One of the librarian scholars gave it a try and asked ChatGPT to write survey questions for a research scenario.

Here’s her prompt: I need to write a survey to find out how academic librarians engage in professional development. ACRL will use the survey findings to develop the Guidelines for Professional Development for Academic Librarians, which they believe will provide guidance for academic librarians as they explore professional development opportunities and make professional development plans. Can you write 4 questions for this survey?

ChatGPT’s response:
Certainly! Here are four sample questions for your survey on how academic librarians engage in professional development:

1. How frequently do you engage in professional development activities related to your role as an academic librarian?
• Rarely or never
• Occasionally (1-2 times per year)
• Regularly (3-5 times per year)
• Frequently (more than 5 times per year)

2. What types of professional development activities do you find most beneficial in enhancing your skills and knowledge as an academic librarian? (Select all that apply)
• Workshops and conferences
• Webinars and online courses
• Professional association memberships and events
• Formal education (e.g., degree programs)
• Informal learning (e.g., self-study, reading journals)
• Mentoring and coaching programs
• Other (please specify)

3. How do you typically seek out information about professional development opportunities? (Select all that apply)
• Internal communication channels (e.g., email, intranet)
• External websites and online resources
• Professional networks and colleagues
• Professional association publications and newsletters
• Supervisors and managers
• Other (please specify)

4. What factors influence your decision to participate in a professional development activity? (Select all that apply)
• Relevance to current job responsibilities and career goals
• Availability of funding or financial support
• Recommendations from colleagues or supervisors
• Reputation and credibility of the provider or organization
• Flexibility in terms of time and scheduling
• Recognition or credentialing opportunities
• Other (please specify)

Looking at these survey questions, I think ChatGPT did a nice job in operationalizing the key concepts in the research question based on the research goal. There’s room for improvement in wording survey questions that uphold the basic rules for survey design such as being concise and unambiguous, avoiding leading questions or double barreled questions, and making sure response categories are exhaustive and mutually exclusive. Still, this can be a good start if we are get stuck in our instrument design and don’t know where to begin.

There’s been discussion about using AI in manuscript writing such as this Science article “As scientists explore AI-written text, journals hammer out policies“. Looks like we will need more discussion about using AI in assisting the research process such as designing survey questionnaire, interview guide or a content analysis codebook.

A Great Online Student Conference

I’m the chair of the RSCA Advisory Committee at the College of Professional and Global Education (CPGE) at SJSU. RSCA stands for Research, Scholarship, and Creative Activity. The college held its first Online Student Conference in spring 2022, and our committee served as faculty advisors for this conference. The conference seeks to connect students across the college and promote student work. It aims to provide an opportunity for CPGE students to share their school or professional work, help students communicate and connect with each other, and ultimately foster a stronger sense of community among students.

A total of 16 students submitted recorded presentations to be featured on the conference website. Their presentations covered a wide range of topics ranging from diversity/inclusion to sustainability. Five students did such an excellent job in their presentations that they won the conference’s Outstanding Presentation Award. Let’s take a look at the five award-winning student presentations.

Ashley Minnich’s presentation, ”Mental, Physical, and Social Benefits of Dance: A Client Project“, described a client project where the information request was for scientific and peer-reviewed scholarly literature published within the last 20 years about the mental, physical, and social benefits of dance for the organization to use in future marketing materials. The project was executed by conducting a reference interview with the client, evaluating potential databases for topic relevancy, developing a search strategy and evaluating results, presenting results to the client concisely for their evaluation, and mutually concluding that the information need was met.

Beatriz Martinez delivered a presentation titled “Garden Focus Library Programming: A Green Team Project“. This project emerged from SJSU iSchool’s INFO 282 Libraries Take the Lead on Sustainability. Beatriz worked on a library programming group project. The group developed a garden focused library program that comprises many different forms of activities and events aiming to promote sustainable gardening to all patron demographics.

Rosario Mireles’s presentation, “Generations of Regeneration“, was about a project also resulting from INFO 282 Libraries Take the Lead on Sustainability. This project proposed an action plan targeting community members of all ages (children to grandparents) and addressing the Three Pillars of Sustainability (Environment, Equity, Economic) through regular presentations and interactive workshops at a hypothetical library’s teaching garden.

Katherine Hellman’s presentation, “Weeding is Fundamental! What Comes After Weeding?“, was once again an assignment from INFO 282 Libraries Take the Lead on Sustainability. This project assessed the environmental impact of weeding materials from library collections and offered three sustainable alternatives to sending books and periodicals to landfills.

Carlos Quevedo delivered a presentation titled “Emerging Needs in the Field of Video Game Preservation“. In this presentation, Carlos shared his experience doing research on Digital Media Preservation as it pertains to Video Games through his internship at AFI as well as his role as a Research Assistant to iSchool Professor Darra Hofman. His work enabled him to foresee the future work that needs to be done in order to make the preservation of Video Games a more fruitful endeavor in the future.

In addition to these award winning presentations, other student presentations at the CPGE Online Student Conference also represented the wonderful work our students have done. Congratulations to all the students who presented at the conference and shared their amazing work with the college community and beyond.

Impact of the COVID-19 Pandemic on Chinese College Students’ Information Behavior and Library Needs

Last year, my research collaborators and I conducted a study to examine how the pandemic had impacted Chinese college students’ use of the library. The study took place in April and May 2020. At that time, Chinese college campus had been shut off and all the teaching and learning had been moved online for two months. We collected data using a combination of journaling and in-depth interviews, focusing on the library use and library needs of Chinese college students during the COVID-19 pandemic.

Our findings indicate that students generally lack awareness and understanding of the library’s online resources and would like the library to provide support in ways that could help them enhance productivity and lessen anxiety in the new reality of learning. Libraries should consider making more outreach efforts and offering programs and events to ameliorate isolation and improve students’ sense of community. We hope that our study can provide academic libraries with a nuanced view of user needs and thus help them make informed decisions to serve their campus communities during the unprecedented health crisis. Through the exploration and documentation of college students’ library use and needs, we also hope to document this critical historical event for the library community.

Now our study has been published in the Library Quarterly. Here’s the citation:

Shi, Y., Li, C. & Luo, L. (2021). Impact of the covid-19 pandemic on Chinese college students’ information behavior and library needs: A qualitative study. Library Quarterly, 91(2), 150-171.

I’m really glad that we were able to contribute to the library and information science (LIS) literature with regards to the pandemic. When years later, the pandemic is finally behind us, I think it will be quite interesting to conduct a content analysis of all the pandemic-related studies/articles in peer reviewed LIS journals to capture and document LIS researchers’ collection contributions.

Recent trip to ACEID 2018 Conference

I attended the 2018 Asian Conference on Education and International Development (ACEID) last week, and it was my first time attending a conference outside of LIS. The majority of the conference participants (from over 40 countries) were from education, and their presentations covered a wide range of education related topics. My presentation was about how librarians and faculty collaborate in higher education, and it was scheduled in a session along with two other presentations focusing on higher ed. Unfortunately, not many people showed up in the audience – the concurrent sessions on K-12 issues outcompeted us. Still, I had a nice discussion with the small audience – before I presented my study, I showed them slides of librarian stereotypes and their true images (e.g. portraits of the book “This is What a Librarian Looks Like”, the New Zealand librarians cosplaying the Kardasians, and examples of the cool things that librarians made happen in the library), and the audience was quite impressed. They agreed that my presentation gave them good ideas of how to reach out to their librarians for help now that they know libraries are such an important component of the learning ecology in higher ed.

I had much fun interacting with education researchers at the conference and learned a lot myself. There’s a even research about how PMS (pre-menstrual syndrome) affects middle school girls’ academic performance in school. How interesting.

The conference took place in Kobe Japan during the beautiful cherry blossoms, and there were lines of cherry trees right outside of the conference venue, so I got to witness the fleeting and delicate beauty that has such significant cultural symbolism in Japan. It was indeed quite a sight!

IRDL 2017 and Interview as a research method

Last week I went to the beautiful campus
of Loyola Marymount University for IRDL 2017. I’m glad that this wonderful
research methods training program received funding to continue for another
three years – more academic librarians will benefit from it and gain important
skills to become more competent and confident practitioner researchers. I had
great conversations with the participants this year. They were all working on
interesting projects – e.g. how ARL libraries design their fundraising page on
the library website, how students understand or misunderstand library jargon,
how students’ interpretation of research differs from faculty’s expectation and
observation of their research behavior, just to name a few. I can’t wait to
read more about these great projects in journal publications.

In the past week, I taught during the
day, and worked on other projects at night (yes, it’s a productive week for
me!). Particularly, I took full advantage of LMU library’s subscription to SAGE
Research Methods, and read a number of articles about the research method –
interview. There are different ways to use interview in gathering qualitative
data. Here’s a summary of them from the SAGE Encyclopedia of Qualitative
Research:

Convergent
interviewing – a technique that aims to collect, analyze, and interpret
people’s experiences, opinions, attitudes, beliefs, and knowledge that converge
around a set of interviews. It was created primarily to address issues in under-researched
areas. It permits in-depth interviewing by promoting a cyclical research
process that requires ongoing analysis as part of the overall strategy. Interviewers
engage in a constant comparative reflexive process that permits detailed rich
content and theoretical sampling as researchers seek to continuously test
emerging interpretations from early interviews in subsequent interviews.
Cognitive
interviewing – it encompasses a variety of approaches for eliciting qualitative
data on how participants interpret and respond to a wide variety of situations.
Cognitive interviewing increasingly is used in the evaluation of technology
interfaces such as websites and tools for informatics. It is used in education
to understand how students think about content and respond to test items and in
marketing to understand how to evaluate products better.
Conversational
interviewing – an approach used by research interviewers to generate verbal data
through talking about specified topics with research participants in an
informal and conversational way. Interviewers and interviewees rely on
taken-for-granted assumptions about how everyday talk occurs and how speakers
make meaning of one another’s utterances. In emphasizing features of mundane conversation,
conversational interviewers strive to facilitate a research environment in
which participants feel free to participate in extended discussions of research
topics in a less hierarchical environment than that convened in structured
interview settings.
Narrative interview
– an interview that is organized to facilitate the development of a text that can
be interpreted through narrative analysis. Narrative analysis is guided by a
theory of narrative, and these theories of narrative vary in the influence of
the reader, the text, and the intent of the author on interpretation. For this
reason, the content and structure of a narrative interview will depend both on
the theory of narrative being used in the analysis and on the research
question.
Interactive interviewing
– an interpretive practice for getting
an in-depth and intimate understanding of people’s experiences with emotionally
charged and sensitive topics such as childbirth, illness, loss, and eating
disorders. Emphasizing the communicative and joint sense-making that occurs in
interviewing, this approach involves the sharing of personal and social
experiences of both respondents and researchers, who tell (and sometimes write)
their stories in the context of a developing relationship.
The critical
incident technique (CIT) – originally, it refers to a set of procedures to
collect direct observations of human behaviors in a way that facilitates their
use in solving practical problems and developing broad psychological
principles. Over the years, it has been increasingly applied to studying psychological
states or experiences, and emphasis has shifted from direct observation by
experts to retrospective self-report in interviews.
Co-constructed
narratives – this refers to stories jointly constructed by relational partners
about epiphanies in their lives. This approach offers a way for participants to
actively construct a version of a relational event that provides insight,
understanding, and an in-depth and complex reflection on what occurred. As
such, this mode of doing research provides an alternative to traditional interviewing,
especially when the topic under consideration is emotionally charged, personal,
and sensitive.

Creative Research Methods for LIS Research II.

In library research, an important source
of data is input from the community of library users. How are users using
library collections/services/facilities? How satisfied are they with their
library experience? What are the inadequacies in meeting their needs? User
input is critical in helping library professionals understand how successful
the library is in serving the user community. Traditionally, librarians gather
user input through survey, in-depth interview and focus group. To make the
research process more engaging to both the researcher and library users, they
may consider more creative alternatives, such as the photovoice method. Through
the utilization of photographs and stories, this method allows community
members to identify and represent issues of importance to them, enabling policy
makers to develop a greater understanding of the issues, and formulate
effective and comprehensive strategies to address them in a way that is also
meaningful for the community.

Photovoice is a qualitative method often
used in community-based
participatory research (CBPR), where community members take photos related
to a particular issue and tell their stories behind the photos in a facilitated
discussion. Community members are believed to be more imaginative and observant
of community issues than the most experienced photographers and photo
journalists. In library and information science (LIS), photovoice is still
relatively new and not widely used. Recently I wrote an article to review this
method and discuss its implications for LIS research. The article is published
in Library Hi Tech, titled “Photovoice:
A Creative Method to Engage Library User Community”.

Since Photovoice is a CBPR method, it is
often applied in action-oriented projects, where a partnership among the key
stakeholders is established and all partners are equitably involved in the
process with the aim of combining knowledge and action for community
improvement . The key stakeholders include researchers, community members and
decision makers, and they participate in the photovoice process collaboratively
to explore community issues and produce an action plan to address them. However,
photovoice projects are known for being time consuming as they usually extend
over several weeks, which can be burdensome for some community participants and
lead to high attrition rates. The cost related to photography and exhibition
can be a concern too. Without community members’ sustained commitment and
adequate funding, it is unlikely for the photovoice project to succeed.

Photovoice provides an engaging way for
libraries to examine user needs, perceptions or behavior in order to provide
better services to them. Combining visuals and narratives, this method may help
capture a nuanced view of topics that are sensitive, vague or difficult to
articulate verbally, and thus may generate data unexpected when using
interviews or focus groups. The collaborative nature of the photovoice method
allows community members take on the role of co-researchers as they participate
in photovoice projects. It provides an empowering mechanism for library users
to become actively engaged in issues pertinent to the user community and
advocate for their concerns, needs and desires. With active user involvement,
library outreach efforts are more likely to be successful.

Creative Research Methods for LIS Research I.

Recently I have been reading “Creative Research Methods in the Social Sciences: A Practical Guide” by Helen Kara. It’s refreshing to read about the various approaches in arts-based research, research using technology, mixed-methods research and trans formative research. It made me think how these methods can be applied in LIS research.

One of the methods is vignettes, which I happened to co-write a paper about with two wonderful librarians, Allison Benedetti and John Jackson. Here’s an excerpt from our paper:

“Vignettes are short
stories about hypothetical characters in hypothetical circumstances, to whose
situation the interviewee is invited to respond. As a
methodological tool, vignettes can be used in focus group interviews, in-depth
interviews, or survey interviews, where the interviewee is invited to draw upon
their own experience, and provide perceptions, opinions, beliefs, attitudes,
and diagnostic predictions about how the fictional character in the vignette
will behave. When
observing or placing an individual in a particular context would not be
possible for logistical or ethical reasons, vignettes are often considered. Approximating a real-world situation, vignettes
allow for features of the context to be specified so that the interviewee can
make normative statements about a set of social circumstances rather than
provide their responses in a vacuum.

Vignettes
are often presented as a written narrative that the interviewee can read. Vignettes
must carry sufficient detail to allow the interviewee to visualize the
hypothetical circumstances as an actual situation. Particularly, the
situational elements of a vignette need to be carefully specified and the main
characters in the vignette are usually given names. Following each vignette,
the researcher may ask an open-ended question or a closed question with a set
of response categories from which the interviewee can choose. Probes are used
when necessary for the interviewee to elaborate on their responses.

In LIS research, vignettes have not been widely used. Given that vignettes
are helpful to depersonalize sensitive topics and encourage respondents to talk
more openly, they could be potentially useful for researching the attitudes and
behaviors of scholars related to publishing, copyright, open access practices,
and practically any area in which libraries are curious about the behaviors and
habits of users. Vignettes could also be used with students to teach about or
demonstrate concepts related to plagiarism and academic integrity. User
experience and web design practices have long been utilizing personas or
archetypes to evaluate designs and functionality of interfaces; there may be a
place for vignettes to augment these practices, perhaps with remote or online
usability studies.”

I would like to continue explore the use of creative methods in LIS research. Writing a blog series about this would help me document my exploration. So this post would be the first in this series. The next method I hope to explore is photovoice. A member of the 2016 IRDL cohort is considering using it. How great is that!

IRDL 2016

The third and final Institute for Research Design in Librarianship
(IRDL) took place in June. IRDL is an IMLS-funded
project that seeks to create a learning opportunity and a support system for
academic librarians who want to improve their research skills and increase
their research output. Twenty-one librarian scholars gathered at the beautiful
campus of Loyola Marymount University and went through a 9-day intensive
research methods training program. This year, quite a number of librarians were
working on topics involving populations that are understudied in library
research. Here are a few examples:

1.
How international students use the library and
how does the library contribute to their success and growth in life

2.
How Chinese international students interpret
“Authority is constructed and contextual”

3.
How scholars who study materials in non-English
languages but publish in English use citation management software

4.
What is the experience of librarians of color in
residency programs

5.
What are the barriers for women to seek
leadership positions in libraries

During the Institute, there were
two hours scheduled each day for the participating librarians to consult the
instructors and IRDL directors about their research projects. I enjoyed these
consultations immensely. It is always pleasant to chat with people who are
enthusiastic about their research. The projects about understudied populations
reinforced my idea of offering a research methods course that focuses on
researching special populations. Every semester there are always students in
our MLIS degree program proposing to conduct research about the library and
information needs/behavior of a population that is traditionally underserved or
understudied in libraries. For instance, in the past semester, a student
proposed a survey study to examine the library needs of people who are
quadriplegic. As our libraries strive to meet the needs of diverse user populations,
it would be helpful if our librarians know how to properly design studies to
generate practice-informing evidence about different populations, especially
the underserved or less privileged ones.

See, IRDL always gives me new
ideas and refreshes my perspectives as a researcher/educator. Two weeks flew by
very quickly. I appreciate the opportunity to be part of this great effort that
seeks to enhance the quality of scholarship among librarians and foster a
culture of research-mindedness. Marie and Kris, the two wonderful IRDL project
directors, are truly insightful in making IRDL happen, and I’m sure that the
IRDL participants will make them proud by raising the bar of library research.

[The photo was actually taken during the first year of IRDL in 2014 – Greg, Michael and I were the instructors. It was so nice to see Greg and Michael every year at IRDL (and Emily this year). We make a great team 🙂 ]

Research-based Library Practices Summit in Doha, Qatar

Last week I had the pleasure of
going to Doha, Qatar to give a workshop at the Research-based Library Practices
Summit. The Summit was organized by librarians from University College London
and Carnegie Mellon University in Qatar and attended by librarians from
multiple countries in the Gulf region. My workshop focused on providing an
overview of the frequently used quantitative and qualitative research methods
in library and information science such as survey, focus group interview,
in-depth interview and content analysis, and explaining the process of making a
methodological decision based on the research question. The objective of the
workshop is for participants to gain knowledge and skills to determine which
qualitative and quantitative methods are most appropriate to address their
research questions. I used the following questions to help them analyze their research
question and make the methodological decision:

·
Are qualitative research methods appropriate for
your research question?

a.
How much previous research exists on your topic?

b.
Do you intend to capture a nuanced, in-depth
view of your topic?

c.
Do you intend to describe your observations via
the identification of themes/patterns?

d.
Is there existing qualitative content you can
analyze, or do you have to collect original qualitative data?

e.
Which qualitative data collection method do you
plan to use – focus group interview, in-depth interview or field observations?

·
Are quantitative research methods appropriate
for your research question?

a.
Is your research question a relationship
question or a causality question?

b.
Has there been extensive research conducted on
your topic that the range of Reponses or variations of behavior are already
known?

c.
Do you intend to quantify your observations and produce
generalizable conclusions?

d.
Is there existing statistics you can analyze, or
do you have to collect original quantitative data?

e.
Which quantitative data collection method do you
plan to use – quantitative content analysis, quantitative observations, or
survey?

Overall the workshop went pretty
well. We had a lot of good discussions. I was impressed with the participants’
eagerness to conduct research to improve their practice. This kind of “research-mindedness”
is exactly what we need in the profession.

During my stay in Doha, I had the
opportunity to visit the state-of-the-art Qatar National Library. It’s still
under construction, so we had to wear safety hat and vest on the tour. It’s an
amazing building. I loved the open design and the high level of automation.
There is even a restaurant in the library – a real one with a kitchen, not just
a café where you can only get sandwiches. Unfortunately we weren’t allowed to
take pictures inside. But here’s a picture of us in front of the building.

Here are a couple of pictures I
took of a gorgeous corridor in the Education City, where the Summit was taking
place. Gorgeous view, isn’t it?

Quantitative research workshop

Quantitative research has always
been a challenging topic in my teaching of research methods. Quite a number of
my students are innately disinclined to deal with numbers and find statistical
analysis uninteresting and even unnerving. So every time I cover quantitative research,
I always start with this quote from Earl Babbie’s The Practice of Social
Research:

““Empirical research is first and
foremost a logical rather than a mathematical operation. Mathematics is merely
a convenient and efficient language for accomplishing the logical operations inherent
in quantitative data analysis. This textbook is not intended to teach you
statistics or torture you with them. Rather, I want to sketch out a logical
context within which you might learn and understand statistics.”

It is important that students understand
statistics are just a means to an end – a tool that we can use to help us accomplish
our research objective by making sense of quantitative data. In my teaching, I
focus primarily on the conceptual understanding of statistics. Students are
expected to master what the frequently used statistical measures are, when to
use them, for what types of variables and what kinds of analytic objectives, and
don’t have to worry about the computational process. They can explore that on
their own – there are many tutorials on YouTube.

At this year’s SCELC Research Day, I gave a
workshop on quantitative data analysis, and this “conceptual understanding”
strategy seemed to have worked well. We had interesting discussions around the
tables and it’s great to see how the librarians planned to use the statistics
covered in the workshop in the analysis of their existing data sets. I just
wish there were more time for us to do some hands on exercises. Nonetheless, it
was a great experience talking about research methods with librarians, as always.