Using ChatGPT in Research Design

In the past two weeks, we completed the research methods training portion of 2023 IRDL Online. IRDL is a continuing education program for academic and research librarians designed to create a growing community of confident librarian-researchers. The research methods training portion was provided in the form of an online summer research workshop, where I was an instructor. It’s incredibly energizing and exciting to talk to a group of research-minded librarians about conducting research to improve professional practice. One theme that arose from our discussions was the use of AI in research design. One of the librarian scholars gave it a try and asked ChatGPT to write survey questions for a research scenario.

Here’s her prompt: I need to write a survey to find out how academic librarians engage in professional development. ACRL will use the survey findings to develop the Guidelines for Professional Development for Academic Librarians, which they believe will provide guidance for academic librarians as they explore professional development opportunities and make professional development plans. Can you write 4 questions for this survey?

ChatGPT’s response:
Certainly! Here are four sample questions for your survey on how academic librarians engage in professional development:

1. How frequently do you engage in professional development activities related to your role as an academic librarian?
• Rarely or never
• Occasionally (1-2 times per year)
• Regularly (3-5 times per year)
• Frequently (more than 5 times per year)

2. What types of professional development activities do you find most beneficial in enhancing your skills and knowledge as an academic librarian? (Select all that apply)
• Workshops and conferences
• Webinars and online courses
• Professional association memberships and events
• Formal education (e.g., degree programs)
• Informal learning (e.g., self-study, reading journals)
• Mentoring and coaching programs
• Other (please specify)

3. How do you typically seek out information about professional development opportunities? (Select all that apply)
• Internal communication channels (e.g., email, intranet)
• External websites and online resources
• Professional networks and colleagues
• Professional association publications and newsletters
• Supervisors and managers
• Other (please specify)

4. What factors influence your decision to participate in a professional development activity? (Select all that apply)
• Relevance to current job responsibilities and career goals
• Availability of funding or financial support
• Recommendations from colleagues or supervisors
• Reputation and credibility of the provider or organization
• Flexibility in terms of time and scheduling
• Recognition or credentialing opportunities
• Other (please specify)

Looking at these survey questions, I think ChatGPT did a nice job in operationalizing the key concepts in the research question based on the research goal. There’s room for improvement in wording survey questions that uphold the basic rules for survey design such as being concise and unambiguous, avoiding leading questions or double barreled questions, and making sure response categories are exhaustive and mutually exclusive. Still, this can be a good start if we are get stuck in our instrument design and don’t know where to begin.

There’s been discussion about using AI in manuscript writing such as this Science article “As scientists explore AI-written text, journals hammer out policies“. Looks like we will need more discussion about using AI in assisting the research process such as designing survey questionnaire, interview guide or a content analysis codebook.

My new book – “Enhancing Library and Information Research Skills: A Guide for Academic Librarians”

My two wonderful colleagues, Kris
Brancolini and Marie Kennedy, and I wrote a book together based on our
experience with the Institute for Research
Design in Librarianship (IRDL), an IMLS-funded program that provides
professional development opportunities for academic librarians to improve their
research skills. The book is titled “Enhancing Library and Information Research
Skills: A Guide for Academic Librarians”, in which we covered the whole
spectrum of being a practitioner-researcher in the academic library setting.

It was a great experience working with
Kris and Marie on this book. They both are strong advocates for academic
librarians’ engagement in research and use of research evidence to inform
decision making. We truly hope that this book will help academic librarians
around the nation to become more aware of the value of research to academic
librarianship, develop a solid understanding of the research process, and ultimately
improve their confidence and competency in conducting and applying research in
their professional practice.

IRDL 2017 and Interview as a research method

Last week I went to the beautiful campus
of Loyola Marymount University for IRDL 2017. I’m glad that this wonderful
research methods training program received funding to continue for another
three years – more academic librarians will benefit from it and gain important
skills to become more competent and confident practitioner researchers. I had
great conversations with the participants this year. They were all working on
interesting projects – e.g. how ARL libraries design their fundraising page on
the library website, how students understand or misunderstand library jargon,
how students’ interpretation of research differs from faculty’s expectation and
observation of their research behavior, just to name a few. I can’t wait to
read more about these great projects in journal publications.

In the past week, I taught during the
day, and worked on other projects at night (yes, it’s a productive week for
me!). Particularly, I took full advantage of LMU library’s subscription to SAGE
Research Methods, and read a number of articles about the research method –
interview. There are different ways to use interview in gathering qualitative
data. Here’s a summary of them from the SAGE Encyclopedia of Qualitative
Research:

Convergent
interviewing – a technique that aims to collect, analyze, and interpret
people’s experiences, opinions, attitudes, beliefs, and knowledge that converge
around a set of interviews. It was created primarily to address issues in under-researched
areas. It permits in-depth interviewing by promoting a cyclical research
process that requires ongoing analysis as part of the overall strategy. Interviewers
engage in a constant comparative reflexive process that permits detailed rich
content and theoretical sampling as researchers seek to continuously test
emerging interpretations from early interviews in subsequent interviews.
Cognitive
interviewing – it encompasses a variety of approaches for eliciting qualitative
data on how participants interpret and respond to a wide variety of situations.
Cognitive interviewing increasingly is used in the evaluation of technology
interfaces such as websites and tools for informatics. It is used in education
to understand how students think about content and respond to test items and in
marketing to understand how to evaluate products better.
Conversational
interviewing – an approach used by research interviewers to generate verbal data
through talking about specified topics with research participants in an
informal and conversational way. Interviewers and interviewees rely on
taken-for-granted assumptions about how everyday talk occurs and how speakers
make meaning of one another’s utterances. In emphasizing features of mundane conversation,
conversational interviewers strive to facilitate a research environment in
which participants feel free to participate in extended discussions of research
topics in a less hierarchical environment than that convened in structured
interview settings.
Narrative interview
– an interview that is organized to facilitate the development of a text that can
be interpreted through narrative analysis. Narrative analysis is guided by a
theory of narrative, and these theories of narrative vary in the influence of
the reader, the text, and the intent of the author on interpretation. For this
reason, the content and structure of a narrative interview will depend both on
the theory of narrative being used in the analysis and on the research
question.
Interactive interviewing
– an interpretive practice for getting
an in-depth and intimate understanding of people’s experiences with emotionally
charged and sensitive topics such as childbirth, illness, loss, and eating
disorders. Emphasizing the communicative and joint sense-making that occurs in
interviewing, this approach involves the sharing of personal and social
experiences of both respondents and researchers, who tell (and sometimes write)
their stories in the context of a developing relationship.
The critical
incident technique (CIT) – originally, it refers to a set of procedures to
collect direct observations of human behaviors in a way that facilitates their
use in solving practical problems and developing broad psychological
principles. Over the years, it has been increasingly applied to studying psychological
states or experiences, and emphasis has shifted from direct observation by
experts to retrospective self-report in interviews.
Co-constructed
narratives – this refers to stories jointly constructed by relational partners
about epiphanies in their lives. This approach offers a way for participants to
actively construct a version of a relational event that provides insight,
understanding, and an in-depth and complex reflection on what occurred. As
such, this mode of doing research provides an alternative to traditional interviewing,
especially when the topic under consideration is emotionally charged, personal,
and sensitive.

IRDL 2016

The third and final Institute for Research Design in Librarianship
(IRDL) took place in June. IRDL is an IMLS-funded
project that seeks to create a learning opportunity and a support system for
academic librarians who want to improve their research skills and increase
their research output. Twenty-one librarian scholars gathered at the beautiful
campus of Loyola Marymount University and went through a 9-day intensive
research methods training program. This year, quite a number of librarians were
working on topics involving populations that are understudied in library
research. Here are a few examples:

1.
How international students use the library and
how does the library contribute to their success and growth in life

2.
How Chinese international students interpret
“Authority is constructed and contextual”

3.
How scholars who study materials in non-English
languages but publish in English use citation management software

4.
What is the experience of librarians of color in
residency programs

5.
What are the barriers for women to seek
leadership positions in libraries

During the Institute, there were
two hours scheduled each day for the participating librarians to consult the
instructors and IRDL directors about their research projects. I enjoyed these
consultations immensely. It is always pleasant to chat with people who are
enthusiastic about their research. The projects about understudied populations
reinforced my idea of offering a research methods course that focuses on
researching special populations. Every semester there are always students in
our MLIS degree program proposing to conduct research about the library and
information needs/behavior of a population that is traditionally underserved or
understudied in libraries. For instance, in the past semester, a student
proposed a survey study to examine the library needs of people who are
quadriplegic. As our libraries strive to meet the needs of diverse user populations,
it would be helpful if our librarians know how to properly design studies to
generate practice-informing evidence about different populations, especially
the underserved or less privileged ones.

See, IRDL always gives me new
ideas and refreshes my perspectives as a researcher/educator. Two weeks flew by
very quickly. I appreciate the opportunity to be part of this great effort that
seeks to enhance the quality of scholarship among librarians and foster a
culture of research-mindedness. Marie and Kris, the two wonderful IRDL project
directors, are truly insightful in making IRDL happen, and I’m sure that the
IRDL participants will make them proud by raising the bar of library research.

[The photo was actually taken during the first year of IRDL in 2014 – Greg, Michael and I were the instructors. It was so nice to see Greg and Michael every year at IRDL (and Emily this year). We make a great team 🙂 ]

Improving Librarians’ Research Confidence

Last week I attended the 2016 conference of Qualitative and Quantitative
Methods in Libraries, and presented a paper I co-authored with two wonderful
colleagues from Loyola Marymount University (LMU), Marie and Kris. In this presentation, we talked about the
mapping between librarians’ research confidence and the curriculum of the Institute of Research Design for Librarianship (IRDL), a
federally funded program that provides research methods training for
librarians. For each topic covered in the training program (e.g. research
question development, research design, data collection, data analysis, research
dissemination), librarians’ confidence was measured before and after the
program, and increase was detected across the board. This was consistent with the
preliminary findings of another study we are conducting. In that study, we seek
to understand the long term impact of IRDL by asking the participants to
describe three incidents where they strongly felt the benefits of IRDL in their
work or research. So far we have seen quite a number of narratives about how
they felt more confident when working with faculty, talking to people at
conferences and interpreting the published literature.

According to Bandura, mastery experiences build confidence through
success and provide an individual with the ability to persevere in the face of
obstacles, which is especially important in performing difficult tasks, such as
conducting research in the traditionally practical library profession (lack of
confidence is often cited as a barrier to librarians’ research engagement). I’m
glad to see that IRDL has been able to improve librarian’s research confidence
and help them become more confident and competent practitioner researchers. The
third cohort of IRDL will gather in LMU for their training in less than a week,
and I look forward to meeting them!

2015 Institute of Research Design for Librarianship

Last week we concluded the second
year of the Institute of Research Design for
Librarianship (IRDL), an IMLS-funded project that seeks to create a
learning opportunity and a support system for academic librarians who want to
improve their research skills and increase their research output. It’s another
great 9-day institute with librarian scholars from around the country. This
year, the project directors made some changes based on the feedback from last
year’s cohort so that the scholars could have more time to write and consult
with us instructors. I truly enjoyed talking to each scholar about their
research projects. I was impressed by the innovation and initiative
demonstrated in their projects – for example, one scholar was interested in
implementing a texting-based outreach and reference service and see if that
impacts students’ library anxiety, and another scholar was thinking about
creating library tutorials using Vine and see if that would be an effective way
to impart information literacy skills. I can’t wait for them to complete their
study and get published.

Among this year’s cohort, three
were graduates from my home institution, SJSU School of Information, and two of
them took classes with me – one was in my Research Methods class, and another
was in my Reference and Information Services class. It’s great to see them
become active researchers in this profession and engage in evidence based
practices to improve their work. I’m very proud of them. This is exactly what I’m
hoping to see through research methods education both in LIS degree programs
and beyond (like IRDL).

The campus of Loyola Marymount
University was as beautiful as ever. I feel lucky that IRDL is held at LMU library, in a spacious room with a
gorgeous view. Many thanks to Marie and Kris, the two wonderful IRDL project
directors, who took good care of us while we were there. Looking forward to
next year already!

2014 Institute for Research Design in Librarianship

In the last two weeks of June, the first IRDL cohort gathered in the beautiful library of the beautiful Loyola Marymount University and went through an intensive professional development program focusing on research design and methodology (see my previous post about this program). As a research methods geek, I was thrilled and honored to be one of the instructors there. I was impressed with the IRDL scholars’ motivation, interest and enthusiasm in conducting quality research and enhancing the rigor of LIS research and publications. Their research topics covered a variety of LIS practices, including the quickly emerging area of data librarianship. During my first two days there, I got to talk with all four members of the cohort whose topics were related to data management, data literacy and data service. Our conversations were illuminating and gave me a lot to think about in terms of preparing out students for this area. Among the twenty-five scholars, four were SJSU SLIS alum – one of them took the Reference and Information Services course with me, and another one was my advisee. It’s great to see them at IRDL and chat with them about their experience at SLIS. I’m so proud of what they have accomplished and what they will be accomplishing after IRDL.

There so many memorable moments throughout the program. The highlight was the scholars’ presentations on the last day, where they talked about their research idea and how it had taken shape during the two weeks. It’s very rewarding to see that they are leaving IRDL with a fairly concrete research plan and ready to dive in and churn out peer-reviewed articles one after another. Hopefully by the end of the three-year IRDL grant, we will be able to compile a list of publications resulting from the scholars’ research projects at IRDL.

In addition to the great discussions and interactions with the IRDL scholars and my co-instructors Greg and Michael, I really appreciated how well LMU library had taken care of us – the classroom was spacious with a great view, the catering service was wonderful, and our host Kris, Marie and Carol were so thoughtful and warm. I can’t thank them enough for making this experience a great memory of summer 2014. I’m already looking forward to 2015!

Oh, before I forget, I tweeted more in the past two weeks than I did in the previous five years, and I think I will keep tweeting at that frequency. Social media, I’m serious about you now.