Using ChatGPT in Research Design

In the past two weeks, we completed the research methods training portion of 2023 IRDL Online. IRDL is a continuing education program for academic and research librarians designed to create a growing community of confident librarian-researchers. The research methods training portion was provided in the form of an online summer research workshop, where I was an instructor. It’s incredibly energizing and exciting to talk to a group of research-minded librarians about conducting research to improve professional practice. One theme that arose from our discussions was the use of AI in research design. One of the librarian scholars gave it a try and asked ChatGPT to write survey questions for a research scenario.

Here’s her prompt: I need to write a survey to find out how academic librarians engage in professional development. ACRL will use the survey findings to develop the Guidelines for Professional Development for Academic Librarians, which they believe will provide guidance for academic librarians as they explore professional development opportunities and make professional development plans. Can you write 4 questions for this survey?

ChatGPT’s response:
Certainly! Here are four sample questions for your survey on how academic librarians engage in professional development:

1. How frequently do you engage in professional development activities related to your role as an academic librarian?
• Rarely or never
• Occasionally (1-2 times per year)
• Regularly (3-5 times per year)
• Frequently (more than 5 times per year)

2. What types of professional development activities do you find most beneficial in enhancing your skills and knowledge as an academic librarian? (Select all that apply)
• Workshops and conferences
• Webinars and online courses
• Professional association memberships and events
• Formal education (e.g., degree programs)
• Informal learning (e.g., self-study, reading journals)
• Mentoring and coaching programs
• Other (please specify)

3. How do you typically seek out information about professional development opportunities? (Select all that apply)
• Internal communication channels (e.g., email, intranet)
• External websites and online resources
• Professional networks and colleagues
• Professional association publications and newsletters
• Supervisors and managers
• Other (please specify)

4. What factors influence your decision to participate in a professional development activity? (Select all that apply)
• Relevance to current job responsibilities and career goals
• Availability of funding or financial support
• Recommendations from colleagues or supervisors
• Reputation and credibility of the provider or organization
• Flexibility in terms of time and scheduling
• Recognition or credentialing opportunities
• Other (please specify)

Looking at these survey questions, I think ChatGPT did a nice job in operationalizing the key concepts in the research question based on the research goal. There’s room for improvement in wording survey questions that uphold the basic rules for survey design such as being concise and unambiguous, avoiding leading questions or double barreled questions, and making sure response categories are exhaustive and mutually exclusive. Still, this can be a good start if we are get stuck in our instrument design and don’t know where to begin.

There’s been discussion about using AI in manuscript writing such as this Science article “As scientists explore AI-written text, journals hammer out policies“. Looks like we will need more discussion about using AI in assisting the research process such as designing survey questionnaire, interview guide or a content analysis codebook.

A Great Online Student Conference

I’m the chair of the RSCA Advisory Committee at the College of Professional and Global Education (CPGE) at SJSU. RSCA stands for Research, Scholarship, and Creative Activity. The college held its first Online Student Conference in spring 2022, and our committee served as faculty advisors for this conference. The conference seeks to connect students across the college and promote student work. It aims to provide an opportunity for CPGE students to share their school or professional work, help students communicate and connect with each other, and ultimately foster a stronger sense of community among students.

A total of 16 students submitted recorded presentations to be featured on the conference website. Their presentations covered a wide range of topics ranging from diversity/inclusion to sustainability. Five students did such an excellent job in their presentations that they won the conference’s Outstanding Presentation Award. Let’s take a look at the five award-winning student presentations.

Ashley Minnich’s presentation, ”Mental, Physical, and Social Benefits of Dance: A Client Project“, described a client project where the information request was for scientific and peer-reviewed scholarly literature published within the last 20 years about the mental, physical, and social benefits of dance for the organization to use in future marketing materials. The project was executed by conducting a reference interview with the client, evaluating potential databases for topic relevancy, developing a search strategy and evaluating results, presenting results to the client concisely for their evaluation, and mutually concluding that the information need was met.

Beatriz Martinez delivered a presentation titled “Garden Focus Library Programming: A Green Team Project“. This project emerged from SJSU iSchool’s INFO 282 Libraries Take the Lead on Sustainability. Beatriz worked on a library programming group project. The group developed a garden focused library program that comprises many different forms of activities and events aiming to promote sustainable gardening to all patron demographics.

Rosario Mireles’s presentation, “Generations of Regeneration“, was about a project also resulting from INFO 282 Libraries Take the Lead on Sustainability. This project proposed an action plan targeting community members of all ages (children to grandparents) and addressing the Three Pillars of Sustainability (Environment, Equity, Economic) through regular presentations and interactive workshops at a hypothetical library’s teaching garden.

Katherine Hellman’s presentation, “Weeding is Fundamental! What Comes After Weeding?“, was once again an assignment from INFO 282 Libraries Take the Lead on Sustainability. This project assessed the environmental impact of weeding materials from library collections and offered three sustainable alternatives to sending books and periodicals to landfills.

Carlos Quevedo delivered a presentation titled “Emerging Needs in the Field of Video Game Preservation“. In this presentation, Carlos shared his experience doing research on Digital Media Preservation as it pertains to Video Games through his internship at AFI as well as his role as a Research Assistant to iSchool Professor Darra Hofman. His work enabled him to foresee the future work that needs to be done in order to make the preservation of Video Games a more fruitful endeavor in the future.

In addition to these award winning presentations, other student presentations at the CPGE Online Student Conference also represented the wonderful work our students have done. Congratulations to all the students who presented at the conference and shared their amazing work with the college community and beyond.

Impact of the COVID-19 Pandemic on Chinese College Students’ Information Behavior and Library Needs

Last year, my research collaborators and I conducted a study to examine how the pandemic had impacted Chinese college students’ use of the library. The study took place in April and May 2020. At that time, Chinese college campus had been shut off and all the teaching and learning had been moved online for two months. We collected data using a combination of journaling and in-depth interviews, focusing on the library use and library needs of Chinese college students during the COVID-19 pandemic.

Our findings indicate that students generally lack awareness and understanding of the library’s online resources and would like the library to provide support in ways that could help them enhance productivity and lessen anxiety in the new reality of learning. Libraries should consider making more outreach efforts and offering programs and events to ameliorate isolation and improve students’ sense of community. We hope that our study can provide academic libraries with a nuanced view of user needs and thus help them make informed decisions to serve their campus communities during the unprecedented health crisis. Through the exploration and documentation of college students’ library use and needs, we also hope to document this critical historical event for the library community.

Now our study has been published in the Library Quarterly. Here’s the citation:

Shi, Y., Li, C. & Luo, L. (2021). Impact of the covid-19 pandemic on Chinese college students’ information behavior and library needs: A qualitative study. Library Quarterly, 91(2), 150-171.

I’m really glad that we were able to contribute to the library and information science (LIS) literature with regards to the pandemic. When years later, the pandemic is finally behind us, I think it will be quite interesting to conduct a content analysis of all the pandemic-related studies/articles in peer reviewed LIS journals to capture and document LIS researchers’ collection contributions.

Ideas for the COVID-19 Pandemic Related Research in Librarianship

Earlier this week I had an opportunity to talk to librarians at the University Library at the University of California at San Diego about the trends and methods of research in academic librarianship. They asked wonderful questions, and one of them was about the kinds of research that librarians may consider doing during the pandemic. The COVID-19 pandemic has certainly disrupted our lives in significant ways, but there may also be new research opportunities arising from it. Conducting research related to the pandemic can help library professionals better understand our user needs and provide more meaningful and effective responses to this public health crisis.

A few months ago when the pandemic just broke out, the Journal of the American Society for Information Science and Technology published a commentary titled “Global health crises are also information crises: A call to action” (Xie et al ,2020)“. In this article, a group of information researchers discussed specific things that information scientists can do to “help individuals and society as a whole survive global health crises like COVID‐19, deal with the aftermath, and be better prepared for the next crisis”. They recommended the following research directions:

Misinformation/disinformation particularly during global health crises
Health literacy—including eHealth literacy
Information behavior during lock downs
Vulnerable populations—a case for accessible and usable solutions
Information dissemination, sharing, and integration among multiple forms of digital data
eHealth tools
Predictive methods
Digital archiving
Ethical considerations

I think it would be helpful for library researchers to put out a similar “call to action”, encouraging librarians to investigate research topics related to the pandemic as well. Some of the potential topics may include:

Usage of library services during the pandemic, how it changes from before – especially the use of online library resources and services
Library needs of users during the pandemic – e.g. for academic libraries, how do their students and faculty would like the library to provide support to assist them as they study from home? for special user populations, especially those suffering from the digital divide, what can libraries to ensure equity when providing services during the closure of physical library locations?
Librarians’ well-being – what are librarians’ health and safety concerns with regards to working during the pandemic if they have to return to work?
How librarians can help address misinformation related to the pandemic – what can the library do to better help library users become more critical consumers of information and avoid being victimized by misinformation?
Libraries’ response to the pandemic – did the library have a crisis management/communication plan, how did the library make decisions on their responses to this crisis?

Information behavior during the “Infodemic” – Ideas for academic librarians

This COVID-19 pandemic has unleashed an information crisis. The World Health Organization (WHO) announced that the society not only needs to fight the pandemic, but also the “infodemic,” which it defined as “an overabundance of information—some accurate and some not—that makes it hard for people to find trustworthy sources and reliable guidance when they need it“. This “infodemic” could add another layer of challenge as college students strive to navigate the turbulent time.

College students are likely to fall prey to the spiraling misinformation during the infodemic. Standford researcher Sam Wineburg and his team have repeatedly found that college students were not well-equipped with the ability to critically evaluate online information, and they struggled to effectively evaluate online claims, sources, and evidence. Their research indicated that college students could be deceived by easily manipulated features of websites, such as official-looking logos and domain names, and they tend to read a webpage vertically, staying within a website to evaluate its reliability. In the meantime, high production values, links to reputable news organizations and polished “About” pages were likely to sway students into trusting the contents of the site without much skepticism.

Academic libraries have been increasingly involved in helping college students battle misinformation. Here are a couple of recent articles that explored academic librarians’ efforts on this front: “Academic library guides for tackling fake news: A content analysis” by Sook Lim; and “News Credibility: Adapting and Testing a Source Evaluation Assessment in Journalism” by Piotr S. Bobkowski and Karna Younger.

Yesterday, I read about the following study on TIME magazine, and thought this might be something that academic librarians may find useful as they guide students’ information behavior during the “infodemic”.

“A new paper in Psychological Science explores not only why people believe Internet falsehoods but also how to help them become more discerning about what they share. One of the leading reasons misinformation about COVID-19 gains traction is that it’s a topic that scares the daylights out of us. The more emotional valence something we read online has, the likelier we are to pass it on.

That’s in keeping with earlier research out of MIT, published in 2018, showing that fake news spreads faster on Twitter than does the truth. The reason, the researchers in that study wrote, was that lies are “more novel than true news … [eliciting] fear, disgust and surprise,” just the things that give sharing its zing.

Political leanings also influence sharing. A 2019 Science study, from researchers at Northeastern and elsewhere, showed that neither the left nor the right has a monopoly on sharing fake news; mostly people are just choosing content that fits their ideologies.

To dig deeper still into sharing decisions, Rand and colleagues developed a two-part study. In the first, they assembled a sample group of 853 adults and first asked them to take a pair of tests. One, known as the Cognitive Reflection Test (CRT), measures basic reasoning processes. The other measured basic science knowledge. The sample pool was then divided in half. Both halves were shown the same series of 30 headlines—15 false and 15 true—about COVID-19, but they were instructed to do two different things. One group was asked to determine the accuracy of the headlines. The other was asked if they would be inclined to share the headlines.

The results were striking. The first group correctly identified the truth or falsehood of two-thirds of the headlines. The second group—freed from having to consider accuracy—reported that they would share half the headlines, equally divided between true ones and false ones.

The study did find that people who scored higher on the CRT and basic science tests were a little less indiscriminate. The solution, clearly, is not to force people to pass a reasoning test before they’re admitted online. The second part of the study provided a better answer.

For that portion, a different sample group of 856 adults was again divided in two and again shown the same set of headlines. This time, neither group was asked to determine the accuracy of the headlines; instead, both were asked only if they would share them. But there was still a difference: one group was first shown one of four non- COVID-19-related headlines and asked to determine whether it was true or false. That priming—asking the subjects to engage their critical faculties—made a big difference: the primed group was one-third as likely as the unprimed one to share a false headline.

Thinking critically—especially about the truth of a headline—reduces the spread of fake news.”

Library Responses to COVID-19: What I Observed

The COVID-19 pandemic has been quite disruptive to many aspects of our lives. As we all struggle to manage the new reality, our libraries are also doing their best to continue providing services to their communities during this trying time. I have been following library responses to the COVID-19 crisis on social media and here are some of my observations and experiences.

American Library Association (ALA)’s Pandemic Preparedness page has a lot of great resources that could help libraries better understand how to respond to the crisis.
Libraries establish specific pages that list COVID-19 related resources that are reliable and trustworthy, such as the one from Portland State University Library.
A creative librarian from Hoover Library at McDaniel College put on book displays in his/her own home, reminding people that these books are all available through the library’s eBook collection.

(image source: Hoover Library’s FaceBook Page)
Storytimes have gone virtual! A police officer from Puyallup, WA, a frequent guest at Puyallup Public Library’s storytime, recorded a virtual one for kids.

(image source: Puyallup Police Department FaceBook Page)
Libraries’ 3D printers have new uses now – they can print face shields for health care providers.

(image source: blogto.com)
Although the physical location is closed, some libraries still continue to provide curbside delivery services to help patrons check out materials. Woburn Library even made TikTok video to promote this service to their community!

(image source: Woburn Public Library FaceBook Page)
Libraries are considering turning bookmobiles into free WiFi trucks – what a thoughtful idea!

(image source: vice.com)

Despite all the challenges the pandemic has posed, it does provide opportunities for libraries to highlight their wonderful online resources. My local library, San Jose Public Library, has done a fantastic job in that regard. Since the closure of the library building three weeks ago, they have been sending a weekly email to stay in touch with everybody and encourage people to use their online resources.

Week 1, an email containing Frequently Asked Questions that really helped keeping us informed and putting our minds at ease. It’s a relief to know that all of our book will be automatically renewed till after the library reopens.

week1

Week 2, an email emphasizing SJPL’s online sources specifically oriented to K-12 kids and educators, such as tutor.com and a bunch of EBSCO databases.

week2

Week 3, an email reminding us that if we don’t have a library card already, we can apply for an eLibrary Card online, so that we can freely use the library’s online resources.

week3

The most recent week, an email giving us a summary about how people are actively using the library’s online services such as virtual reference, which has been extended to 24/7, and participating the Spring into Reading program, an program that encourages people to use the library’s eBooks and other eResources.

week4

These emails always bring a smile to my face. 🙂

Our libraries and librarians are the best!

Assessing Scholarly Communication Services: A National Forum in May 2020

I’m a member of a great project team that’s working to put together the National Forum of the Assessment of Scholarly Communications Programs. The project is funded by IMLS and led by wonderful librarians from Sacramento State University and San Jose State University.

The forum will be held online via Zoom on May 4th and 5th. Attendees of the forum will include experts from library assessment that will present and lead discussions on how existing assessment techniques can be implemented for scholarly communication services. The forum will result in a report with recommendations for standards and a comprehensive set of best practices in assessing the range of services that comprise scholarly communication program.

My role in this grant project is to provide methodological expertise in the data gathering and analysis. I have been closely involved in designing and implementing the data collection instruments to gather input from librarians and campus stakeholders that will inform the project goal. We used the Research Lifecycle from the University of Central Florida to frame our data collection instruments, and the data has truly opened my eyes to the wide variety of practices in scholarly communication programs and their role in the campus research enterprise. Can’t wait to hear more from librarians and campus stakeholders at the Forum!

Image result for research lifecycle at university of central florida

New course on health literacy and public library

I taught a new seminar course this summer, discussing the role of public libraries in supporting the community’s health literacy through the provision of consumer health information. I have done some research in this area and have grown increasingly interested in what libraries can do in meeting the community’s health information needs. The Public Library Association has listed “Health” as one of their initiatives, and the National Network of Libraries of Medicine is also actively working with public libraries in the development of consumer health information collections/services/programs. Event ALA’s Libraries Transform Campaign includes “health” as an area where public libraries can make a difference.

In this class, we looked at how public libraries can provide consumer health information from the following perspectives: the public’s health information seeking behavior, health reference services (e.g. key health information resources, evaluation criteria, health reference interview), consumer health collection development, programming, outreach and collaboration, outcome evaluation and needs assessment, and professional development of public librarians. I asked students to do field research and here are some highlights of their observations how consumer health information is provided in public libraries:

Targeting special populations – e.g. a resource center for veterans where health and wellness is a focus
Targeting a particular health issue – e.g. a desert library has a resource center dedicated to cooling, hydration and other heat-related health concerns
Various fitness programs – e.g. yoga, zumba, Taichi
Workshops covering a wide array of health topics – e.g. healthy lifestyle, mental health
Health-themed book displays in the library; online resources (databases, curated list of free websites) also have a designated “health and wellness” section
Strong partnerships with community health stakeholders in the development of programs and services

It was really an interesting course to teach despite the huge amount of work. My next step is to identify public librarians who have gone through MLA’s Consumer Health Information Specialization training and interview them, and their perspective and experience will surely help enrich student learning next time I teach the class.

Institute for Research Design in Librarianship – Raising the Bar of Research in Academic Librarianship

The Institute for Research Design in Librarianship (IRDL) is an IMLS-funded grant program that provides professional development for academic librarians to enhance their research confidence and research competency. Since the summer of 2014, a group of 20-25 academic librarians are selected to participate in the program each year. The program consists of a 7-day bootcamp of research methods (lectures, exercises, consultations) and a support network (e.g. monthly check-ins, mentors) in the following year to help librarians complete their research projects. I’m truly proud to be involved in this project as a lead instructor, and have come to the beautiful campus of Loyola Marymount University in the past six summers to engage in a great experience of teaching research methods with academic librarians from around the country that are talented and motivated to conduct research to improve practice and to enhance the research culture in the profession.

We just concluded the final IRDL bootcamp of the six-year grant program last week. Again I was impressed by the librarian scholars’ variety of research interests and their drive to conduct quality research to raise the bar of LIS scholarship. A few examples of their research topics – how IL instruction can improve student curiosity, how resilience training incorporated in IL instruction may impact underrepresented students’ retention in STEM programs, how film makers/producers seek information in their creative process, what is the techno-stress level of librarians working in digital scholarship, how librarians teach with/about secondary data in their IL instruction, how academic libraries are providing almetrics services, whether scavenger hunt through augmented reality improves students’ perception and satisfaction with library use, and how academic libraries can help students tackle their non-academic needs. I enjoyed my consultations with each one of the scholars as they tried to flesh out the proper research design/methods for their projects.

Although the grant program will end after this year, IRDL has left great legacy. Over one hundred academic librarians have participated in the program and they will become the leaders and influencers in the realm of research and scholarship in academic librarianship. I look forward to seeing them continue to flourish and make impact in the profession as they infuse quality research with their practice.

Finally, a couple of memories from last week:

PhotoJun08,83232AM — Doing the cabbage toss on the last day to review what we have learned

PhotoJun03,55146PM — The spot on LMU campus where we often took our walking breaks – the view is gorgeous

A Crazy Week of Conferencing

I traveled to two international conferences consecutively last week, which was hectic but rewarding. The first conference I attended was the the 2019 Asian Conference on Education and International Development, where I presented my research on how public libraries provide programs to help mitigate summer learning loss among k-12 students (see slides below). This was my second time going to a conference on education, and I really enjoyed all the pedagogical innovations shared by the presenters. Particularly the use of gamification in classroom was intriguing – I have begun to think, in online classes, how could I apply elements of game playing to motivate and engage students?

My second conference was the Design Thinking Global Workshop organized by Guangzhou Library in China, where Chinese public librarians met with Design Thinking experts from Aarhus Library in Denmark, Chicago Public Library and Singapore National Library and experienced the application of Design Thinking in library service/program development. I gave a presentation (in Chinese) on impact assessment and discussed the basic process of measuring the impact of a project (see slides below)- after all, once the library implements a project using the Design Thinking approach, they won’t know how well the project is doing, or how impactful the project is, until proper assessment is conducted.

After the whirlwind of the past week, I’m just glad that it’s spring break this week so I can breathe a bit while catching up on work!