Research methods in library assessment

I gave a workshop about research
methods in library assessment at Santa Clara University Library earlier this
month. It was a wonderful opportunity to connect with SCU librarians. The
workshop focused on two things: 1) how to develop the research question in a
library assessment project; and 2) how to determine the proper research methods
for conducting the assessment. Research question develop is the critical first
step in any research project – Albert Einstein once said “If I had an hour to
solve a problem I’d spend 55 minutes thinking about the problem and 5 minutes
thinking about solutions.” In the workshop, we first talked about the progressive
focusing process of formulating the research question (overall goal –>
objectives –> research questions), and discussed the criteria we can use to
evaluate the quality of a research question. ACRL’s 2015 January Assessment in
Action Report titled “ACADEMIC LIBRARY CONTRIBUTIONS TO STUDENT SUCCESS:
Documented Practices from the Field” provided some examples of research
questions in library assessment, and we spent some time examining each one of
them (see the picture below) – it was a great way to look closely at the process of developing research
questions. Then we reviewed the major research designs (based on the analytic
objective, time dimension, and degree of variable manipulation) and the frequently
used qualitative and quantitative methods in library assessment. Quantitative
methods seem to be more popular in assessment projects.

I’m glad to have this opportunity
to chat with SCU librarians about research methods – I admire their initiative
to engage in library assessment using valid and reliable research methods. This
workshop made me think about, as LIS research methods instructors, what we can
do to work more closely with libraries and librarians that conduct assessment.
Looks like I need a lunch date with some of my fellow methods instructors and
my assessment librarian friends. 🙂

A recent international research collaboration

International collaboration is always a refreshing and even
enlightening experience to me. I have worked with two librarians from Tsinghua
University Library in China on a couple of projects and absolutely enjoyed it. I
have known them for more than 10 years so we have a very efficient and pleasant
relationship. Last year when I was at IFLA,
I met a librarian from Ghana and we had good conversations about library
research. This spring we worked on a project together to evaluate the reference
services at University of Education, Winneba (UEW) in Ghana. We decided to
approach the evaluation from the user perspective, and identified the Key
Performance Indicators (KPIs) based on the RUSA Guidelines
for Behavioral Performance. A user survey was conducted to measure the KPIs
and to examine how users use and perceive library reference services. The
findings were illuminating. For example, regarding how the reference service
should be improved, some users recommended that librarians be better trained.
It’s interesting that these users realized that the service inadequacies were a
result of insufficient personnel training. It is also interesting to note that
personal interests is the second most popular motivation for users to use the
reference service, which is auspicious and shows that students and faculty
trust reference librarians with their personal information needs. And yet, some
of the things we take for granted such as electricity and computers/copiers,
were still a concern for some library users at UEW in Ghana – they felt there
weren’t enough of them to meet their needs.

Overall the study was a great learning experience for me. I’m
glad that our findings will help UEW library determine the service areas where
improvements are most needed and develop necessary training programs to address
them. On a personal note, the study opened my eyes to how reference services
work at an academic library in Ghana, and how library users are experiencing the
services. It will be published in Library Review soon, so keep an eye on it if
you are interested.

2015 Institute of Research Design for Librarianship

Last week we concluded the second
year of the Institute of Research Design for
Librarianship (IRDL), an IMLS-funded project that seeks to create a
learning opportunity and a support system for academic librarians who want to
improve their research skills and increase their research output. It’s another
great 9-day institute with librarian scholars from around the country. This
year, the project directors made some changes based on the feedback from last
year’s cohort so that the scholars could have more time to write and consult
with us instructors. I truly enjoyed talking to each scholar about their
research projects. I was impressed by the innovation and initiative
demonstrated in their projects – for example, one scholar was interested in
implementing a texting-based outreach and reference service and see if that
impacts students’ library anxiety, and another scholar was thinking about
creating library tutorials using Vine and see if that would be an effective way
to impart information literacy skills. I can’t wait for them to complete their
study and get published.

Among this year’s cohort, three
were graduates from my home institution, SJSU School of Information, and two of
them took classes with me – one was in my Research Methods class, and another
was in my Reference and Information Services class. It’s great to see them
become active researchers in this profession and engage in evidence based
practices to improve their work. I’m very proud of them. This is exactly what I’m
hoping to see through research methods education both in LIS degree programs
and beyond (like IRDL).

The campus of Loyola Marymount
University was as beautiful as ever. I feel lucky that IRDL is held at LMU library, in a spacious room with a
gorgeous view. Many thanks to Marie and Kris, the two wonderful IRDL project
directors, who took good care of us while we were there. Looking forward to
next year already!

Research topics from my research methods class in fall 2014

image

 My research methods class in fall 2015 focused on research. Students in the class were expected to develop a survey research proposal. Here’s a collection of their survey study topics. Many of them are needs assessment and program/service evaluation studies.

  • Does attending the “Sopa de letras” workshop series for parents cause an increase in behaviors that are considered beneficial for literacy and learning of children?
  • What programs would college bound teens like to see the library incorporate into its services in order to help them with college preparation?
  • Do series-based library workshops improve the job skill sets of unemployed or low-income adults?
  • Why are patrons using or not using the library databases?
  • What kinds of professional development opportunities should the library offer to help teachers better understand the services the library could offer in relation to the Common Core State Standards?
  • What services (staff preparedness, physical layout), collections and programs do public libraries in the state of California offer their visually impaired patrons?
  • What is the effectiveness of the Mill Valley Public Library’s current marketing tactics in their efforts to market library programs and events?
  • At what rates do elementary school librarians and teachers use digital audio players and media for audio reinforcement over physical audio players and media, or vice versa, and why?
  • What about the existing library teen services/collection/spaces/programs do KDBS students think is adequate, inadequate, and needs improvement?
  • Does the pedagogical approach of Reading Workshop impact the range of genre selection for independent reading for upper elementary students (3rd-5th grade) from their school library? 
  • How do musicians perceive the usability of the intrinsic qualities of digital scores? 
  • What are the reasons for immigrant populations’ non-use of the library? 
  • Do adults have interest in attending arts classes at the Cambridge Public Library? 
  • What genre/types of books are young adults most interested in?
  • Does creating a young adult space in a library enhance the overall library experience for people in this age group? 
  • Do Spanish-speaking patrons feel their public libraries provide materials and programming that reflect their needs and interest of their communities?
  • What specialized accommodations for existing services do higher education academic libraries provide for students with auditory and visual disabilities?
  • Is there a relationship between academic librarians’ instructional practices and their background in and knowledge of instructional pedagogies, design, and standards? 
  • What are the reasons of teen use or non-use of the teen section of the Monmouth Public Library website? 
  • Does Facebook marketing contribute to library program attendance? 
  • Why are the communities of Latino/Hispanic parents not aware that public libraries offer bilingual children’s books?
  • What is the awareness of grandparents raising grandchildren regarding public services in the Boise, Idaho area? 
  • What services, collections, and programs are libraries offering to better serve patrons with autism spectrum disorder (ASD)? Are library staff members receiving ASD awareness training? 
  • What marketing practices are currently employed by public libraries to promote their digital information service? 
  • What factors do college students consider when choosing to read a book for pleasure? 
  • How has training in assistive resources, or lack thereof, of Maricopa County Librarians impacted their ability to assist patrons with disabilities? 
  • Do information literacy (IL) courses provided by the public library during school visits improve teenage patrons’ ability to assess and utilize the information from the public library’s digital resources? 
  • What information resources do teenagers use when visiting the public library? How satisfied are teens with the results they received while interacting with those information sources? 
  • What factors influence the participation of ethnic communities during public library story times? 
  • How satisfied are San Diego Public Library patrons with the library’s job-seeking resources?
  • How efficiently are the services and information resources provided by the Chicago Public Library system satisfying the needs and concerns of the lesbian, gay, bisexual, transgender and queer/or questioning community in Chicago, Illinois? 
  • What diabetes resources are a necessary part of a health library collection in order to best serve the Hispanic population to manage diabetes? 
  • What are public library staff member’s opinions and levels of satisfaction with the following aspects of the Symphony software interface: aesthetics and functionality? 

Setting the research agenda

image

A few weeks ago I gave a talk at SJSU Gateway PhD students’ virtual residency about how to set the research agenda. To prepare for that talk, I looked back in the past 11 years and thought about how I have been planning, conducting and disseminating my research since I was a doctoral student. I was able to summarize a few useful (hopefully) tips from my experience and share them with our PhD students. I’m posting them here too.

1. Set aside blocks of time designated for “research thinking”. I often do my “thinking” while I’m cleaning the house or cooking – such thinking could be about anything related to research (e.g. ideas for the next research project, how to interpret the data in a completed project and what arguments to make, etc.)

2. Reading favorite journals/blogs/Websites to keep up to date with research developments in one’s field.

3. Record/organize all research-related thoughts/ideas for later consideration – I use Evernote, but a traditional notebook will do too.

4. Join/form a journal club with colleagues/students – exchanging critical evaluation of the published literature with my peers is intellectually stimulating.

5. Serve on the editorial board or as a peer reviewer for journals/grants/conferences – this is a great opportunity to see the most recent research in one’s field.

6. When attending conferences/workshops, it is worth paying attention to work-in-progress presentations.

7. If interested in a research topic, it would be helpful to teach a graduate seminar course on that. Teaching and research are mutually beneficial.

8. Be flexible – set long term goals and short term objectives, and conduct elf-examination at regular intervals and adjust accordingly.

9. Develop a writing routine – this is hard and takes much self-discipline.

10. Quantify the publication targets – for example, every year aim for X number of journal articles, X number of conference presentations, etc.

11. Identify 3-5 journals or conferences for research dissemination – be familiar with their topical coverage, editorial style, submission guidelines and everything else that needs to be noted.

12. Collaborate – potential collaborators may be colleagues, students, researchers connected at conferences or even from other disciplines.

13. Enhance research profile via various online venues – social media (e.g. blog/tweet about your research, create a presence on ResearchGate, academia.edu or Google Scholar), institutional repository, publisher’s marketing tools

14. Be active and be part of the ongoing scholarly conversation/debate.

Practicing qualitative research

Earlier this week I moderated a workshop about qualitative research as part of the event “Research Day” at Loyola Marymount University Library. The goal of “Research Day” was to raise librarians’ awareness and understanding of qualitative research. My workshop was to provide an opportunity for librarians to practice in-depth interview and focus group interview – two prevalent qualitative research methods. I only had 3 hours, so I thought a lot about how to best make use of the time. Finally I decided to follow an organic four-step process as I do in my research methods class – 1) the participants would start by developing a qualitative research question; 2) then based on their research question, they would chose either in-depth interview or focus group interview as the data collection method; 3) they would then develop an interview guide and practice conducting the interview; and 4) finally they would reflect upon their experience and talk about their understanding of qualitative research.

Overall I was able to stick to my agenda – but at the end we weren’t able to spend as much time on reflection as I had planned. Step 3 was monstrously time-consuming. Still, I felt the workshop gave everyone a basic sense of what it’s like to conduct qualitative research. I created four handouts to guide the participants through the four steps, and I’m posting them here. Hopefully it will be beneficial to other people who are also interested in practicing qualitative research.

Sources:

Rubin, H. J., & Rubin, I. S. (2011). Qualitative Interviewing: The Art of Hearing Data. Thousand Oaks, CA: Sage.

Guest, G., Namey, E., & Mitchell M. (2013). Collecting qualitative data: A field manual for applied research. Thousand Oaks, CA: Sage.

Krueger, R., & Casey, M. (2014). Focus groups: a practical guide for applied research (5th ed.). Thousand Oaks, CA: Sage.

ITIE 2014

Earlier this week I attended the Information Technology in Education (ITIE) 2014 Conference. It’s a biennial conference organized by the Evergreen Education Foundation (EEF) and its library partners in China. This year the conference was held in Changsha and the partnering library was Hunan Provincial library. The conference theme was Oral History in Libraries. Many rural school library projects funded by EEF in the past two years are oral history projects, where school librarians work with teachers and students to collect and document oral history. I was very fortunate to serve as a grant reviewer for these projects and was impressed by the diversity and depth displayed. I got to meet several of the grant awardees at the conference and learn about their project progress. How wonderful!

The conference invited two venerable scholars as keynote speakers – Dr. Don Ritchie, US Senate Historian, and Dr. Weiming Tu, a renowned ethicist, Professor of Philosophy and founding Dean of the Institute for Advanced Humanistic Studies at Peking University. Their enlightening talks about oral history research and spiritual humanism (respectively) introduced me to philosophies and paradigms I knew very little about. I also made connections with a number of librarians and library directors from China. I had great conversations with them and learned a lot about their visions and ideas regarding library development in China. The Director of Guangzhou City Library (a very large metropolitan public library), from the photo below, was a trail-blazer and he even hired an American librarian to work in the “services for diverse populations department”. Many exciting things are happening in the library arena in China.

My own presentation, “Digital storytelling in the Library”, also went pretty well. I became interested in this topic when exploring how to use digital storytelling as a means to enhance health literacy. Hopefully I will have more opportunities to work on this topic. I find Daniel Pink’s book “A whole new mind” quite inspiring when it comes to the power of storytelling. I recommended it in my presentation, but I don’t know if it has been translated into Chinese. I hope Yes!

2014 IFLA

Last week I went to IFLA for the satellite meeting on information ethics, and the offsite summit on library education. It was a great trip – particularly at the satellite meeting, I met researchers and practitioners from all over the world and had engaging discussions with them about ethical issues in libraries. My own presentation at the meeting was about the ethical dilemmas reference librarians encounter at work. Reference librarians are at the forefront of libraries’ public service. It is imperative for them to maintain the highest possible standards of diligence and ethical conduct under time restraints that often require compromise. In my teaching of reference and information services, ethics is a critical topic. However, I haven’t been able to find any good readings on this topic, so I decided to conduct a survey study to explore how reference librarians handle ethical dilemmas at work, write a paper about it, and then use it for my reference class. The findings were very illuminating – I shared them in a 20-min presentation, and then a 40-min discussion ensued among the audience. This format of presentation was quite unique – it’s more like a seminar, and the presentation is to set the tone for an in-depth discussion on the topic, which actually worked quite well for me and gave me a lot to think about. One of the researchers I met there suggested that everybody studying ethics should study philosophy first, and he himself is currently getting a master’s degree in philosophy – how amazing!

The summit on library education was organized by the IFLA Section on Education and Training. We had a good turnout, and I was one of the eight presenters in the Ignite session about the future of LIS education. It was a little stressful to do an Ignite presentation for the first time. In my five minutes, I told a futuristic story of how a 23-year old girl named Alice, interested in digital knowledge management in the farming industry, receives her education in LIS in 2050. It went actually pretty well – thanks to the power of storytelling! The summit gathered LIS educators from different countries. It’s eye-opening to hear them talk about what LIS education is like in the middle east, Africa, south America, east Asia and other regions. I also met two people from North Korea – they were both wearing pins of Kim Jong Il and Kim Jong Un. I tried to talk to them but they basically ignored me. I wonder how libraries work and how LIS education is provided in North Korea. Hmm…..

Well, as always, IFLA was a great conference to attend, despite sitting next to a 10-month old sick baby for 9 hours and getting vomited on twice on my way home. Oh well. The fall semester starts today, and I’d better get ready for the hecticness.

Formulating the research question

image

Last week, students in our SJSU Gateway PhD program gathered in San Jose for a one-week residency. This year we have 9 incoming students, and it’s great to chat with them about their research interests and share their passion and excitement of embarking on a new journey. I have two new students this year – one of them is interested in study the relevance of information literacy skills, and another one seeks to better understand the role of medical librarians in improving the public’s health literacy. We spent quite a bit of time talking about developing their research questions in the past week.

The formulation of research question, is probably the utmost important step in a research process. It anchors the entire research study. Yet I have seen too many studies that proceeded without a clearly defined research question. I always ask students to think about what they want to achieve with their study , what is the research problem they hope to resolve, and what their research objectives are. Answers to these questions are helpful for us to understand what exactly it is we try to study, and hence formulate a research question to properly reflect that. Once we have the research question, we can then move on to operationally define each concept and variables in the question, and then start thinking about research design and methodology.

In the many research methods textbooks I have read, research question formulation and development is a topic that’s often lacking. One book that addresses it relatively well is Babbie’s “The Practice of Social Research” (I wrote a blog post about it). And that’s one of the reasons I choose it to be the textbook for my research methods class. The new semester is starting in just a couple of weeks, and in the fall, I will be teaching a new research methods course that focuses on survey research. One of the course assignments is to complete a proposal for a research study that’s appropriate for survey research. Thus, students will need to come up with a research question that has to be answered by a survey study. I wonder how that will impact the question development process. Will make it harder or easier? I think it’s the latter. We’ll see.

2014 Institute for Research Design in Librarianship

imageIn the last two weeks of June, the first IRDL cohort gathered in the beautiful library of the beautiful Loyola Marymount University and went through an intensive professional development program focusing on research design and methodology (see my previous post about this program). As a research methods geek, I was thrilled and honored to be one of the instructors there. I was impressed with the IRDL scholars’ motivation, interest and enthusiasm in conducting quality research and enhancing the rigor of LIS research and publications. Their research topics covered a variety of LIS practices, including the quickly emerging area of data librarianship. During my first two days there, I got to talk with all four members of the cohort whose topics were related to data management, data literacy and data service. Our conversations were illuminating and gave me a lot to think about in terms of preparing out students for this area. Among the twenty-five scholars, four were SJSU SLIS alum – one of them took the Reference and Information Services course with me, and another one was my advisee. It’s great to see them at IRDL and chat with them about their experience at SLIS. I’m so proud of what they have accomplished and what they will be accomplishing after IRDL.

There so many memorable moments throughout the program. The highlight was the scholars’ presentations on the last day, where they talked about their research idea and how it had taken shape during the two weeks. It’s very rewarding to see that they are leaving IRDL with a fairly concrete research plan and ready to dive in and churn out peer-reviewed articles one after another. Hopefully by the end of the three-year IRDL grant, we will be able to compile a list of publications resulting from the scholars’ research projects at IRDL.

In addition to the great discussions and interactions with the IRDL scholars and my co-instructors Greg and Michael, I really appreciated how well LMU library had taken care of us – the classroom was spacious with a great view, the catering service was wonderful, and our host Kris, Marie and Carol were so thoughtful and warm. I can’t thank them enough for making this experience a great memory of summer 2014.  I’m already looking forward to 2015!

Oh, before I forget, I tweeted more in the past two weeks than I did in the previous five years, and I think I will keep tweeting at that frequency. Social media, I’m serious about you now.