Research, evaluation, and program planning all require good quality data collection. For good quality data collection, you must know what data you need, how you will collect it, and how you will ensure consistency (for example, that all the people answering a question interpret it in the same way). Some data can be collected through observations or document review. Data collection often requires the use of “instruments” such as surveys or questionnaires, interview guides, and focus group protocols. A program logic model can help you understand the data you need to collect; it helps clarify what your project intends to do and what it hopes to accomplish, as well as document the potential impact to the agency. That information can reveal the information you need to collect about your activities and results.
Resources on instrument design, testing, and implementation are provided below along with tips for developing a logic model for your program. Developing quality instruments can be a challenging process; starting with a few examples can make it easier. CVR has a collection of sample instruments measuring victimization, community and victim needs, satisfaction with victim services, and victim outcomes, as well as some sample logic models. You can search this collection below. We are also continuing to build this collection, and welcome additional submissions.
Tools and Tips
Questionnaire Design (Pew Research Center). This guide provides tips for creating questions that accurately measure the opinions, experiences, and behaviors of individuals. This guide details the important steps involved in questionnaire design including question development, use of open- and closed-ended questions, question wording, question order, and pilot testing and conducting focus groups and pretests to examine how individuals respond to the questions.
Cognitive Interviewing: A “How To” Guide (Research Triangle Institute). Cognitive interviewing is a process to make sure people understand what the questions are asking and are interpreting them the same way you are. This guide walks you through the process of testing your questions through cognitive interviewing.
Assessing Community Needs and Resources (Center for Community Health and Development). This guide contains 24 sections detailing how to identify and assess community concerns, needs, and assets. This guide includes information on how to collect your own data (primary data collection) and how to find and use existing data (secondary data collection). Detailed guidance on how to develop and implement data collection through focus groups, surveys, and interviews is provided.
Evaluation Toolkit: Using a Logic Model (The Pell Institute and Pathways to College Network). While the Pell Institute concentrates on education issues, we found this resource very helpful. This guide explains the logic model and walks you through the five basic elements (inputs, activities, outputs, outcomes, and impact). Be sure to check out the How to Create a Logic Model section for tips on how to make one for your program.
National Crime and Victimization Data Analysis Tools
Two national data collections on crime and victimization are conducted through the Department of Justice, the Federal Bureau of Investigation’s Uniform Crime Reporting (UCR) program and the related National Incident-Based Reporting System (NIBRS) used in many jurisdictions; and the Bureau of Justice Statistics’ National Crime Victimization Survey (NCVS). A brief description of the UCR and NCVS can be accessed here. The FBI provides more details on NIBRS on its website. A more in-depth introduction to the NCVS can be found in this webinar presentation by staff from the Bureau of Justice Statistics.
The FBI is developing its Crime Data Explorer (CDE) to make its UCR data more accessible. The CDE includes UCR data collected under the traditional summary system as well as the detailed NIBRS format. The CDE tool allows users to search, sort and compare national and state data as well as agency-level crime statistics. It also provides a way to download selected reports and data. Details about the CDE and the UCR data included in the tool are available here.
The Office of Juvenile Justice and Delinquency Prevention (OJJDP) hosts an “Easy Access to NIBRS” data tool. This tool focuses specifically on victims of violence and capitalizes on many of the details collected by NIBRS including victim demographics, victim-offender relationship, weapon, and residential location. OJJDP provides information on data decisions made in creating the underlying NIBRS data accessed by the tool as well as a glossary of terms for concepts used in the tool. Reading and understanding these methods will help users accurately interpret their findings.
BJS created the NCVS Victimization Analysis Tool (NVAT). Users can analyze violent and property victimization by select victim, household, and incident characteristics. Users can access preset “quick” tables that provide trends in crime and reporting to the police as well as enter customize searches to generate tables that meet their needs. The NVAT User’s Guide is a helpful resource for those new to data tools and the NCVS. The NVAT also provides users with documentation about its methodology and definition of terms used in the tool.