U.S. Department of Justice Office of Justice Programs Bureau of Justice Statistics National Crime Victimization Survey Effects of the Redesign on Victimization Estimates April 1997, NCJ-164381 The full text of this report is available through: * the BJS Clearinghouse, 1-800-732-3277 * on the Internet at http://www.ojp.usdoj.gov/bjs/ * on the BJS gopher: gopher://www.ojp.usdoj.gov:70/11/bjs/ * on the National Criminal Justice Reference Service Electronic Bulletin Board (set at 8-N-1, call 301-738-8895, select BJS). By Charles Kindermann, Ph.D. Bureau of Justice Statistics James Lynch, Ph.D. American University David Cantor, Ph.D. Westat The National Crime Victimization Survey (NCVS) -- a major source of the Nation's statistics on criminal victimization -- has undergone an extensive redesign. A collaborative effort on this redesign among several institutions and agencies, including the Bureau of Justice Statistics and the Bureau of the Census, began in the late 1970's and focused principally on improving the accuracy and utility of crime measurement. In 1992 the long-planned redesign of the survey was introduced for half of the sample in such a way that comparisons could be made. This report analyzes the differences in estimates from the two designs. In the discussion that follows, the survey prior to the redesign is referred to as the National Crime Survey (NCS), and that after the redesign is referred to as the National Crime Victimization Survey (NCVS). ****************** Improving the NCS ****************** The NCS, 1973-92, and the NCVS, 1992 to the present, have each year collected crime victimization information from a sample of about 100,000 individuals living in about 50,000 households. For the NCS the victimizations were categorized as personal crimes of violence (rape, robbery, and assault), personal crimes of theft, and household crimes (burglary, household larceny, and motor vehicle theft). In the mid-1970's the National Academy of Sciences reviewed the NCS. ------------------------------------------------------------------------------ Footnote Panel for the Evaluation of Crime Surveys, Bettye K. Eidson Penick, ed., Surveying Crime, Washington, D.C.: National Academy of Sciences, 1976. ------------------------------------------------------------------------------ While the survey was found to be an effective instrument for measuring crime, reviewers identified aspects of the methodology and scope of the NCS that could be improved. The Academy proposed that researchers investigate the following: * an enhanced screening section that would better stimulate respondents' recall of victimizations, thus reducing underreporting due to forgotten incidents * screening questions that would sharpen the concepts of criminal victimization and diminish the effects of subjective interpretations of the survey questions * additional questions on the nature and consequences of victimizations that would yield useful data for analysis. In response, BJS sponsored a research consortium whose purpose was to investigate the issues raised in the review and to make recommendations that would improve the accuracy and utility of the NCVS. The redesign consortium completed its work in 1985. -------------------------------------------------------------------------------- Footnote New Directions for the National Crime Survey, BJS Technical Report, NCJ-115571, March 1989. -------------------------------------------------------------------------------- More recently, the issue of specifically improving the measurement of sex crimes and domestic violence resulted in the formation of a special committee associated with the American Statistical Association's Committee on Law and Justice Statistics. The special committee developed enhanced questions and clarification queries on rape, sexual assault, and domestic violence to get better estimates of these crimes that are difficult to measure. The Bureau of the Census subjected the changes recommended by the special committee, as well as those of the redesign consortium, to additional testing. Modifications proving successful in this testing were introduced into the survey. From January 1992 through June 1993, the full NCS-NCVS sample was divided into two parts. Half of the sample was administered the NCVS method, and the other half, the NCS method. This overlap procedure was designed to permit the continuous publication of estimates of the year-to-year change in crime rates with comparable data while the new design was introduced. The procedure was also intended to provide measurable differences between the halves (table 1). Table 1. Comparison of estimated NCS and NCVS victimization rates, 1992 Number of victimizations per 1,000 persons or households Post- Pre- NCVS/NCS redesign NCVS redesign NCS ratio Personal crimes 49.6 34.4 1.44 * Crimes of violence 47.8 32.1 1.49 * Rape 1.8 0.7 2.57 * Robbery 6.1 5.9 1.03 Assault 40.0 25.5 1.57 * Aggravated 11.1 9.0 1.23 * Simple 28.9 16.5 1.75 * Personal theft 1.8 2.4 0.75 Property crimes 325.3 264.5 1.23 * Household burglary 58.6 48.9 1.20 * Household theft 248.2 195.5 1.27 * Motor vehicle theft 18.5 20.1 0.92 *The ratio of the NCVS to the NCS estimates was statistically significant at the 90-percent level of confidence. ********************************************************* Effects of the new design on estimates of crime rates ********************************************************* In general the redesigned procedures achieved their intended effect of producing higher estimates of crime rates than had the previously used procedures. Estimated rates for the following categories were higher: personal crimes (44% higher), crimes of violence (49%), rapes (157%), assaults (57%), property crimes (23%), burglaries (20%), and thefts (27%). A statistically significant difference could not be found for robbery, personal theft, and motor vehicle theft. The increase in estimates of crimes of violence largely reflected the increase in assault estimates, especially those for simple assaults, which account for 58% of all violent crimes. Simple assaults, the less serious of the assault categories, are committed by persons without a weapon and result in either no injury or a minor injury. Effects of the new design for different types of events The new method results in higher estimates of violent crime rates regardless of the attribute of crime events examined (table 2). Table 2. Effects of the redesign on victimization rates, by selected attributes of crime events for total violent and total household crime, 1992 Total violent crime Total household crime Estmated rates Estmated rates per 1,000 persons per 1,000 households Post- Pre- Standard Post- Pre- NCVS/ Standard redesign redesign NCVS/NCS error (SE) redesign redesign NCS error (SE) Attribute NCVS NCS ratio of the ratio NCVS NCS ratio of the ratio Victim-offender relationship Stranger 26.5 19.5 1.4* 0.09 . . . . . . . . . . . . Nonstranger 22.5 12.8 1.8* 0.13 . . . . . . . . . . . . Completed 15.8 11.7 1.4* 0.11 297.2 234.7 1.3* 0.03 Attempted 33.5 20.4 1.6* 0.11 28.1 29.8 0.9 0.07 Crime reporting to police Reported 21.0 16.0 1.3* 0.10 110.0 96.0 1.1* 0.05 Not reported 27.5 15.7 1.8* 0.12 212.8 165.1 1.3* 0.04 . . .Not applicable. *The ratio of the NCVS to the NCS estimates was statistically significant at the 90-percent level of confidence. However, the new method has a larger impact on the estimates for nonstranger and attempted crimes and crimes not reported to the police than on stranger, completed, and reported crime. The new screening strategy was designed to elicit reports of crime for these categories that were felt to be underreported and appears to have had that effect. For household crimes, the new method results in higher estimates of rates for completed crimes and for crimes either reported or unreported to the police, but the magnitude of the redesign effect is smaller than for violent crime. Nonetheless, the changes that were made to the screening section to improve recounting to interviewers appear to have had some effect. ************************************************************ Effects of the redesign within categories of victims ************************************************************ Ratios of rates from the new method to rates from the old method for selected population groups help to determine if the redesign had a differential effect on population subgroups (tables 3 and 4). To test if there are differential effects, the differences between the ratios were computed and tested for significance. Those that tested significant at the 0.90 confidence level are discussed below. Table 3. Ratios of NCVS rates to NCS rates for violent crimes, by selected victim characteristics, 1992 Ratio of the rates from after the redesign to the rates before the redesign Victim Total violent Assault Personal characteristic crime/a Robbery Aggravated Simple theft Age 12-17 1.6 * 1.3 1.1 1.9 * 1.2 18-24 1.3 * 0.9 1.1 1.5 * 1.5 25-34 1.6 * 1.0 1.4 1.8 * 0.4 * 35-44 1.8 * 0.8 1.2 2.7 * 0.6 * 45-64 1.6 * 1.1 2.0 * 1.6 * 1.1 65 or older 1.1 1.4 1.1 1.0 0.9 Race White 1.6 * 1.0 1.3 * 1.9 * 0.8 Black 1.2 1.0 0.8 1.7 * 0.6 * Other 2.2 * 1.9 2.7 1.9 1.1 Household income 0-$14,999 1.4 * 1.2 1.1 1.5 * 0.7 * $15,000-$34,999 1.9 * 1.1 1.2 1.8 * 0.9 $35,000-$49,999 1.8 * 0.9 1.5 2.4 * 0.4 * $50,000 or over 1.9 * 0.9 1.8 * 2.4 * 1.2 Sex Male 1.5 * 1.0 1.3 * 2.0 * 0.8 Female 1.6 * 1.1 1.1 1.7 * 0.7 * Locality Urban 1.4 * 1.0 1.2 1.8 * 0.6 * Suburban 1.7 * 1.2 1.4 * 2.0 * 1.0 Rural 1.5 * 1.2 1.0 1.7 * 1.1 Note: The standard errors of the estimated ratios are presented at the end of the report. *The ratio of the NCVS to the NCS estimates was statistically significant at the 90-percent level of confidence. /a Because of an insufficient number of cases, rape is not shown separately but is included in total violent crime. ---------------------------------------------------------------------------- Table 4. Ratios of NCVS rates to NCS rates for household crimes, by selected victim characteristics, 1992 Ratio of the rates from after the rede- sign to the rates before the redesign Victimized Total House- Motor household house- hold vehicle characteristic hold crime Burglary theft theft Age of head of household 12-17 3.4 2.3 3.9 . . . 18-24 1.0 0.9 1.0 0.8 25-34 1.2 * 1.1 1.2 * 0.9 35-44 1.2 * 1.2 1.3 * 1.2 45-64 1.4 * 1.5 * 1.4 * 0.8 65 or older 1.2 * 1.4 * 1.2 * 0.8 Race of head of household White 1.2 * 1.1 1.3 * 0.9 Black 1.3 * 1.4 * 1.3 * 0.9 Other 1.2 * 1.4 1.3 * 0.5 Household income 0-$14,999 1.2 * 1.2 * 1.2 * 0.8 $15,000-$34,999 1.2 * 1.1 1.2 * 0.9 $35,000-$49,999 1.4 * 1.4 * 1.5 * 1.0 $50,000 or over 1.3 * 1.2 1.3 * 1.2 Sex of head of household Male 1.2 * 1.1 * 1.3 * 0.9 Female 1.2 * 1.3 * 1.2 * 1.1 Locality Urban 1.2 * 1.3 * 1.2 * 0.9 Suburban 1.3 * 1.2 * 1.3 * 1.0 Rural 1.2 * 1.2 1.4 * 1.2 Note: The standard errors of the estimated ratios are presented at the last of the report. . . .Not applicable. *The ratio of the NCVS to the NCS estimates was statistically significant at the 90-percent level of confidence. The number of respondents who provided data for each particular subgroup varied greatly, depending on the size of the subpopulation. Care should taken, therefore, in interpretating the difference in the ratios without considering the standard error of the ratio. Those ratios that appear to be large may be based on a relatively small number of cases. For example, the standard error for the ratio for households with a head age 12 to 17 is relatively large (SE=2.72 for total household crime), especially for crimes with low prevalence, such as household theft (SE=4.78). (For standard errors, see Appendix tables 2 and 3 on page 7.) In general the redesign had the effect of increasing the number of crimes counted by the survey. In most cases violent crime rates had higher estimates for all groups of respondents when the new methods were used. This was especially true for simple assaults for which nearly every population subgroup that was examined had higher rates when the new method was employed. There was virtually no difference between the new and the old method for robbery for any of the subgroups examined. For crimes of violence, the new procedures increased victim recounting more for -- -- whites than for blacks -- other racial groups than for blacks -- persons age 33 to 44 than for persons 18 to 24 and for persons age 65 or older -- persons with household incomes $15,000 or more than for persons with household incomes below $15,000 -- suburban residents than for urban residents. For robbery and personal theft, the effects of the new procedures did not differ across types of respondents. For aggravated assault, the new procedures increased victim recounting more for whites than for blacks. For simple assault, the increases in victim recall with the new design were greater for -- -- persons age 35 to 44 than for persons of other ages -- persons age 12 to 17 than for persons age 65 or older -- persons with household incomes $15,000 or more than for persons with household incomes below $15,000. In general, except for motor vehicle theft, the new methods had the effect of increasing the number of household crimes recounted to survey interviewers for every group of respondents. Again, all of the following differences in the effect of the new design between respondent groups are statistically significant at the 0.90 level. The new procedures increased recounting more for -- -- suburban residents than for urban residents (total household crime) -- blacks than for whites (burglary). For household larceny and motor vehicle theft, the new procedures had no differential effect across any of the categories in the variables examined. The general pattern of the effects of the new design on the recounting to the interviewers by different groups of respondents was to increase recounting more for traditionally low-victimization groups than for traditionally high-victimization groups. The one exception was the increased recounting of burglary by black victims. Although the recounting of victimization has increased for virtually all groups, these increases were greater for whites than for blacks, for higher-income rather than lower-income groups, and for the middle aged, as opposed to the young or very old. ************************************************* Understanding the effects of design changes ************************************************* The changes made in the crime survey were designed to encourage more complete recall and recounting of crime events. ------------------------------------------------------------------------- Footnote For a discussion of redesign effects not considered in this report, see Michael Rand and Bruce Taylor, "The National Crime Victimization Survey Redesign: New Understandings of Victimization Dynamics and Measurement," Orlando, FL: Annual meetings of the American Statistical Association, August 13-17, 1995. ------------------------------------------------------------------------- Development work done before the redesign indicated that a substantial proportion of crimes were not recounted in the survey for a number of reasons. -------------------------------------------------------------------------- Footnote Albert D. Biderman, et al., Final Report of Research and Development for the Redesign of the National Crime Survey, Washington, DC: Bureau of Social Science Research, Inc., 1986. -------------------------------------------------------------------------- In some cases, the screening interview did not provide enough cues to stimulate respondents to recall and recount eligible events. In others, respondents were uncertain whether they should recount incidents that, although they had all the elements of a crime, did not conform to the stereotype of crime. Crimes committed by family members, for example, fall into this gray area. On the basis of this information, changes were made in the screening interview to provide more cues and to clarify that gray-area events should be recounted. (See the Appendix for the altered screener questions, page 6.) Other development work indicated that Computer Assisted Telephone Inter-viewing (CATI) encouraged the recounting of victimization, presumably because of the enhanced administrative control over the interview process. ------------------------------------------------------------------------ Footnote David Hubble and B.E. Wilder, "Preliminary Results from the National Crime Survey CATI Experiment," New Orleans, LA: Proceedings of the American Statistical Association, Survey Methods Section, August 22-25, 1988. ------------------------------------------------------------------------ CATI ensures that interviewers deliver the cues prescribed in the screening interviews. The proportion of the sample interviewed with CATI also increased. In light of these changes in the design, it is understandable that respondents, in general, recounted more victimizations in the new design than the old. They were given a larger number of cues to assist in the recall and recounting of eligible crime events. CATI also provided greater control over the interview process to ensure that every respondent received all of the additional guidance and cues. It is less clear why these changes in survey design would increase in varying amounts the recounting of some crimes by some groups of respondents, and not others. ************************************************************************ Gray-area events: Incidents failing to fit popular conceptions of crime ************************************************************************ One reason for the differential effects of the new survey procedures across types of events and respondents may be the nature of the cues added to the screening interview. Particular attention was given to cueing for nonstereotypic crimes, such as those that involve offenders who are not strangers. There was good reason to believe that these types of events were not recounted in the old design. The additional cues for recall of these types of events could have produced greater reporting of these crimes. This explanation for the differential recounting of events is consistent with the differences in the patterns of recounting observed in the overlap sample. The effect of the new design was greater, for example, for crimes involving nonstrangers than for those involving strangers. Increases with the new design were greater for attempted as opposed to completed crimes. The new design increased recounting less for stereotypic crimes such as robbery than it did for the more ambiguous crimes such as assault where the inclusion of gray-area events is more of a question. The increased cueing for gray-area events and the subsequent higher rates of recounting in the new design may also explain the apparent differences in the effect of the design for different types of respondents. The increases in recounting for the new design may be less for young black respondents, for example, than for other age and racial groups because more of the violence that afflicts them involves robbery or some equally unambiguous crime than is the case for young white respondents. The appropriateness of this and other explanations for the differences in the effect of the design for different types of respondents can only be determined with further analysis of these data. The observed differences between the old and the new design suggest that the changes in the design had the desired effects. The recounting of victimizations to interviewers increased generally. The new design increased the recounting of events reported to the police as well as those not reported to the police. Increases with the new design were greater, however, for gray-area events than for more stereotypic crimes. Recounting also increased, with a few exceptions, more for respondents with traditionally lower rates of victimization than it did for those with traditionally higher rates. ********* Sources ********* Biderman, Albert D., David Cantor, James P. Lynch, and Elizabeth Martin. Final Report of Research and Development for the Redesign of the National Crime Survey. Washington, DC: Bureau of Social Science Research, Inc., 1986. BJS. New Directions for the National Crime Survey. BJS Technical Report, NCJ-115571, March 1989. Groves, Robert M. and W.L. Nicholls. "The Status of Computer Assisted Telephone Interviewing: Part II-Data Quality Issues." Journal of Official Statistics. Vol. 2, No. 2, 1986, p.117-134. Hubble, David. "The National Crime Victimization Survey Redesign: New Questionnaire and Procedures Development and Phase-In Methodology." Orlando, FL: Annual meetings of the American Statistical Association, 1995. Hubble, David. "National Crime Survey's New Questionnaire Phase-in: Preliminary Results." Tucson AZ: International Conference on Measurement Errors in Surveys, November 11-14,1990. Hubble, David and B.E. Wilder. "Preliminary Results from the National Crime Survey CATI Experiment." New Orleans, LA: Proceedings of the American Statistical Association: Survey Methods Section, August 22-25,1988. Martin, Elizabeth with Robert M. Groves, Jay Matlin, and Carolyn Miller. Report on the Development of Alternative Screening Procedures for the National Crime Survey. Washington, DC: Bureau of Social Science Research, Inc.,1986. Panel for the Evaluation of Crime Surveys, B. K. E. Penick, ed. Surveying Crime. Washington, DC: National Academy of Sciences, 1976. Persley, Carolyn. "The National Crime Victimization Survey Redesign: Meas- uring the Impact of New Methods." Orlando, FL: Annual meetings of the American Statistical Association, August 13-17, 1995. Rand, Michael and Bruce Taylor. "The National Crime Victimization Survey Redesign: New Understandings of Victimization Dynamics and Measurement." Orlando, FL: Annual meetings of the American Statistical Association, August 13-17, 1995.April 1997 NCJ-164381 ****************************************************************** Appendix 1. Comparisons of NCVS and NCS crime screener questions New (NCVS, beginning January 1992) 1. Has anyone attacked or threatened you in any of these ways -- a. With any weapon, for instance, a gun or knife -- b. With anything like a baseball bat, frying pan, scissors, or stick -- c. By something thrown, such as a rock or bottle -- d. Include any grabbing, punching, or choking, e. Any rape, attempted rape or other type of sexual assault -- f. Any face to face threats -- OR g. Any attack or threat or use of force by anyone at all? Please mention it even if you were not certain it was a crime. 2. Incidents involving forced or unwanted sexual acts are often difficult to talk about. Have you been forced or coerced to engage in unwanted sexual activity by -- a. Someone you didn't know before b. A casual acquaintance OR c. Someone you know well. Old (NCS, 1972-92) 1. Did anyone take something directly from you by using force, such as by a stickup, mugging, or threat? 2. Did anyone TRY to rob you by using force or threatening to harm you? 3. Did anyone beat you up, attack you, or hit you with something, such as a rock or bottle? 4. Were you knifed, shot at, or attacked with some other weapon by anyone at all? 5. Did anyone THREATEN to beat you up or THREATEN you with a knife, gun, or some other weapon, NOT including telephone threats? 6. Did anyone TRY to attack you in some other way? 1. Were you attacked or threatened OR did you have something stolen from you -- a. At home including the porch or yard -- b. At or near a friend's relative's, or neighbor's home -- c. At work or school -- d. In place such as a storage shed or laundry room, a shopping mall, restaurant, bank or airport -- e. While riding in any vehicle -- f. On the street or in a parking lot -- g. At such places as a party, theater, gym, picnic area, bowling lanes, or while fishing or hunting. OR h. Did anyone ATTEMPT to attack or attempt to steal anything belonging to you from any of these places? 2. People often don't think of incidents committed by someone they know. Did you have something stolen from you OR were you attacked or threatened by -- a. Someone at work or school -- b. A neighbor or friend -- c. A relative or family member -- d. Any other person you've met or known? 3. Did you call the police to report something that happened to YOU which you thought was a crime? 4. Did anything happen to you which you thought was a crime, but did NOT report to the police? 1. Was anything stolen from you while you were away from home, for instance, at work, in a theater or restaurant, or while traveling. 2. Did you call the police to report something that happened to YOU that you thought was a crime? 3. Did anything happen to YOU that you thought was a crime, but did NOT report to the police? ******************************************************************* Appendix table 2. Standard errors of the estimated ratios NCVS/NCS, presented in table 3 Standard errors of the estimated ratios NCVS/NCS Total Victim violent Assault characteristic crime Robbery Aggravated Simple Personal theft Age 12-17 0.14 0.26 0.18 0.22 0.42 18-24 0.11 0.18 0.16 0.17 0.45 25-34 0.15 0.19 0.23 0.22 0.16 35-44 0.21 0.21 0.22 0.41 0.21 45-64 0.22 0.30 0.52 0.30 0.39 65+ 0.29 0.58 0.51 0.42 0.35 Race White 0.10 0.14 0.14 0.14 0.15 Black 0.12 0.17 0.14 0.28 0.20 Other 0.48 0.88 1.17 0.56 0.73 Household income 0-14,999 0.12 0.19 0.16 0.17 0.19 15,000-34,999 0.15 0.20 0.18 0.19 0.27 35,000-49,999 0.22 0.24 0.33 0.37 0.19 50,000+ 0.22 0.24 0.37 0.35 0.38 Sex Male 0.11 0.13 0.14 0.18 0.20 Female 0.12 0.19 0.16 0.16 0.15 Locality Urban 0.11 0.14 0.16 0.18 0.14 Suburban 0.14 0.20 0.19 0.20 0.23 Rural 0.16 0.37 0.19 0.22 0.60 ******************************************************************** Appendix table 3. Standard errors of the estimated ratios NCVS/NCS, presented in table 4 Standard errors of the estimated ratios NCVS/NCS Victim Total Motor household household Household vehicle characteristic crime Burglary theft theft Age of head 12-17 2.72 2.74 4.78 . . . 18-24 0.04 0.04 0.06 0.16 25-34 0.05 0.10 0.06 0.12 35-44 0.05 0.11 0.06 0.17 45-64 0.06 0.14 0.07 0.12 65+ 0.09 0.19 0.11 0.19 Race of head of household White 0.03 0.07 0.04 0.09 Black 0.06 0.16 0.08 0.14 Other 0.12 0.35 0.15 0.17 Household income 0-14,999 0.05 0.10 0.06 0.11 15,000-34,999 0.05 0.10 0.06 0.12 35,000-49,999 0.07 0.20 0.09 0.18 50,000+ 0.06 0.14 0.06 0.21 Sex of head of household Male 0.04 0.08 0.04 0.09 Female 0.04 0.10 0.05 0.13 Locality Urban 0.04 0.10 0.07 0.10 Suburban 0.05 0.09 0.08 0.12 Rural 0.06 0.13 0.13 0.31 ***************************************************************************** The Bureau of Justice Statistics is the statistical agency of the U.S. Department of Justice. Jan M. Chaiken, Ph.D. is director. BJS Technical Reports provide detailed explanations of statistical issues concerning data collection and analysis. Usually the findings from the data described are reported in BJS Bulletins or Special Reports. This report was written by Charles Kindermann of BJS, James Lynch of American University, and David Cantor of Westat. Tom Hester, BJS, edited and produced the report. Marilyn Marbrook, assisted by Yvonne Boston and Jayne Robinson, supervised final publication. April 1997 NCJ-164381 ****************************** Publication series information This report belongs to the BJS Technical Report series. The previous last publication in the series was Using NIBRS Data to Analyze Violent Crime (October 1993, NCJ-144785). This report is associated with two others in a set devoted to the National Crime Victimization Survey: the Bulletin series, Criminal Victimization, 1994-95 (NCJ-162032) and the Special Report series, Criminal Victimization, 1973-95 (NCJ-163069). Readers who want additional data, analyses, and graphs about criminal victimization in the United States should access the BJS Internet Web site: http://www.ojp.usdoj.gov/bjs/