CYSR 22:11-12     Home  |   Library  |   Children and Youth Services Review  |   Welfare of Children  
Gateway to resources
  Library  |

Children and Youth Services Review Volume (issue): 22 (11-12) 2000

Assessing Risk in Child Maltreatment
Guest Editors, Eileen Gambrill, Aron Shlonsky

Previous issue | Next issue
Statistical Estimation Of Child Abuse Rates from Administrative Databases 951 -- 971
Douglas G. Simpson, Peter B. Imrey, Olga Geling, Susan Butkus (University of Illinois)

Increasingly, child welfare agencies need to provide statistical summary reports on the safety of the children for whom they are responsible. Often these summaries include ordinary period rates of child abuse derived from administrative data. These indices fail to adjust for the length of exposure to risk, such as time in foster care during the year. Since duration of care often differs by living arrangement, race, gender, and other variables, the use of such period rates to measure child safety or abuse may bias comparisons over time or across groups. This bias may lead to misperceptions of trends in safety over time, and of the comparative safety of different modes of care. This article discusses fundamental issues in the extraction, from administrative data, of valid measures of child welfare outcomes targeted to specific populations. In addition, it provides an introduction to exposure adjustment of child welfare measures based on information that is generally readily available in administrative databases. Cohort-based incidence density rates are recommended in preference to period prevalence from cross-sectional data. Survival modeling/multiple event history analysis is described for more complex situations. The ready availability of such analytic tools suggests further directions for quantitative research in child welfare monitoring.
Looks Can Be Deceiving: Using a Risk Assessment Instrument to Evaluate the Outcomes of Child Protection Services 935 -- 949
Charles Gene Lyle, Elliott Graham (Ramsey County Community Human Services Department)


This study explores the use of a risk assessment instrument based on the Illinois CANTS-17B in the child protection services division of a large urban public social services agency. It addresses the usefulness of the instrument as an outcome measure tool, that is, as means of measuring successful case outcomes based on reductions in maltreatment risk between case opening and case closing. Two separate studies were conducted in which the initial and closing risk levels on the 14-item scale were compared. Results were highly statistically significant in both studies, with the difference being in the expected and desired direction: a decrease in risk scores at case closing. However, a more detailed exploration of the data and of the practice issues involved in the agency setting strongly suggests that these differences are largely due to the artificial inflation of initial risk scores by caseworkers in order to ensure children’s acceptance for ongoing child protection services. These factors are discussed in detail, along with the policy decisions that ensued from the study.

 
An Examination of Relationships Between Children's Protective Services Social Worker Assessment of Risk and Independent LONGSCAN Measures of Risk Constructs 897 -- 933
Diana J. English, J. Christopher Graham (State of Washington Office of Children’s Administration Research )

Risk assessment as a Child Protective Services (CPS) decision-making model has been implemented in the field since the late 1980s. Research activities on risk assessment have included comparative analyses, prioritization, classification studies, implementation, and cultural sensitivity analyses. In the last decade, the major research focus has been on studies related to predictive validity. Little research on risk assessment has focused on other types of construct validity, including convergent validity. Using data from the Longitudinal Studies of Child Abuse and Neglect (LONGSCAN) and CPS investigation records, this study examined the correlations between CPS workers’ ratings of risk on nine risk factors and independent measures of the same risk constructs collected by research interviewers. The data indicate significant correlations with caregiver risk factors such as physical/mental/emotional impairment, however there is little correlation with child risk factors associated with developmental or behavioral issues, or socio-economic factors such as stress and social support. Implications of the findings for policy and practice are discussed.  
Modeling the Reliability and Predictive Validity of Risk Assessment in Child Protective Services 873 -- 896
Michael J. Camasso, Radha Jagannathan

In a time of shrinking resources policy makers and administrators in Child Protective Services are increasingly turning to tools such as structured risk assessment to manage service demand. The reliability and predictive validity of risk assessment is questionable, however, and concerns continue about the validity of using lists of explicit criteria in protective services decision-making. In this research the issues of reliability and validity are addressed using an explicated confirmatory factor analysis model. A sample of 239 cases that included 432 children brought to CPS attention for allegations of physical abuse, neglect and child/family problems are evaluated for risk of abuse or neglect using the Washington State Risk Assessment Matrix (WARM). The study employed a three-wave panel design. Results show that a widely used risk assessment instrument exhibits high levels of measurement error and increasing stability over time, which limit the instrument’s capacity to predict new allegations of abuse and neglect. Measurement error reduces the instrument’s reliability while stability, in light of changes in allegation status and service intensity, reveals a consistency or stiffness that weakens predictive validity. Recommendations are offered for constructing risk assessments that are both psychometrically sound and diagnostically useful.
The Relative Validity of Actuarial- and Consensus-Based Risk Assessment Systems 839 -- 871
Christopher Baird, Dennis Wagner


In an effort to improve decision making in child protective services (CPS), most states have, over the last two decades, implemented risk assessment systems to guide staff faced with making critical decisions in limited time frames.  Generally, these systems are characterized as consensus-based or actuarial models.  This study is the first to directly compare the relative validity of these two approaches.

Three risk assessment instruments, two consensus-based and one actuarial, were completed on cohorts of cases from four different jurisdictions and outcome information was collected over an 18-month follow-up period.  Rates of subsequent investigations, substantiations, and placements were computed for cases classified at low, moderate, and high risk levels in each model.  Results clearly demonstrate that the actuarial approach more accurately classifies cases to different risk levels.  These actuarial models, therefore, have the greatest potential to improve CPS decision making and better protect America ’s at risk children.
Risk Assessment in Context 813 -- 837
Eileen Gambrill, Aron Shlonsky (University of California, Berkeley)

This article provides an overview of the context in which decisions about risk are made in child welfare including personal, task, and environmental factors that may contribute to uncertainty and less-than-optimal decision making, as well as some of the methodological challenges posed by the use of current risk assessment instruments. Actuarial, consensus-based, and clinical instruments are discussed and the more successful track record of actuarial decision-making in child welfare and related fields is highlighted. Methodological challenges to assessing risk are also presented including lack of reliability and validity of measures, definitional dilemmas, temporal issues such as changes in risk over time, absence of base rate data, predicting for individuals and sensitivity and specificity of measures. Implications for the design and implementation of risk assessment tools are considered in light of contextual influences and methodological limitations. Lastly, an overview of the contents of Part One of this special issue on risk assessment is provided.
see also

Assessing and Managing Risk in Child Protective Services Volume 23:1
Guest Editors, Aron Shlonsky and Eileen Gambrill

© Copyright 1999-2002, Elsevier Science, All rights reserved.