There is a limit

California wildfire researchers work with NASA's scientific data

by Gregory Saville

Yesterday I watched NASA’s meteorologists mapping California’s wildfires with LiDAR and climate sensors to help reduce risks of damage and death in the future. It is remarkable what scientists can do when they use obvious, decent data. As a university undergrad, I studied physical geography and meteorology. That's why the NASA documentary resonated. In university, and in the video, I learned that it is high-quality data that makes the difference. 

Geographers study things like weather patterns, how epidemics spread, and how water controls our lives. Geography was the first science in the early environmental movement with climate studies on global warming long before it was rejected by extremist Climate Deniers. 

Like the NASA video, it is all amazing stuff. At the core of it is real-life, measurable data with observable patterns. 

Imagine my shock when I transferred to a degree program in urban studies, city development, and urban planning. Social geography data was so very different than science data! Social data is not always obvious, well-known, or observable. It uses surveys, tax, census, land use, and economic and demographic data. Social data can be difficult to collect and much of it is incomplete. Useful theories exist only after decades of research.

The very best applied and theoretical science at NASA's
Johnson Space Center's Mission Control Center - author NASA

CRIME PATTERNS 

In criminology, the situation is worse. Crime studies usually emerge from police data - in other words, citizen reports to the police. Sometimes this shows basic patterns and sometimes not. If citizens don't trust the police,  they may not call the police for anything. How can we know if unreported incidents distort the real story? 

Criminologists conduct self-report surveys and victim surveys to close the gap. However, the dirty secret of crime patterns is that crime data is horrid. Since data is the basis on which scientific theory proceeds, unreliable data cannot produce good science. If you thought weather prediction was a fuzzy science, at least it uses decent data. 

Crime pattern studies, sadly, almost exclusively rely on police reports to analyze crime. In SafeGrowth we use a method called the risk assessment matrix to collect different forms of physical, social, and cultural data along with police stats. It's not perfect, but it is not dependent on police stats. 

Now imagine crime prediction! Researchers serve us terminologies like time decay factors, graph signal processing, and the deep generative adversarial network. They slice and dice data into sophisticated algorithms and impressive 3-D graphs. All of this emerges, of course, from the worst imaginable data (at least in scientific terms). My geography classes seldom mentioned crime data. Now I know why.  

 

Fortune-telling attempts prediction without data. This Japanese fortune-telling
service had no luck predicting the demise of its own advertising tower
- Author hanonimas, Creative Commons

RECENT FINDINGS

I still keep up with some of the latest crime research – a recent study on predicting crime with time and location data and another trying to predict homicide locations.

Homicide research is the type I trust most (it's difficult to ignore a murder victim and most get reported). The homicide prediction study concludes:

“Our results are important because, from a policy perspective, optimal deployment of scarce police resources should be guided by high resolution spatial strategies”

Hm… 

Well, first, police resources should be guided by evidence-based policy driven by reliable data! That is a long way off. 

Second, predictions from mathematical algorithms and graph signal processing should be based on real-life context. For example, will deploying police resources to the right places cut homicides? If alleyway bar fights or outdoor robberies cause the majority of homicides, then police patrols and arrests might work. Policing gang behavior to cut outdoor gang shootings will definitely help.

Unfortunately, studies reveal that, in the US and most other Western countries, most murder victims know their attackers, many offenses occur inside the domestic household, or the victim and attacker live in the same neighborhood. It’s hard to imagine how police deployment will make much difference. 

NASA Sunspot predictions for Solar Cycle 23 and 24 from
solar observation data - Creative Commons

HOMICIDE RESEARCH

One homicide study puts it this way: 

“Responses [to homicide] must place more responsibility on communities for identifying disputes at an early stage in their development, before they turn fatal.”

Since most homicides happen in the same troubled neighborhoods over and over, strategies like SafeGrowth build capacity in troubled neighborhoods to help residents, with their local police, to transform violence-breeding neighborhoods into livable, crime-resistant places. 

Not much prediction is needed there…just municipal services, competent training, and resources to get started. I believe strongly in research and evidence to create viable prevention programs. But let’s not get ahead of ourselves with delusions of scientific prediction when the data is so poor and well-known solutions are already at hand. 

Let's admit the truth of crime data and the limit to what it says.