Tag Archive:
HunchLab

Mining Data for Safer Communities

While a key part of the open government movement is releasing raw data, data on its own is not transformative.   Synthesizing data to gain insights into the communities in which we live is really the key.   This progression to a data-driven society is not only important for those outside of government but also departments within government.

A good example of driving decisions with data is how police departments have leveraged crime data to better police our communities to improve public safety.  Most police agencies examine historic crime data to determine where and when crime hot-spots are occurring and then deploy their resources to react to the changing environment.    Departments are beginning to go beyond looking at simply the dots on the map to also determining what is occurring that is unusual, as well as modeling crime to forecast risk in the coming days and weeks.    It is impossible to address an emerging crime problem without first identifying that something unusual is occurring, and with departments producing ever more volumes of data, law enforcement agencies need tools to automatically sift through datasets to produce synthesized information upon which they can act.   Ideally, this information can also be shared with the public to further accelerate its impact on fighting crime.

We invite you to join us at our next webinar to learn how HunchLab helps police agencies to better protect our communities by mining through historic data and alerting officers when something unusual is occuring in their assigned area.

Please register to join us on Wednesday, August 24, 2011 from 1:00 PM – 2:00 PM EDT:

New HunchLab Website & Webinar

Our web design and marketing teams have been hard at work on a new website for HunchLab, our web-based crime analysis software.  We’re pleased to announce the launch of this new resource.  The product has come a long way in the past few years thanks to the dedicated work of our developers, the support of our partners, and generous funding from the National Science Foundation.

Want to learn more about HunchLab?  Register for our upcoming webinar on March 16th from 1:00pm-2:00pm EDT entitled:

Crime Analysis & Early Warning Made Easy for Officers with HunchLab

HunchLab: Contagious Crime and Genetic Algorithms

Near repeat pattern display within HunchLab

For the past few months, the law enforcement team has been developing analysis and risk forecasting features for HunchLab, our geographic crime analysis, early warning and risk forecasting system,  as part of our latest  National Science Foundation research grant.

As we mentioned in a previous blog post, we’ve been working with Jerry Ratcliffe from Temple University to implement a web-based version of his near-repeat pattern analysis.   But what is a near-repeat pattern and what does it mean to you?

Imagine that your neighbor’s apartment is burgled.  Intuitively, you might imagine that, following this event, the burglar won’t strike nearby right away.  But you would be wrong.  Professor Ratcliffe and his colleagues have found that there is a sort of “contagion” effect to some types of crimes, including burglary, shootings and some types of theft.   This means that there is actually a relatively high risk that you will be re-victimized by a burglar in the few days and weeks after the initial event.  In addition, your neighbors are more likely to be victims as well.

It turns out that this “near repeat” risk can be quantified.  So, now, imagine that your local law enforcement agency could describe the elevated risk of subsequent burglaries and thereby better allocate policing resources to help deter further incidents in the area.  The near-repeat pattern analysis lets us determine how far — in both spatial and temporal measures– this contagion effect extends around a crime incident.  It enables us to identify clustering in historic crime incidents across the city and then apply the model we find to current incidents and generate a map of the risk landscape.  These maps can be used to adjust officer deployment strategies to best mitigate risk.

For example, if your residence is burgled within Philadelphia, your risk of subsequent burglaries within the next 14 days goes up by close to 500%.   Similarly, if your neighbors’ house up to 400 feet away from you is burgled, your risk increases by 28% for the next 14 days.

But wait.  There’s more.  A second feature we’ve been working on is called Hunch Focusing.  Within HunchLab, we cover a police department’s jurisdiction with sets of statistical tests (Hunches) that run on a schedule to determine if abnormal patterns in crime incidents have emerged.  When HunchLab detects an abnormality within a specific Hunch it generates an email alert to the appropriate police staff so that they can figure out how to address it.  For people who have seen HunchLab in action, this is not new – these are features that have been available for a few years.  But what if we could automatically refine the parameters of a Hunch to tell the officer even more about the incidents that led to the alert?   Does the pattern extend to other nearby areas?  Is the pattern durable even if users make small changes in how they are testing the data?

We’re working to help law enforcement officials answer these questions by letting the system “focus” the Hunch, sort of like a camera lense that is being adjusted to bring an image into focus.  To do this, we are applying a technique known as “genetic algorithms“.  This approach takes a particular set of parameters and adjusts or “mutates” them slightly and tests the fitness of the results.  We take the initial Hunch and create a collection of mutations in which we change the parameters slightly.  We then take these mutations and breed them among each other to generate a new collection of Hunches from which we select the best ones.  This process of mutation and breeding occurs through several generations until our outcome is a collection of valid Hunches that are much better than our initial Hunch.  They might show that the pattern extends further North from our initial Hunch or that it persists even if we compare against the last 2 years of crime data instead of only the last year.

We’re excited to share what we’ve been working on within HunchLab with you.  We believe that these more advanced tools will help law enforcement agencies improve public safety for us all.  And we are working on more cutting edge improvements that we’ll unveil over the next few months.  Let us know if you would like to see a demo.

HunchLab: New Face, New Features for Improved Visualization

hunchlab_logoHunchLab, our web-based geographic crime visualization, early warning and risk forecasting software just got a makeover.  As we continue to venture further into new avenues of crime analysis, it’s clear that the ‘big picture’ is rarely just a photo taken from above.  There are many methods for visualizing raw statistics, conducting point analysis, and reporting on the findings. We continue to research the latest methods and have built the strongest of these into HunchLab. These powerful new visualizations are also inspiring new functionality.  Over the past several months, we’ve been exploring application dashboard interfaces which allow for the straightforward display of the results of complex analysis.

Hunchlab Dashboard View

HunchLab's Intelligence Dashboard

HunchLab began as an early warning system that can detect changes in the geographic clustering crime events and then subsequently notify a geographically specific list of users when an emerging cluster is detected.

We have now extended HunchLab to not just help you find hidden trends, but also to visualize the key metrics of your organization.  With the new Intelligence Dashboard, any HunchLab user can interactively view key trends and patterns.

Toward Risk Forecasting
The ‘hunch’ in HunchLab comes from what is known as ‘Abductive’ reasoning.  It is the method of combining intuition (a hunch) with facts towards the production of actionable information.  One of the best known supporters of ‘abductive’ reasoning was the character Sherlock Holmes, a great crime solving inspiration for us here at Azavea.  He might have well been talking about HunchLab when he said “We balance probabilities and choose the most likely.  It is the scientific use of the imagination.”* Although every great crime solver must focus on the circumstances of the individual event, HunchLab is built to examine the aggregate, the big picture.  Its historical statistical analysis has focused on locating spikes and anomalies in the data from the recent past, but what about the future?

Hunch Details View

A point map shows the incidence of crime in a designated geographic area. The Time-of-Day/Day-of-Week chart shows the temporal density of the events that occurred during any particular hour within that timeframe.

The most natural and challenging extension of HunchLab is that of risk forecasting.  We’re investigating the use of volumes of historic data and the best available techniques towards the goal of making certain types of forecasts available to every HunchLab user.  It’s worth noting that we are not pretending we or HunchLab can predict the future.  Perhaps it’s better expressed by Mr. Holmes who said, “while the individual man is an insoluble puzzle, in the aggregate he becomes a mathematical certainty.  You can, for example, never foretell what any one man will do, but you can say with precision what an average number will be up to.  …  So says the statistician.” **  You are probably wondering where we are going with this.  We hope to roll out the specifics over the next year, but we are actively working with criminologists, statisticians and geographers to create a few different types of forecasting approaches including:  near repeat patterns; daily workload forecasting based on shift, day-of-week and season; the impact of non-crime events, such as weather, holidays and sports events; and generation of risk maps based on a variety of factors.

Come talk to us about HunchLab at:

Int’l Assoc. Law Enforcement Intelligence Analysts Conf. (IALEIA) in Orlando, FL – May 3 – 7

Law Enforcement Information Management Conf. (LEIM)  in Atlanta, May 24 -27.

* The Hound of the Baskervilles
** The Sign of the Four

HunchLab: Heat Map and Kernel Density Calculation for Crime Analysis

Hunchlab_heatmap_timeday

Results for a “Hunch” set by a user with crime points overlaid on a heat map that shows crime density in a geographic area of interest. The Time-of-Day/Day-of-Week chart below shows crime density at various points in time at that location.

The purpose of HunchLab, Azavea’s geographic early warning system, is not only to identify changes in the geographic patterns in crime and alert users about detected crime spikes, but also to help users analyze and make sense of information about those spikes.

The “Hunch Detail” section of the application provides several tools geared for exactly that purpose. For the past several months we have made significant changes to it.  In addition to thematic mapping and Time-of-Day/Day-of-Week graphs, the latest upgrade provides a heat map layer.  This feature provides a visualization of crime density in a hunch’s geographic area during a specific timeframe.

When designing this new feature, we faced the question of how to best determine density. One way would be to divide an area into square cells and take a simple count of crimes per cell, but these results may not give a clear indication of actual crime density.  If there are two different cells, neither of which contains any crime points, then the density values for both will be zero.  But if one is in the middle of an area with no crime, while the other is adjacent to a cluster of crimes, this difference should really be reflected.  So HunchLab makes use of a more sophisticated process known as “kernel density estimation” which overcomes this problem

“What is kernel density analysis?” you may ask.  Well, rather than treating each crime event as if it only impacted the exact point where it occurred, the effect of each event is spread over the surrounding area.  The distance over which the effect is spread can vary and is specified by a parameter known as the “search radius”.  A “kernel” defines the pattern to use for distributing the event’s impact.  Several kernels are available, each with a different pattern.  The simplest kernel distributes the impact uniformly over the surrounding area, but most use mathematical formulas to give a greater weight closer to the crime event itself.

We’ve not only added this new density calculation feature to HunchLab but also made it into a reusable library that we can add to other applications.  The new library leverages the Azavea Raster Grid (ARG) format already developed for Azavea’s DecisionTree product as well as other investments from previous projects.  In HunchLab, users can adjust several parameters, such as the search radius to use during analysis and the rendering approach to use for displaying the results (equal interval or quantile).  The heat map layer is just one of several tools available to help users analyze the crime spikes that HunchLab detects, with even more on the way!

Azavea Wins Another Prestigious SBIR Award from NSF for HunchLab – Leveraging Spatial Statistics to Validate Human Intuition and Fight Crime

"The ability to detect and analyze changes in the geographic patterns of crime and disorder is an innovation in policing which holds the potential to enhance the organizational capacity of police departments across the country."

Just over a year ago, we proudly announced that Azavea had been awarded a Small Business Innovation Research (SBIR) Phase I grant from the National Science Foundation (NSF) for the development of our HunchLab product, a set of innovative software tools that scour the current and historic data of a police department, search for changes in geographic patterns, apply spatial statistics to test for significance, and send alerts to relevant law enforcement personnel. Today, we are thrilled to announce that NSF has awarded us Phase II funding to further development of this software and its commercialization.

Our Phase I project proved the feasibility of building the application, and the Phase II project will refine the application and build additional functionality, including alternate workflows for different user types, developing a new user interface, expanding the alert infrastructure, and building text mining capabilities. The result will be a commercial version of the software.

Preventing crime is a more sophisticated task than simply mapping incidents or arrests and deploying resources accordingly. The ability to detect and analyze changes in the geographic patterns of crime and disorder is an innovation in policing which holds the potential to enhance the organizational capacity of police departments across the country.

HunchLab was inspired by the Crime Spike Detector that Azavea developed to help the Philadelphia Police Department identify when and where unusual increases in crime are occurring. Read our white paper for more information. The Crime Spike Detector, which has been in operation since June 2005, uses a spatial statistics algorithm developed in conjunction with Dr. Tony Smith (University of Pennsylvania) to compare current crime to historical crime across the city. Each night this ‘data mining’ service checks for spikes in different types of crime. Unusual increases result in an email being sent to the relevant district captain. The email details the severity of the spike and links to an online report with maps, charts and tables, enabling analysis of the result. .

Although HunchLab will initially be developed to assist with crime detection, tools such as the Spike Detector and HunchLab are applicable in any application where events display geographic changes in distribution, such as disease occurrence, consumer buying patterns, real estate sales, and property and mortgage fraud.

HunchLab is supported by the Small Business Innovation Research program of the National Science Foundation, Directorate for Engineering, Division of Industrial Innovations and Partnerships, Award Number (IIP-0750507).

This is the fourth time in two years that Azavea has been awarded an SBIR grant. Previous awards were SBIR Phase I awards from the National Science Foundation, the U.S Department of Education and the U.S Department of Agriculture.

SBIR Grant Award Announcement: HunchLab – Leveraging Spatial Statistics to Validate Human Intuition

As part of their daily activities, police officers often formulate hunches based on observations and other sources of information. Large amounts of crime data already exist in electronic form, so officers have been using information management systems and visualization tools to help sift through this data. Despite the availability of these tools, hunches remain difficult to confirm or deny.

We are pleased to announce that the National Science Foundation recently awarded Azavea a Phase I Small Business Innovation Research (SBIR) grant to design and evaluate ‘HunchLab’, a prototype system that will enable police officers to develop and evaluate hunches.

‘HunchLab’ was inspired by the Crime Spike Detector that Azavea developed to help the Philadelphia Police Department (PPD) identify when and where unusual increases in crime are occurring. The Crime Spike Detector, which has been in operation since June 2005, uses a spatial statistics algorithm developed in conjunction with Dr. Tony Smith (University of Pennsylvania) to compare current crime to historical crime across the city. Each night this ‘data mining’ service checks for spikes in different types of crime. Unusual increases result in an email being sent to the relevant district captain. The email details the severity of the spike and links to an online report with maps, charts and tables, enabling analysis of the result (learn more). Although ‘HunchLab’ will initially be developed to assist with crime detection, tools such as the Spike Detector and ‘HunchLab’ are applicable in any application where events display geographic changes in distribution, such as disease occurrence, consumer buying patterns and real estate sales.

‘HunchLab’ is supported by the Small Business Innovation Research program of the National Science Foundation, Directorate for Engineering, Division of Industrial Innovations and Partnerships, Award Number (IIP-0637589).