Hate Crime Classification using NDQIS

Hate crime classification using NDQIS

June 2022 was a significant milestone in the timeline of LGBTQ+ rights in the UK. Not only was it Pride Month, but this year marked the 50th anniversary of the first Pride event to be held in the UK. Back in 1972, the parade in London attracted around 2,000 participants, whereas over 1.5 million people were expected to visit the city for this year’s parade. This in itself is worthy of a blog post, as Datactics prides itself on the diversity of its workforce and the inclusivity within our growing teams. However not only is this a cause which we can all support, but analysing crimes against this demographic is something in which we are actively engaged on a daily basis.  All too frequently we hear about news of attacks on members of the LGBTQ+ community: physical attacks on individuals, offensive content posted online, vandalism to property and so on. As much as we’d like to think that we’re a more tolerant society than we used to be, unfortunately the data suggests otherwise. Recorded hate crimes motivated against a person’s sexual orientation have increased every year from 2016.[1] 

So where does Datactics come in, and what difference are we making? 

Firstly, let me speak bluntly. We’re not on the front line pursuing offenders or supporting victims. This responsibility lies with a wide range of people from police officers to magistrates and from charity workers to counsellors, all of whom have my utmost respect and admiration for the challenging work which they do. Instead, we’re directly involved in the development of a service which will provide the government and police constabularies with tangible data relating to hate crime. Areas of concern will consequently be identified, which in turn will influence local and national policies and feed into reporting for the Office for National Statistics, support decision-making and ensure that resources are deployed effectively and in a timely manner. I’m referring to the National Data Quality Improvement Service- NDQIS.

Hate crime classification using NDQIS

From our previous blog posts, you may be aware that NDQIS is already being used within the context of knife enabled crime in England and Wales,[2] and working closely with crime specialists in the Home Office, we are now making excellent progress in developing equivalent processes for analysing hate crime. In a nutshell, the NDQIS system ingests police reports and uses preconfigured logic to categorise these as being hate crimes or not. It does so by using semantic analysis[3] of “plain English” free text crime records, combined with labels which have been used to flag the offence type and any relevant aggravating factors. Often this is a fully automatic, computer assisted classification process, but for the more ambiguous records, manual review by a data steward may be required. This person will be a suitably trained member of the police force which recorded the crime, and will be able to use their experience and judgement to determine whether or not a hate crime is likely to have occurred. 

What exactly constitutes a hate crime? 

While I have so far discussed hate crimes based upon a person’s sexual orientation or gender identity – or even perceived persuasions and identities – there are, of course, people within our society who are victims of hate crime for other reasons. The most obvious of these groups are offences involving people from different racial backgrounds or those who hold differing religious beliefs. Statistically, crimes against racial and/or religious groups have also been increasing. By way of example, crime statistics of physical assaults on people of the Jewish faith rose by 78% between 2020 and 2021.[4] Here, NDQIS continues its categorisation further, to identify which hate crime category is relevant for each police report, bearing in mind that people can be victimised for multiple reasons, in which case a single report may appear within multiple categories. The following diagram illustrates this flow of data: 

Hate crime classification using NDQIS - National Data Quality Improvement Service

As can be seen above, there are currently five monitored categories of hate crime, four of which I have already mentioned: race, religion, sexual orientation and transgender. The fifth, but certainly no less important category relates to crimes against people with a disability. Despite a 9% reduction in disability hate crimes being reported nationally between 2020 and 2021,[4] there is clearly still work to be done, as some regions observed an increase in such crimes. For example, Cheshire reported a 22% increase over the same period.[5] 

In recent times, there have been discussions concerning the possibility of adding additional categories. Most recently, the UK government rejected a Private Members’ Bill which would have seen misogyny recognised as a motivating factor in the context of hate crime.[6] Following the tragic death of her daughter Sophie in August 2007, the late Sylvia Lancaster OBE – who sadly died in April – campaigned to recognise alternative subcultures within hate crime legislation.[7] Closer to home, here in Northern Ireland, sectarianism is an additional monitored hate crime category,[8] however NDQIS isn’t currently used by the Police Service of Northern Ireland. 

Should data for any of these categories – or indeed any other category – be required in the future, Datactics will be ready to implement changes to the NDQIS system, to support law enforcement and government legislators with their guidance and policy implementation.  

Final word… 

I began this blog by talking about Pride, so I’ll wrap up by expressing my own pride in playing a small but hopefully beneficial part in helping to steer policies in a direction which will help in tackling hate crime and benefit the victims, and prevent such crimes from happening to others. I may be a single cog within a complex machine, but if the machine can lead to positive and meaningful changes within our society, then I’m all for it. As the saying goes: hate can hurt, but never win.  

Have you been the victim of a hate crime or want to report a hate crime? Please speak to the police. You should call 101, or 999 in an emergency. Alternatively, contact True Vision, Stop Hate UK or Crimestoppers

And for more from Datactics, find us on LinkedinTwitter or Facebook.



  1. LGBT Tolerance ‘going backwards’ as hate crimes upBBC News, 4th February 2022 
  1. Knife Crime – Using Technology to improve results at ONSRoisin Floyd, Datactics, 25th June 2021 
  1. Semantic Analysis of Free Text and Why It MattersShauna Leonard, Datactics, 8th June 2022 
  1. Antisemitism Worldwide Report 2021Tel Aviv University, 27th January 2022 
  1. Cheshire hate crime rise sees more disabled people targetedBBC News, 26th January 2022 
  1. Ministers to reject making misogyny a hate crime in England and WalesThe Guardian, 21st February, 2022 
  1. SLF TimelineSophie Lancaster Foundation, 2020 
  1. Incidents and Crimes with a Hate Motivation recorded by the Police in Northern Ireland – PSNI, 31st March 2022. 

Photograph credits:  

Clem Onojeghuo 

Fotis Fotopoulos 

James Eades 

Used under license. 


Get ADQ 1.4 today!

With Snowflake connectivity, SQL rule wizard and the ability to bulk assign data quality breaks, ADQ 1.4 makes end-to-end data quality management even easier.