Prof. Toby Breckon

Threat Image Projection in 3D CT Baggage Security Imagery

Issue: Within baggage screening operations, the incidence of actual threat object occurrence is low presenting difficulties for training and competence assessment of human operators.

Approach: Threat Image Projection (TIP) is used into baggage security scanners in order to assess human security operator performance in the detection of threat objects. Using TIP, images of realistic threat objects are automatically projected into the existing scan images of the passenger bags in order to artificially create varied in-situ threat objects in order to monitor operator performance.

By exposing human operators to threat items during their normal screening operations, TIP has the potential to enhance vigilance and attention within overall threat detection performance.

TIP was previously limited to conventional 2D multiple-view X-ray baggage screening systems. Here we extend the TIP concept to threat item insertion in 3D Computed Tomography (CT) screening systems.
[Video shows before (left) and after TIP object insertion (right).]

3 results

2020
[wang20tip]		A Reference Architecture for Plausible Threat Image Projection (TIP) Within 3D X-ray Computed Tomography Volumes (Q. Wang, N. Megherbi, T.P. Breckon), In Journal of X-Ray Science and Technology, IOS Press, Volume 28, No. 3, pp. 507-526, 2020.Keywords: 3D baggage, CT baggage scanning, threat detection in baggage, TIP, threat image projection. [bibtex] [pdf] [doi] [arxiv] [demo]
2013
[megherbi13radon]		Radon Transform based Metal Artefacts Generation in 3D Threat Image Projection (N. Megherbi, T.P. Breckon, G.T. Flitton, A. Mouton), In Proc. SPIE Optics and Photonics for Counterterrorism, Crime Fighting and Defence, SPIE, Volume 8901, No. B, pp. 1-7, 2013.Keywords: 3D baggage, 3D baggage classification, CT baggage scanning, threat detection in baggage, TIP, threat image projection. [bibtex] [pdf] [doi] [demo]
2012
[megherbi12tip]		Fully Automatic 3D Threat Image Projection: Application to Densely Cluttered 3D Computed Tomography Baggage Images (N. Megherbi, T.P. Breckon, G.T. Flitton, A. Mouton), In Proc. Int. Conf. on Image Processing Theory, Tools and Applications, IEEE, pp. 153-159, 2012.Keywords: 3D baggage, 3D baggage classification, CT baggage scanning, threat detection in baggage, TIP, threat image projection. [bibtex] [pdf] [doi] [demo]

Anomaly Detection in Security X-ray Imagery

Deep Convolutional Neural Networks

Issue: anomaly detection within X-ray security imagery as a methodology for concealment detection within complex electronic items.

Approach: we consider varying segmentation strategies to enable the use of both object level and sub-component level anomaly detection via the use of secondary convolutional neural network (CNN) architectures.

Application: Relative performance is evaluated over an extensive dataset of exemplar cluttered X-ray imagery, with a focus on consumer electronics items.

We find that sub-component level segmentation produces marginally superior performance in the secondary anomaly detection via classification stage, with true positive of ∼98%of anomalies, with a ∼3% false positive.

2 results

2019
[bhowmik19subcomponent]		On the Impact of Object and Sub-Component Level Segmentation Strategies for Supervised Anomaly Detection within X-Ray Security Imagery (N. Bhowmik, Y.F.A. Gaus, S. Akcay, J.W. Barker, T.P. Breckon), In Proc. Int. Conf. on Machine Learning Applications, IEEE, pp. 986-991, 2019.Keywords: x-ray security screening, automatic threat detection, anomaly detection, airport security, deep learning, CNN, baggage. [bibtex] [pdf] [doi] [arxiv] [demo]
[gaus19anomaly]		Evaluating a Dual Convolutional Neural Network Architecture for Object-wise Anomaly Detection in Cluttered X-ray Security Imagery (Y.F.A. Gaus, N. Bhowmik, A. Akcay, P.M. Guillen-Garcia, J.W Barker, T.P. Breckon), In Proc. Int. Joint Conference on Neural Networks, IEEE, pp. 1-8, 2019.Keywords: anomaly detection, mask R-CNN, baggage security, x-ray security screening, automatic threat detection, airport security, deep learning, region-based convolutional neural networks, CNN, R-CNN, mask R-CNN. [bibtex] [pdf] [doi] [arxiv]

Prohibited Item Detection in Security X-ray Imagery

Deep Convolutional Neural Networks

Issue: the use of deep Convolutional Neural Networks (CNN) with transfer learning for object classification and detection problems within X-ray baggage security imagery.

Approach: to overcome the limited availability of object of interest (threat) examples within X-ray security screening imagery, we employ a transfer learning paradigm such that a pre-trained CNN, primarily trained for generalized image classification in the photographic image domain can be optimized explicitly to X-ray imagery in the transmission image domain.

Application: an extensive comparison is performed against previous feature-based approaches and with several region-based CNN detection variants.

Evaluation over multiple datasets show robust performance with 0.885 mean average precision (mAP) for a six-class object detection problem, rising to 0.974 mAP for a two-class firearm detection task which translates as a 99%+ probability of detection and less than a 1% false alarm rate for firearm detection.

Additional recent studies explore the transferability (adaptability) of trained CNN models to across varying X-ray scanners, the impact of physical adversarial objects on performance and the use of epipolar geometry to improve cross-view object detection.

7 results

2022
[isaac22multiview]		Multi-view Vision Transformers for Object Detection (B.K.S. Isaac-Medina, C.G. Willcocks, T.P. Breckon), In Proc. Int. Conf. on Pattern Recognition, IEEE, pp. 4678-4684, 2022.Keywords: multi-view x-ray, vision transforms, multi-view geometry, x-ray security screening, automatic threat detection, firearms detection, airport security, deep learning. [bibtex] [pdf] [doi] [software] [talk] [poster]
2021
[bhowmik21energy]		On the Impact of Using X-Ray Energy Response Imagery for Object Detection via Convolutional Neural Networks (N. Bhowmik, Y.F.A. Gaus, T.P. Breckon), In Proc. Int. Conf. on Image Processing, IEEE, pp. 1224-1228, 2021.Keywords: X-ray, false colour, effective-z, object detection, luggage, baggage, aviation security, airport security, deep neural networks, deep learning. [bibtex] [pdf] [doi] [arxiv] [talk] [poster]
2020
[isaac20multiview]		Multi-view Object Detection Using Epipolar Constraints within Cluttered X-ray Security Imagery (B.K.S. Isaac-Medina, C.G. Willcocks, T.P. Breckon), In Proc. Int. Conf. Pattern Recognition, IEEE, pp. 9889-9896, 2020.Keywords: multi-view x-ray, x-ray security screening, automatic threat detection, firearms detection, airport security, deep learning, epipolar geometry, epipolar lines. [bibtex] [pdf] [doi] [talk] [poster]
2019
[gaus19transferability]		Evaluating the Transferability and Adversarial Discrimination of Convolutional Neural Networks for Threat Object Detection and Classification within X-Ray Security Imagery (Y.F.A. Gaus, N. Bhowmik, S. Akcay, T.P. Breckon), In Proc. Int. Conf. on Machine Learning Applications, IEEE, pp. 420-425, 2019.Keywords: x-ray security screening, automatic threat detection, firearms detection, airport security, deep learning, region-based convolutional neural networks, CNN, R-CNN, RetinaNet, baggage. [bibtex] [pdf] [doi] [arxiv] [demo] [poster]
2018
[akcay18architectures]		On Using Deep Convolutional Neural Network Architectures for Automated Object Detection and Classification within X-ray Baggage Security Imagery (S. Akcay, M.E Kundegorski, C.G. Willcocks, T.P. Breckon), In IEEE Transactions on Information Forensics & Security, IEEE, Volume 13, No. 9, pp. 2203-2215, 2018.Keywords: x-ray security screening, automatic threat detection, firearms detection, airport security, deep learning, region-based convolutional neural networks, CNN, R-CNN, R-FCN, RCNN, Faster RCNN, ResNet, YOLO. [bibtex] [pdf] [doi] [demo]
2017
[akcay17region]		An Evaluation Of Region Based Object Detection Strategies Within X-Ray Baggage Security Imagery (S. Akcay, T.P. Breckon), In Proc. Int. Conf. on Image Processing, IEEE, pp. 1337-1341, 2017.Keywords: x-ray security screening, automatic threat detection, firearms detection, airport security, deep learning, region-based convolutional neural networks, CNN, R-CNN, R-FCN, RCNN, Faster RCNN, ResNet. [bibtex] [pdf] [doi] [demo]
2016
[akcay16transfer]		Transfer Learning Using Convolutional Neural Networks For Object Classification Within X-Ray Baggage Security Imagery (S. Akcay, M.E. Kundegorski, M. Devereux, T.P. Breckon), In Proc. Int. Conf. on Image Processing, IEEE, pp. 1057 -1061, 2016.Keywords: convolutional neural networks, deep learning, transfer learning, image classification, baggage X-ray security. [bibtex] [pdf] [doi]

Prohibited Item Detection in Security X-ray Imagery

Traditional Hand-crafted Features

Issue:we explore the use of various feature point descriptors as visual word variants within a Bag-of-Visual-Words (BoVW) representation scheme for image classification based threat detection within baggage security X-ray imagery.

Approach: Using a classical BoVW model with a range of feature point detectors and descriptors, supported by both Support Vector Machine (SVM) and Random Forest classification, we illustrate the current performance capability of approaches following this image classification paradigm over a large X-ray baggage imagery data set.

Application: An optimal statistical accuracy of 0.94 (true positive: 83%; false positive: 3.3%) is achieved using a FAST-SURF feature detector and descriptor combination for a firearms detection task.

The performance achieved characterises the potential for BoVW based approaches for threat object detection within the future automation of X-ray security screening against other contemporary approaches in the field.

2 results

2016
[kundegorski16xray]		On using Feature Descriptors as Visual Words for Object Detection within X-ray Baggage Security Screening (M.E. Kundegorski, S. Akcay, M. Devereux, A. Mouton, T.P. Breckon), In Proc. Int. Conf. on Imaging for Crime Detection and Prevention, IET, pp. 12 (6 .)-12 (6 .)(1), 2016.Keywords: x-ray security screening, automatic threat detection, firearms detection, bag of visual words, feature descriptors, airport security. [bibtex] [pdf] [doi] [demo] [talk]
2013
[turcsany13xray]		Improving Feature-based Object Recognition for X-ray Baggage Security Screening using Primed Visual Words (D. Turcsany, A. Mouton, T.P. Breckon), In Proc. Int. Conf. on Industrial Technology, IEEE, pp. 1140-1145, 2013.Keywords: x-ray, security screening, object recognition. [bibtex] [pdf] [doi]

3D Object Detection and Classification in CT Baggage Security Imagery

Traditional Hand-crafted Features

Issue: the use of automated object segmentation and volume based shape recognition for integration into existing Computed Tomography (CT) based baggage and postal security screening.

Approach: An 3D extension to the established Scale Invariant Feature Transform (SIFT) for the detection of specific complex objects within the same 3D baggage imagery and the subsequent incorporation of this, and alternative CT specific feature point approaches, into a 3D Bag of Visual Words models.

In the video we see an exemplar references 3D gun object matched to multiple locations within an item of baggage being screened (using RANSAC-filtered 3D SIFT point matching). A maximally consistent set of matches is identified from this set to robustly determine the presence of such objects.

Furthermore, we extend the hierarchical multi-layer Visual-Cortex object recognition pipeline (akin in many ways to recent deep learning models) into 3D, using an SVM for final classification. Two object classes, bottles and guns, are examined in this work where comparative results show a clear difference between this hierarchical approach and SIFT/Bag of Words approach with the former producing higher precision and recall coupled with a lower false positive rate.

Application: Recent years have seen increased use of Computed Tomography (CT) based security screening systems for baggage examination to ensure transport security.

5 results

2015
[flitton15codebooks]		Object Classification in 3D Baggage Security Computed Tomography Imagery using Visual Codebooks (G.T. Flitton, A. Mouton, T.P. Breckon), In Pattern Recognition, Elsevier, Volume 48, No. 8, pp. 2489–2499, 2015.Keywords: luggage security, 3D bag of visual words, 3D object recognition, baggage threat detection, density gradient histogram, density histogram, SIFT, RIFT, material density recogniton, airport security, transport security, CT object recognition. [bibtex] [pdf] [doi]
2014
[mouton14randomised]		3D Object Classification in Baggage Computed Tomography Imagery using Randomised Clustering Forests (A. Mouton, T.P. Breckon, G.T. Flitton, N. Megherbi), In Proc. Int. Conf. on Image Processing, IEEE, pp. 5202-5206, 2014.Keywords: . [bibtex] [pdf] [doi] [poster]
2013
[flitton13interestpoint]		A Comparison of 3D Interest Point Descriptors with Application to Airport Baggage Object Detection in Complex CT Imagery (G.T. Flitton, T.P. Breckon, N. Megherbi), In Pattern Recognition, Elsevier, Volume 46, No. 9, pp. 2420-2436, 2013.Keywords: automatic baggage screening, 3D object recognition, 3D baggage classification, CT baggage scanning, threat detection in baggage, 3D SIFT, 3D RIFT. [bibtex] [pdf] [doi] [demo]
2012
[flitton12cortex]		A 3D Extension to Cortex Like Mechanisms for 3D Object Class Recognition (G.T. Flitton, T.P. Breckon, N. Megherbi), In Proc. Computer Vision and Pattern Recognition, IEEE, pp. 3634-3641, 2012.Keywords: hmax, automatic baggage screening, 3D object recognition, 3D baggage classification, CT baggage scanning, threat detection in baggage. [bibtex] [pdf] [doi] [demo] [poster]
2010
[flitton10baggage]		Object Recognition using 3D SIFT in Complex CT Volumes (G.T. Flitton, T.P. Breckon, N. Megherbi), In Proc. British Machine Vision Conference, BMVA, pp. 11.1-12, 2010.Keywords: automatic baggage screening, 3D object recognition, 3D baggage classification, CT baggage scanning, threat detection in baggage, 3D SIFT. [bibtex] [pdf] [doi] [demo] [poster]