advanced
Volume 20 Issue 5
October  2023
Turn off MathJax
Article Contents
Relative Articles
Cheng-Cheng Ma, Bao-Yuan Wu, Yan-Bo Fan, Yong Zhang, Zhi-Feng Li. Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients. Machine Intelligence Research, vol. 20, no. 5, pp.666-682, 2023. https://doi.org/10.1007/s11633-022-1328-1
Citation: Cheng-Cheng Ma, Bao-Yuan Wu, Yan-Bo Fan, Yong Zhang, Zhi-Feng Li. Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients. Machine Intelligence Research, vol. 20, no. 5, pp.666-682, 2023. https://doi.org/10.1007/s11633-022-1328-1
shu

Effective and Robust Detection of Adversarial Examples via Benford-Fourier Coefficients

doi: 10.1007/s11633-022-1328-1
  • 1.

    National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China

  • 2.

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China

  • 3.

    School of Data Science, The Chinese University of Hong Kong, Shenzhen 518172, China

  • 4.

    Shenzhen Research Institute of Big Data, Shenzhen 518172, China

  • 5.

    AI Lab, Tencent Inc., Shenzhen 518057, China

More Information
  • Author Bio:

    Cheng-Cheng Ma received the B. Sc. degree in automation from Northwestern Polytechnical University, China in 2017. Currently, he is a Ph. D. degree candidate of Institute of Automation, Chinese Academy of Sciences, China. His research interests include adversarial learning and machine learning. E-mail: machengcheng2017@ia.ac.cn ORCID iD: 0000-0002-0502-3960

    Bao-Yuan Wu received the Ph. D. degree in pattern recognition and intelligent systems from National Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences, China in 2014. From November 2016 to August 2020, he was a senior and principal researcher at AI Lab, Tencent Inc., China. Currently, he is an associate professor of School of Data Science, The Chinese University of Hong Kong, Shenzhen (CUHK-Shenzhen), China. He is also the director of the Secure Computing Lab of Big Data, Shenzhen Research Institute of Big Data (SBRID), China. He has published more than 50 top-tier conference and journal papers, including TPAMI, IJCV, NeurIPS, CVPR, ICCV, ECCV, ICLR, AAAI, and one paper was selected as the Best Paper Finalist of CVPR 2019. He is currently serving as an Associate Editor of Neurocomputing, Area Chair of ICLR 2022, AAAI 2022 and ICIG 2021, Senior Program Committee Member of AAAI 2021 and IJCAI 2020/2021. His research interests include AI security and privacy, machine learning, computer vision and optimization. E-mail: wubaoyuan@cuhk.edu.cn (Corresponding author) ORCID iD: 0000-0003-2183-5990

    Yan-Bo Fan received the B. Sc. degree in computer science and technology from Hunan University, China in 2013, and the Ph. D. degree in pattern recognition and intelligent systems from Institute of Automation, Chinese Academy of Sciences, China in 2018, He is currently a senior researcher at AI Lab, Tencent Inc., China. His research interests include computer vision and machine learning. E-mail: fanyanbo0124@gmail.com ORCID iD: 0000-0002-8530-485X

    Yong Zhang received the Ph. D. degree in pattern recognition and intelligent systems from Institute of Automation, Chinese Academy of Sciences, China in 2018. From 2015 to 2017, he was a visiting scholar with the Rensselaer Polytechnic Institute, USA. He is currently with AI Lab, Tencent Inc., China. His research interests include computer vision and machine learning. E-mail: zhangyong201303@gmail.com ORCID iD: 0000-0003-0066-3448

    Zhi-Feng Li received the Ph. D. degree from The Chinese University of Hong Kong, China in 2006. After that, he was a postdoctoral fellow at The Chinese University of Hong Kong, China, and Michigan State University, USA for several years. He is currently a top-tier principal research scientist with Tencent, China. Before joining Tencent, he was a full professor with Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, China. He was one of the 2020 Most Cited Chinese Researchers (Elsevier-Scopus) in computer science and technology. He is currently serving on the editorial boards of Neurocomputing, IEEE Transactions on Circuits and Systems for Video Technology, and Pattern Recognition. He is a Fellow of British Computer Society (FBCS). His research interests include deep learning, computer vision and pattern recognition, and face detection and recognition. E-mail: michaelzfli@tencent.com ORCID iD: 0000-0001-5902-5067

  • Received Date: 2021-12-31
  • Accepted Date: 2022-03-24
  • Publish Online: 2022-04-25
  • Publish Date: 2023-10-01
    Abstract
  • Adversarial example has been well known as a serious threat to deep neural networks (DNNs). In this work, we study the detection of adversarial examples based on the assumption that the output and internal responses of one DNN model for both adversarial and benign examples follow the generalized Gaussian distribution (GGD) but with different parameters (i.e., shape factor, mean, and variance). GGD is a general distribution family that covers many popular distributions (e.g., Laplacian, Gaussian, or uniform). Therefore, it is more likely to approximate the intrinsic distributions of internal responses than any specific distribution. Besides, since the shape factor is more robust to different databases rather than the other two parameters, we propose to construct discriminative features via the shape factor for adversarial detection, employing the magnitude of Benford-Fourier (MBF) coefficients, which can be easily estimated using responses. Finally, a support vector machine is trained as an adversarial detector leveraging the MBF features. Extensive experiments in terms of image classification demonstrate that the proposed detector is much more effective and robust in detecting adversarial examples of different crafting methods and sources compared to state-of-the-art adversarial detection methods.

     

    • 11 The base number can be other numbers than 10. See Table 7 for ablation study on base number.
    22 https://pytorch.org/docs/robust/torchvision/models.html3 https://foolbox.readthedocs.io/en/v1.8.04 https://github.com/rfeinman/detecting-adversarial-samples5 https://github.com/pokaxpoka/deep_Mahalanobis_detector6 https://github.com/xingjunm/lid_adversarial_subspace_detection7https://www.mathworks.com/help/stats/fitcsvm.html
    48 https://image.baidu.com9 https://www.facebook.com
    510 https://github.com/kabkabm/defensegan
    611 https://github.com/s-huu/TurningWeaknessIntoStrength
    712 https://github.com/yk/icml19_public13 https://github.com/jayaram-r/adversarial-detection
    314 https://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.stats.ks_2samp.html
    * These authors contributed equally to this work.
    FullText(HTML)
  • loading
    • References(50)
  • [1]
    K. Simonyan, A. Zisserman. Very deep convolutional networks for large-scale image recognition. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2015.
    [2]
    A. Krizhevsky, I. Sutskever, G. E. Hinton. ImageNet classification with deep convolutional neural networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, USA, pp. 1097–1105, 2012.
    [3]
    G. Huang, Z. Liu, L. van der Maaten, K. Q. Weinberger. Densely connected convolutional networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Honolulu, USA, pp. 2261–2269, 2017. DOI: 10.1109/CVPR.2017.243.
    [4]
    K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 770–778, 2016. DOI: 10.1109/CVPR.2016.90.
    [5]
    X. O. Tang, Z. F. Li. Video based face recognition using multiple classifiers. In Proceedings of the 6th IEEE International Conference on Automatic Face and Gesture Recognition, IEEE, Seoul, Republic of Korea, pp. 345–349, 2004. DOI: 10.1109/AFGR.2004.1301555.
    [6]
    D. H. Gong, Z. F. Li, J. Z. Liu, Y. Qiao. Multi-feature canonical correlation analysis for face photo-sketch image retrieval. In Proceedings of the 21st ACM International Conference on Multimedia, ACM, Barcelona, Spain, pp. 617–620, 2013. DOI: 10.1145/2502081.2502162.
    [7]
    Z. F. Li, D. H. Gong, Y. Qiao, D. C. Tao. Common feature discriminant analysis for matching infrared face images to optical face images. IEEE Transactions on Image Processing, vol. 23, no. 6, pp. 2436–2445, 2014. DOI: 10.1109/TIP.2014.2315920.
    [8]
    Z. Y. Deng, X. J. Peng, Z. F. Li, Y. Qiao. Mutual component convolutional neural networks for heterogeneous face recognition. IEEE Transactions on Image Processing, vol. 28, no. 6, pp. 3102–3114, 2019. DOI: 10.1109/TIP.2019.2894272.
    [9]
    H. B. Qiu, D. H. Gong, Z. F. Li, W. Liu, D. C. Tao. End2end occluded face recognition by masking corrupted features. IEEE Transactions on Pattern Analysis and Machine Intelligence, to be published. DOI: 10.1109/TPAMI.2021.3098962.
    [10]
    X. L. Yang, X. H. Jia, D. H. Gong, D. M. Yan, Z. F. Li, W. Liu. LARNet: Lie algebra residual network for face recognition. In Proceedings of the 38th International Conference on Machine Learning, pp. 11738–11750, 2021.
    [11]
    S. Q. Ren, K. M. He, R. Girshick, J. Sun. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 91–99, 2015.
    [12]
    W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, A. C. Berg. SSD: Single shot MultiBox detector. In Proceedings of the 14th European Conference on Computer Vision, Springer, Amsterdam, The Netherlands, pp. 21–37, 2016. DOI: 10.1007/978-3-319-46448-0_2.
    [13]
    R. Feinman, R. R. Curtin, S. Shintre, A. B. Gardner. Detecting adversarial samples from artifacts, [Online], Available: https://arxiv.org/abs/1703.00410, 2017.
    [14]
    X. J. Ma, B. Li, Y. S. Wang, S. M. Erfani, S. N. R. Wijewickrema, G. Schoenebeck, D. Song, M. E. Houle, J. Bailey. Characterizing adversarial subspaces using local intrinsic dimensionality. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018.
    [15]
    M. K. Varanasi, B. Aazhang. Parametric generalized Gaussian density estimation. The Journal of the Acoustical Society of America, vol. 86, no. 4, pp. 1404–1415, 1989. DOI: 10.1121/1.398700.
    [16]
    C. Pasquini, F. Pérez-González, G. Boato. A Benford-Fourier JPEG compression detector. In Proceedings of IEEE International Conference on Image Processing, IEEE, Paris, France, pp. 5322–5326, 2014. DOI: 10.1109/ICIP.2014.7026077.
    [17]
    V. N. Vapnik. The Nature of Statistical Learning Theory, New York, USA: Springer, 1999.
    [18]
    F. J. Massey Jr. The Kolmogorov-Smirnov test for goodness of fit. Journal of the American statistical Association, vol. 46, no. 253, pp. 68–78, 1951. DOI: 10.1080/01621459.1951.10500769.
    [19]
    X. Li, F. Li. Adversarial examples detection in deep networks with convolutional filter statistics. In Proceedings of the IEEE International Conference on Computer Vision, pp. 5764–5772, 2017.
    [20]
    K. Pearson. LIII. On lines and planes of closest fit to systems of points in space. The London,Edinburgh,and Dublin Philosophical Magazine and Journal of Science, vol. 2, no. 11, pp. 559–572, 1901. DOI: 10.1080/14786440109462720.
    [21]
    J. J. Lu, T. Issaranon, D. Forsyth. SafetyNet: Detecting and rejecting adversarial examples robustly. In Proceedings of IEEE International Conference on Computer Vision, IEEE, Venice, Italy, pp. 446–454, 2017. DOI: 10.1109/ICCV.2017.56.
    [22]
    J. H. Metzen, T. Genewein, V. Fischer, B. Bischoff. On detecting adversarial perturbations. In Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017.
    [23]
    I. J. Goodfellow, J. Shlens, C. Szegedy. Explaining and harnessing adversarial examples. In Proceedings of the 3rd International Conference on Learning Representations, San Diego, USA, 2014.
    [24]
    N. Papernot, P. McDaniel, S. Jha, M. Fredrikson, Z. B. Celik, A. Swami. The Limitations of Deep Learning in Adversarial Settings. In Proceedings of IEEE European Symposium on Security and Privacy, IEEE, Saarbruecken, Germany, pp. 372–387, 2016. DOI: 10.1109/EuroSP.2016.36.
    [25]
    N. Carlini, D. Wagner. Towards evaluating the robustness of neural networks. In Proceedings of IEEE Symposium on Security and Privacy (SP), IEEE, San Jose, USA, pp. 39–57, 2017. DOI: 10.1109/SP.2017.49.
    [26]
    K. Grosse, P. Manoharan, N. Papernot, M. Backes, P. McDaniel. On the (statistical) detection of adversarial examples. [Online], Available: https://arxiv.org/abs/1702.06280, 2017.
    [27]
    R. Z. Gao, F. Liu, J. F. Zhang, B. Han, T. L. Liu, G. Niu, M. Sugiyama. Maximum mean discrepancy test is aware of adversarial attacks. In Proceedings of the 38th International Conference on Machine Learning, pp. 3564–3575, 2021.
    [28]
    N. Carlini, D. Wagner. Adversarial examples are not easily detected: Bypassing ten detection methods. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, ACM, Texas, USA, pp. 3–14, 2017. DOI: 10.1145/3128572.3140444.
    [29]
    Z. H. Zheng, P. Y. Hong. Robust detection of adversarial attacks by modeling the intrinsic properties of deep neural networks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 7924–7933, 2018.
    [30]
    K. Lee, K. Lee, H. Lee, J. Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 7167–7177, 2018.
    [31]
    K. Roth, Y. Kilcher, T. Hofmann. The odds are odd: A statistical test for detecting adversarial examples. In Proceedings of the 36th International Conference on Machine Learning, Long Beach, USA, pp. 5498–5507, 2019.
    [32]
    J. Raghuram, V. Chandrasekaran, S. Jha, S. Banerjee. A general framework for detecting anomalous inputs to DNN classifiers. In Proceedings of the 38th International Conference on Machine Learning, pp. 8764–8775, 2021.
    [33]
    D. Hendrycks, K. Gimpel. Early methods for detecting adversarial images. In Proceedings of the 5th International Conference on Learning Representations, Toulon, France, 2017.
    [34]
    T. Y. Pang, C. Du, Y. P. Dong, J. Zhu. Towards robust detection of adversarial examples. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, Canada, pp. 4584–4594, 2018.
    [35]
    P. Samangouei, M. Kabkab, R. Chellappa. Defense-GAN: Protecting classifiers against adversarial attacks using generative models. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018.
    [36]
    S. Y. Hu, T. Yu, C. Guo, W. L. Chao, K. Q. Weinberger. A new defense against adversarial images: Turning a weakness into a strength. In Proceedings of Advances in Neural Information Processing Systems, Vancouver, Canada, pp. 1633–1644, 2019.
    [37]
    F. Pérez-González, G. L. Heileman, C. T. Abdallah. Benford′s Lawin image processing. In Proceedings of 2007 IEEE International Conference on Image Processing, IEEE, San Antonio, USA, pp. I-405–I-408, 2007. DOI: 10.1109/ICIP.2007.4378977.
    [38]
    I. S. Gradshteyn, I. M. Ryzhik. Table of Integrals, Series, and Products, Cambridge, UK: Academic Press, 2014.
    [39]
    A. Papoulis. Probability, Random Variables, and Stochastic Processes, New York, USA: McGraw-Hill, 1965.
    [40]
    A. Kurakin, I. J. Goodfellow, S. Bengio. Adversarial examples in the physical world. Artificial Intelligence Safety and Security, R. V. Yampolskiy. Ed., New York, USA: Chapman and Hall/CRC, pp. 99–112, 2018.
    [41]
    A. Krizhevsky, V. Nair, G. Hinton. Cifar-10 (Canadian institute for advanced research), [Online], Available: https://academictorrents.com/details/463ba7ec7f37ed414c12fbb71ebf6431eada2d7a.
    [42]
    Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In Proceedings of NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Canada, 2011.
    [43]
    J. Deng, W. Dong, R. Socher, L. J. Li, K. Li, L. Fei-Fei. ImageNet: A large-scale hierarchical image database. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Miami, USA, pp. 248–255, 2009. DOI: 10.1109/CVPR.2009.5206848.
    [44]
    S. M. Moosavi-Dezfooli, A. Fawzi, P. Frossard. DeepFool: A simple and accurate method to fool deep neural networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, IEEE, Las Vegas, USA, pp. 2574–2582, 2016. DOI: 10.1109/CVPR.2016.282.
    [45]
    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, A. Vladu. Towards deep learning models resistant to adversarial attacks. In Proceedings of the 6th International Conference on Learning Representations, Vancouver, Canada, 2018.
    [46]
    A. Creswell, T. White, V. Dumoulin, K. Arulkumaran, B. Sengupta, A. A. Bharath. Generative adversarial networks: An overview. IEEE Signal Processing Magazine, vol. 35, no. 1, pp. 53–65, 2018. DOI: 10.1109/MSP.2017.2765202.
    [47]
    L. Deng. The MNIST database of handwritten digit images for machine learning research. IEEE Signal Processing Magazine, vol. 29, no. 6, pp. 141–142, 2012. DOI: 10.1109/MSP.2012.2211477.
    [48]
    N. E. Lasmar, Y. Stitou, Y. Berthoumieu. Multiscale skewed heavy tailed model for texture analysis. In Proceedings of the 16th IEEE International Conference on Image Processing, IEEE, Cairo, Egypt, pp. 2281–2284, 2009. DOI: 10.1109/ICIP.2009.5414404.
    [49]
    M. Rosenblatt. A central limit theorem and a strong mixing condition. Proceedings of the National Academy of Sciences of the United States of America, vol. 42, no. 1, pp. 43–47, 1956. DOI: 10.1073/pnas.42.1.43.
    [50]
    N. R. Goodman. Statistical analysis based on a certain multivariate complex Gaussian distribution (An introduction). The Annals of Mathematical Statistics, vol. 34, no. 1, pp. 152–177, 1963. DOI: 10.1214/aoms/1177704250.
    • Relative Articles
    • Supplements(0)
    • Cited By
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(5)  / Tables(14)

    Get Citation
    PDF
    XML
    Entire Issue PDF

    用微信扫码二维码

    分享至好友和朋友圈

    Article Metrics

    Article views (768) PDF downloads(49) Cited by()
    Proportional views
    Related
    Browse MIR
    Published Online Current Issue Special Issue Archive Selected Papers
    About MIR
    Aims and Scopes Index information Editorial Board Contact Us Springer Page

    For Authors
    Guide for Authors Submission Template Copyright Agreement
    For Referees
    Guide for Referees Referees Acknowledgement

    MIR Event
    MIR News MIR Awards
    WeChat: 机器智能研究MIR
    Twitter: MIR_Journal

    © Institute of Automation, Chinese Academy of Sciences. Published by Springer Nature and Science Press. All rights reserved. 京ICP备14019135号-25

    Supported by Beijing Renhe Information Technology Co. Ltd  support: info@rhhz.net  

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return