Chang-Yang Wu, Xin Lin, Zhen-Ya Huang, Yu Yin, Jia-Yu Liu, Qi Liu, Gang Zhou. Clause-level Relationship-aware Math Word Problems Solver. Machine Intelligence Research. https://doi.org/10.1007/s11633-022-1351-2
Citation: Chang-Yang Wu, Xin Lin, Zhen-Ya Huang, Yu Yin, Jia-Yu Liu, Qi Liu, Gang Zhou. Clause-level Relationship-aware Math Word Problems Solver. Machine Intelligence Research. https://doi.org/10.1007/s11633-022-1351-2

Clause-level Relationship-aware Math Word Problems Solver

doi: 10.1007/s11633-022-1351-2
More Information
  • Author Bio:

    Chang-Yang Wu received the B. A. degree in sport English from Shanghai University of Sport, China in 2014. Now, he is a master student in computer science at School of Data Science, University of Science and Technology of China (USTC), China. His research interests include data mining and intelligent education systems.E-mail: justwu@mail.ustc.edu.cnORCID iD: 0000-0002-7735-1568

    Xin Lin received the B. Eng. degree in computer science from University of Science and Technology of China, China in 2019. He is currently a Ph. D. degree candidate in computer science at School of Computer Science and Technology at USTC, China. He has published papers in referred conference proceedings, such as AAAI 2021. His research interests include data mining, math word problems, intelligent education systems. E-mail: linx@mail.ustc.edu.cn

    Zhen-Ya Huang received the B. Eng. degree in software engineering from Shandong University, China in 2014 and the Ph. D. degree in applied computer technology from University of Science and Technology of China, China in 2020. He is currently an associate researcher of School of Computer Science and Technology, USTC, China. He has published more than 30 papers in refereed journals and conference proceedings including TKDE, TOIS, KDD, AAAI. He has served regularly in the program committees of a number of conferences, and is reviewer for the leading academic journals. His research interests include data mining, knowledge discovery, representation learning and intelligent tutoring systems. E-mail: huangzhy@ustc.edu.cn

    Yu Yin received the B. Sc. degree in computer science and technology from School of Computer Science and Technology, USTC, China in 2017. He is currently a Ph. D. degree candidate in computer science at School of Computer Science and Technology at USTC, China. He won the first prize in the Second Student Remote Direct Memory Access Programming Competition, in 2014. He has published papers in journals and conference proceedings related with data mining and machine learning, such as AAAI, KDD, ICDM, CIKM, SIGIR and ACM TKDE. His research interests include data mining, intelligent education systems and reinforcement learning. E-mail: yxonic@mail.ustc.edu.cn

    Jia-Yu Liu received the B. Sc. degree in applied mathematics from USTC, China in 2020. Now, he is a master student in data science (computer science and technology) at School of Data Science, University of Science and Technology of China. His research interests include data mining and intelligent education systems.E-mail: jy251198@mail.ustc.edu.cn

    Qi Liu received the Ph. D. degree in computer science from USTC, China in 2013. He is currently a professor at USTC, China. His general area of research is data mining and knowledge discovery. He has published prolifically in refereed journals and conference proceedings, e.g., the IEEE Transactions on Knowledge and Data Engineering, the ACM Transactions on Information Systems, the ACM Transactions on Knowledge Discovery from Data, the ACM Transactions on Intelligent Systems and Technology, KDD, IJCAI, AAAI, ICDM, SDM, and CIKM. He has served regularly in the program committees of a number of conferences, and is a reviewer for the leading academic journals in his fields. He is a member of the ACM, the IEEE, and the Alibaba DAMO Academy Young Fellow. His research is also supported by the National Science Fund for Excellent Young Scholars and the Youth Innovation Promotion Association of Chinese Academy of Sciences (CAS). His research interests include data science, data mining, machine learning: methods and applications, recommender systems, social network analysis. E-mail: qiliuql@ustc.edu.cn (Corresponding author)ORCID iD: 0000-0001-6956-5550

    Gang Zhou received the B. Eng. and M. A.Eng degrees in computer software from Information Engineering University, China in 1996 and 1999, the Ph. D. degree in computer software and theory from Beihang University, China in 2007. He was with the State key of Laboratory of Mathematical Engineering And Advanced Computing, Information Engineering University, as a research fellow. His research interests are big data, knowledge graph, and data mining.E-mail: gzhougzhou@126.comORCID iD: 0000-0001-8609-3954

  • Received Date: 2022-04-11
  • Accepted Date: 2022-06-17
  • Publish Online: 2022-08-25
  • Automatically solving math word problems, which involves comprehension, cognition, and reasoning, is a crucial issue in artificial intelligence research. Existing math word problem solvers mainly work on word-level relationship extraction and the generation of expression solutions while lacking consideration of the clause-level relationship. To this end, inspired by the theory of two levels of process in comprehension, we propose a novel clause-level relationship-aware math solver (CLRSolver) to mimic the process of human comprehension from lower level to higher level. Specifically, in the lower-level processes, we split problems into clauses according to their natural division and learn their semantics. In the higher-level processes, following human′s multi-view understanding of clause-level relationships, we first apply a CNN-based module to learn the dependency relationships between clauses from word relevance in a local view. Then, we propose two novel relationship-aware mechanisms to learn dependency relationships from the clause semantics in a global view. Next, we enhance the representation of clauses based on the learned clause-level dependency relationships. In expression generation, we develop a tree-based decoder to generate the mathematical expression. We conduct extensive experiments on two datasets, where the results demonstrate the superiority of our framework.

     

  • loading
  • [1]
    T. Brants. Natural language processing in information retrieval. In Proceedings of Computational Linguistics in the Netherlands, University of Antwerp, Antwerp, Belgium, pp. 1–13, 2003.
    [2]
    Y. K. Xian, Z. H. Fu, S. Muthukrishnan, G. De Melo, Y. F. Zhang. Reinforcement knowledge graph reasoning for explainable recommendation. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, ACM, Paris, France, pp. 285–294, 2019. DOI: 10.1145/3331184.3331203.
    [3]
    D. X. Zhang, L. Wang, L. M. Zhang, B. T. Dai, H. T. Shen. The gap of semantic parsing: A survey on automatic math word problem solvers. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 9, pp. 2287–2305, 2019. DOI: 10.1109/TPAMI.2019.2914054.
    [4]
    D. P. Huang, S. M. Shi, C. Y. Lin, J. Yin. Learning fine-grained expressions to solve math word problems. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics, Copenhagen, Denmark, pp. 805–814, 2017. DOI: 10.18653/v1/D17-1084.
    [5]
    Z. P. Xie, S. C. Sun. A goal-driven tree-structured neural model for math word problems. In Proceedings of the 28th International Joint Conference on Artificial Intelligence Main track, Macao, China, pp. 5299–5305, 2019. DOI: 10.24963/ijcai.2019/736.
    [6]
    Y. N. Hong, Q. Li, D. Ciao, S. Y. Huang, S. C. Zhu. Learning by fixing: Solving math word problems with weak supervision. In Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 6, pp. 4959–4967, 2021.
    [7]
    S. Roy, D. Roth. Unit dependency graph and its application to arithmetic word problem solving. In Proceedings of the 31st AAAI Conference on Artificial Intelligence, ACM, San Francisco, USA, pp. 3082–3088, 2017.
    [8]
    J. R. Li, L. Wang, J. P. Zhang, Y. Wang, B. T. Dai, D. X. Zhang. Modeling intra-relation in math word problems with different functional multi-head attentions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, pp. 6162–6167, 2019. DOI: 10.18653/v1/P19-1619.
    [9]
    J. P. Zhang, L. Wang, R. K. W. Lee, Y. Bin, Y. Wang, J. Shao, E. P. Lim. Graph-to-tree learning for solving math word problems. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL, pp. 3928–3937, 2020. DOI: 10.18653/v1/2020.acl-main.362.
    [10]
    D. A. Balota, G. B. F. d′Arcais, K. Rayner. Comprehension Processes in Reading. New York, USA: Routledge, 1990. DOI: 10.4324/9780203052389.
    [11]
    T. A. Van Dijk, W. Kintsch. Strategies of Discourse Comprehension. New York, USA: Academic Press, 1983.
    [12]
    M. Adoniou, Q. Yi. Language, mathematics and English language learners. The Australian Mathematics Teacher, vol. 70, no. 3, pp. 3–13, 2014.
    [13]
    X. Lin, Z. Y. Huang, H. K. Zhao, E. H. Chen, Q. Liu, H. Wang, S. Wang. HMS: A hierarchical solver with dependency-enhanced understanding for math word problem. In Proceedings of AAAI Conference on Artificial Intelligence, vol. 35, no. 5, pp. 4232–4240, 2021.
    [14]
    E. A. Feigenbaum, J. Feldman. Computers and Thought. New York, USA: McGraw-Hill, 1963.
    [15]
    D. G. Bobrow. Natural Language Input for a Computer Problem Solving System, Series/Report no. AITR-219, Department of Mathematics, Massachusetts Institute of Technology, Cambridge, USA, 1964.
    [16]
    J. R. Slagle. Experiments with a deductive question-answering program. Communications of the ACM, vol. 8, no. 12, pp. 792–798, 1965. DOI: 10.1145/365691.365960.
    [17]
    C. R. Fletcher. Understanding and solving arithmetic word problems: A computer simulation. Behavior Research Methods,Instruments,&Computers, vol. 17, no. 5, pp. 565–571, 1985. DOI: 10.3758/bf03207654.
    [18]
    Y. Bakman. Robust understanding of word problems with extraneous information. [Online], Available: https://arxiv.org/pdf/math/0701393.pdf, 2007.
    [19]
    N. Kushman, Y. Artzi, L. Zettlemoyer, R. Barzilay. Learning to automatically solve algebra word problems. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, pp. 271–281, 2014. DOI: 10.3115/v1/P14-1026.
    [20]
    A. Mitra, C. Baral. Learning to use formulas to solve simple arithmetic problems. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, pp. 2144–2153, 2016. DOI: 10.18653/v1/P16-1202.
    [21]
    R. Koncel-Kedziorski, H. Hajishirzi, A. Sabharwal, O. Etzioni, S. D. Ang. Parsing algebraic word problems into equations. Transactions of the Association for Computational Linguistics, vol. 3, pp. 585–597, 2015. DOI: 10.1162/tacl_a_00160.
    [22]
    S. M. Shi, Y. H. Wang, C. Y. Lin, X. J. Liu, Y. Rui. Automatically solving number word problems by semantic parsing and reasoning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, pp. 1132–1142, 2015. DOI: 10.18653/v1/D15-1135.
    [23]
    D. Q. Huang, S. M. Shi, C. Y. Lin, J. Yin, W. Y. Ma. How well do computers solve math word problems? Large-scale dataset construction and evaluation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany, pp. 887–896, 2016. DOI: 10.18653/v1/P16-1084.
    [24]
    Y. Wang, X. J. Liu, S. M. Shi. Deep neural solver for math word problems. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark, pp. 845–854, 2017. DOI: 10.18653/v1/D17-1088.
    [25]
    L. Wang, D. X. Zhang, L. L. Gao, J. K. Song, L. Guo, H. T. Shen. MathDQN: Solving arithmetic word problems via deep reinforcement learning. In Proceedings of AAAI Conference on Artificial Intelligence, vol. 32, no. 1, pp. 5545–5552, 2018. DOI: 10.1609/aaai.v32i1.11981.
    [26]
    L. Wang, Y. Wang, D. Cai, D. X. Zhang, X. J. Liu. Translating a math word problem to an expression tree. [Online], Available: https://arxiv.org/pdf/1811.05632.pdf, 2018.
    [27]
    T. R. Chiang, Y. N. Chen. Semantically-aligned equation generation for solving and reasoning math word problems. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Minneapolis, Minnesota, pp. 2656–2668, 2018. DOI: 10.18653/v1/N19-1272.
    [28]
    L. Wang, D. X. Zhang, J. P. Zhang, X. Xu, L. L. Gao, B. T. Dai, H. T. Shen. Template-based math word problem solvers with recursive neural networks. In Proceedings of AAAI Conference on Artificial Intelligence, vol. 33, no. 1, pp. 7144–7151, 2019. DOI: 10.1609/aaai.v33i01.33017144.
    [29]
    Q. Z. Wu, Q. Zhang, J. L. Fu, X. J. Huang. A knowledge-aware sequence-to-tree network for math word problem solving. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 7137–7146, 2020. DOI: 10.18653/v1/2020.emnlp-main.579.
    [30]
    Q. Z. Wu, Q. Zhang, Z. Y. Wei. An edge-enhanced hierarchical graph-to-tree network for math word problem solving. In Proceedings of the Findings of the Association for Computational Linguistics, Punta Cana, Dominican Republic, pp. 1473–1482, 2021. DOI: 10.18653/v1/2021.findings-emnlp.127.
    [31]
    M. Yuhui, Z. Ying, C. Guangzuo, R. Yun, H. Ronghuai. Frame-based calculus of solving arithmetic multi-step addition and subtraction word problems. In Proceedings of the Second International Workshop on Education Technology and Computer Science, IEEE, Wuhan, China, pp. 476–479, 2010. DOI: 10.1109/ETCS.2010.316.
    [32]
    Y. X. Cao, F. Hong, H. W. Li, P. Luo. A bottom-up DAG structure extraction model for math word problems. Proceedings of AAAI Conference on Artificial Intelligence, vol. 35, no. 1, pp. 39–46, 2021.
    [33]
    Y. Zhang, G. Y. Zhou, Z. W. Xie, J. X. Huang. HGEN: Learning hierarchical heterogeneous graph encoding for math word problem solving. IEEE/ACM Transactions on Audio,Speech,and Language Processing, vol. 30, pp. 816–828, 2022. DOI: 10.1109/TASLP.2022.3145314.
    [34]
    Z. W. Liang, J. P. Zhang, L. Wang, W. Qin, Y. S. Lan, J. Shao, X. L. Zhang. MWP-BERT: Numeracy-augmented pre-training for math word problem solving. Available: https://aclanthology.org/2022.findings-naacl.74.pdf, 2022.
    [35]
    J. H. Shen, Y. C. Yin, L. Li, L. F. Shang, X. Jiang, M. Zhang, Q. Liu. Generate & Rank: A multi-task framework for math word problems. In Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic, pp. 2269–2279, 2021. DOI: 10.18653/v1/2021.findings-emnlp.195.
    [36]
    W. J. Yu, Y. P. Wen, F. D. Zheng, N. Xiao. Improving math word problems with pre-trained knowledge and hierarchical reasoning. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, Punta Cana, Dominican Republic, pp. 3384–3394, 2021. DOI: 10.18653/v1/2021.emnlp-main.272.
    [37]
    M. A. K. Halliday, C. M. I. Matthiessen. An Introduction to Functional Grammar. London, UK: Routledge, 2014.
    [38]
    J. Ng, K. Lee, K. H. Khng. Irrelevant information in math problems need not be inhibited: Students might just need to spot them. Learning and Individual Differences, vol. 60, pp. 46–55, 2017. DOI: 10.1016/j.lindif.2017.09.008.
    [39]
    K. Barker, S. Szpakowicz. Interactive semantic analysis of clause-level relationships. In Proceedings of the 2nd Conference of the Pacific Association for Computational Linguistics, Brisbane, Australia, pp. 22–30, 1995.
    [40]
    T. Ohno, S. Matsubara, H. Kashioka, T. Maruyama, H. Tanaka, Y. Inagaki. Dependency parsing of Japanese monologue using clause boundaries. Language Resources and Evaluation, vol. 40, no. 3, pp. 263–279, 2006. DOI: 10.1007/s10579-007-9023-y.
    [41]
    D. S. McNamara, J. Magliano. Toward a comprehensive model of comprehension. Psychology of Learning and Motivation, vol. 51, pp. 297–384, 2009. DOI: 10.1016/S0079-7421(09)51009-2.
    [42]
    T. Mikolov, I. Sutskever, K. Chen, G. Corrado, J. Dean. Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th International Conference on Neural Information Processing Systems, ACM, Lake Tahoe, Nevada, pp. 3111–3119, 2013.
    [43]
    J. Devlin, M. W. Chang, K. Lee, K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Association for Computational Linguistics, Minneapolis, Minnesota, pp. 4171–4186, 2018. DOI: 10.18653/v1/N19-1423.
    [44]
    L. Pang, Y. Y. Lan, J. F. Guo, J. Xu, S. X. Wan, X. Q. Cheng. Text matching as image recognition. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, ACM, Phoenix, USA, pp. 2793–2799, 2016.
    [45]
    O. Vinyals, M. Fortunato, N. Jaitly. Pointer networks. In Proceedings of the 28th International Conference on Neural Information Processing Systems, ACM, Montreal, Canada, pp. 2692–2700, 2015.
    [46]
    K. M. He, X. Y. Zhang, S. Q. Ren, J. Sun. Delving deep into rectifiers: Surpassing human-level performance on imageNet classification. In Proceedings of the IEEE International Conference on Computer Vision, IEEE, Santiago, Chile, pp. 1026–1034, 2015. DOI: 10.1109/ICCV.2015.123.
    [47]
    C. Z. Wu, J. Sun, J. Wang, L. F. Xu, S. Zhan. Encoding-decoding network with pyramid self-attention module for retinal vessel segmentation. International Journal of Automation and Computing, vol. 18, no. 6, pp. 973–980, 2021. DOI: 10.1007/s11633-020-1277-0.
    [48]
    L. J. Zhou, J. W. Dang, Z. H. Zhang. Fault classification for on-board equipment of high-speed railway based on attention capsule network. International Journal of Automation and Computing, vol. 18, no. 5, pp. 814–825, 2021. DOI: 10.1007/s11633-021-1291-2.
  • 加载中

Catalog

    通讯作者: 陈斌, bchen63@163.com
    • 1. 

      沈阳化工大学材料科学与工程学院 沈阳 110142

    1. 本站搜索
    2. 百度学术搜索
    3. 万方数据库搜索
    4. CNKI搜索

    Figures(9)  / Tables(2)

    用微信扫码二维码

    分享至好友和朋友圈

    Article Metrics

    Article views (35) PDF downloads(1) Cited by()
    Proportional views
    Related

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return