A Study Of Legal Case Retrieval Based On Pre-trained Language Model

Legal similar case retrieval,also known as legal case retrieval,is one of the most critical technologies and core functions in realizing an intelligent judicial system.It can efficiently and accurately retrieve similar cases related to the user’s given query case from the massive legal case database.Therefore,legal case retrieval is crucial for the courts to realise the “similar cases,similar judgments” and is also an important cornerstone to promote the consistency and fairness of legal practice.However,there are several challenges facing current legal case retrieval research: 1)Some existing methods,such as keyword matching,fail to provide an in-depth understanding and interpretation of legal cases at the semantic level of the text.In addition,since legal documents are specialized and structured,current retrieval models rarely introduce these key features based on legal documents into the models for modelling.Therefore,it is not easy to learn relevant legal knowledge.On the other hand,most of the legal case retrieval models can also not handle long legal documents effectively.2)Most legal case retrieval approaches are based on pre-trained language models(e.g.,BERT)fine-tuning the interactive architecture.However,despite their excellent accuracy in practical retrieval,the inference speed is prolonged,so it is not easy to achieve real-time inference.Representation-based legal case retrieval models usually contain two independent sub-networks to extract feature information of the query and the candidate documents separately,which allows offline caching of candidate documents as feature vectors for the pending searched,and thus it is fast inference.However,its accuracy is low because there is no interaction between the query and the candidate documents.Therefore,in order to address the above challenges,this paper focuses on the application of legal case retrieval in an intelligent judicial system and carries out several fundamental technical studies and experimental analyses in terms of the analysis of the specificity and structure of legal decision documents,the design of retrieval models and multi-task optimization strategies,and the design of a late bidirectional interaction network for legal retrieval scenarios.The main research work of this paper is summarized as follows:Firstly,to address the problems that current keyword matching methods cannot understand legal cases at the semantic level,the retrieval model does not introduce key features of legal documents,and cannot handle long documents of laws,this paper proposes a two-stage ranking method for retrieving legal cases based on BERT.Specifically,our method first uses the wellknown BM25 ranking function to quickly retrieve the top N candidate cases from the candidate case pool to reduce the pool size in subsequent ranking stages.The pre-trained language model BERT is then used to sort the filtered candidate cases accurately.In particular,we fine-tune the BERT model by combining Pointwise and assisted learning to learn models that understand the deep semantics of legal decision texts.Then,we combine the Pairwise approach to build on the fine-tuned BERT model to train further a model suitable for accurately ranking candidate cases.Finally,we conduct extensive experiments to confirm that the proposed method outperforms several benchmark methods in legal case retrieval.Moreover,we applied our method in the Challenge of AI in Law competition in 2021(CAIL 2021)and won a runner-up in the track of similar legal case retrieval.Secondly,to address the problem that the current interaction and representational-based retrieval models are challenging to apply to legal practical application scenarios directly,we propose a legal case retrieval model based on the late bidirectional interaction attention mechanism,which can fully utilise both advantages while overcoming their disadvantages.Specifically,we use a two-tower BERT model as the backbone network to obtain feature representations of queries and their candidate cases.Then,we developed a bidirectional interaction attention network to generate deep interaction attention signals between queries and candidate cases.Finally,we conducted extensive experiments to show that our model is faster than the interaction-based legal case retrieval model and has better accuracy than the representationbased legal case retrieval model.