The Building a Patent Landscape for Technological Forecasting Tasks

Abstract

Methods and technologies for solving the problem of patent landscape visualization based on cluster analysis of the patent array are considered and used. Algorithms for downloading patent archives, parsing patent documents, clustering patents and visualizing the patent landscape have been developed. A software for clustering patent documents based on the Latent Dirichlet allocation model and visualization of the patent landscape on clustering data using the gensim, PySpark, and sklearn libraries has been implemented. The implemented software has been tested on patents issued by the US Patent and Trademark Office. The accuracy of classification of patents by category has been achieved - 84%. Possible ways to improve the system: (a) add the ability to save the LDA model trained on certain categories of patents to predict the category of a new patent document (b) improve the quality of preprocessing of the patent description text; (c) develop a software on the customer's resources (computing cluster), in order to fully test the Apache Spark framework; (d) saving a large amount of data on the customer's resources (computing cluster) using HDFS technology (Hadoop Distributed File System).