Conceptual Approach for Development of Web Scraping Application for Tracking Information

Abstract

The paper focuses on the issues of development of web scraping applications. The main features of such an application are defined with some specific functionalities that should be implemented. The theoretical framework of the article is based on web scraping as a part of data mining and on data mining as a part of business intelligence. The paper reveals some possible approaches to web scraping in terms of the development of software solutions. The article makes an overview of the research on scraping applications and the functional characteristics of several traditional solutions for web scraping that have proved successful. The author presents a conceptual approach for the development of software solution for web scraping in eight steps. The proposed concept allows for the identification of its strengths, weaknesses, opportunities and threats, that is, the performance of SWOT analysis to provide for the comparison between the traditional solutions for web scraping and the proposed conceptual approach. In conclusion, the paper shows some functional advantages of the proposed conception over the traditional software solutions in the field.