BUILDING AND MANAGING INDEX OF DISTRIBUTED SEARCH ENGINE BASED ON *BFT CONSENSUS ALGORITHM
In this article, we consider an algorithm for building a search index of a distributed search engine that uses the consensus algorithm of the *BFT (Byzantine Fault Tolerance) family, as well as its reorganization caused by the addition or removal of nodes from the search cluster. The article describes the structure of the cluster, addition of data to the index, cluster reorganization algorithm, and also discusses possible related problems and their solutions. At the same time, the limitations imposed by the very nature of distributed computing and the need to comply with BFT requirements are considered. The value of the task is due to the increasing need to use distributed systems to solve different practical tasks, including the organization of search, as well as the need to adapt existing solutions to the conditions of distributed systems and to take into account the limitations imposed by them, as evidenced by the existence of active research in this area and the current in the market situation. The methods and approaches used in this work, with some modifications, can be transferred to similar tasks arising in distributed systems of other types. The results and materials of the article can be used to solve applied problems in the implementation of distributed computing systems, distributed search systems, distributed data storage systems.
Keywords: search, distributed systems, BFT consensus algorithm, load balancing.