FIELD: system for aggregating and indexing data for display to the user.
SUBSTANCE: system contains a plurality of external computing devices, each of which contains a data storage and is configured to transmit data aggregation to the server; an aggregation server configured to receive and aggregate data, comprising a processor and a memory storing machine-readable instructions causing the processor to: determine the type of data stored in said storage of external computing devices; establishing a connection with the said storages with reading data from said storages into the memory of the aggregation server, performed depending on the type of data; saving the read data in the memory of the aggregation server in the form of a temporary file; creating an aggregation database in the form of a cluster for storing read data in the memory of the aggregation server; and storing the read data in the created aggregation database from the temporary file, followed by deleting the temporary file and closing the connection to the storage of the external computing device; a data indexing store, configured to index and store indexed data, where the data indexing store is based on the Solr platform for full-text search with open source code (hereinafter referred to as the Solr storage); in this case, the Solr storage is made with the ability to form and use multiple cores, where each core represents an index and an associated transaction log and configuration files, for indexing data with different structures on one server, and provides: creation of a core configuration file in the storage file system Solr and placing connection parameters to the previously created aggregation database in this file; calling the application program interface (API) of the Solr storage via the http protocol with a command to start kernel indexing; performing a query by the Solr storage on said aggregation database to obtain data stored therein; performing indexing of the received data in the Solr storage and forming a kernel based on the completed indexing; and placing the generated kernel in a part of the Solr storage accessible for data reading by the user's computing device; a user's computing device configured to receive indexed data from the aggregation server, where the provision of the indexed data is performed in accordance with the steps of: receiving by the aggregation server of data reading settings or an identifier of saved settings, followed by reading these settings from the aggregation database; converting said settings into a format accepted by the Solr storage API; sending a request via the http protocol to the Solr storage to the kernel corresponding to the current settings with the converted settings attached in JSON format; the aggregation server receives indexed data from the Solr storage; and transmitting to the user's computing device the indexed data for subsequent output to the user.
EFFECT: increased efficiency of data aggregation and indexing.
2 cl, 1 dwg
Title | Year | Author | Number |
---|---|---|---|
SYSTEM AND METHOD OF PROCESSING GRAPH DATA | 2015 |
|
RU2708939C2 |
METHOD AND SYSTEM FOR DEVICE MANAGEMENT AND DEVICE CONTROL | 2017 |
|
RU2648564C1 |
SYSTEM AND METHOD OF INTERCEPTING FILE STREAMS | 2023 |
|
RU2816551C1 |
METHOD OF CONSTRUCTING A DISTRIBUTED INFORMATION SYSTEM | 2018 |
|
RU2699683C1 |
METHOD AND SYSTEM FOR MOVING DATA IN CLOUD ENVIRONMENT | 2023 |
|
RU2822554C1 |
METHOD FOR EXCLUDING PROCESSES OF ANTIVIRUS SCANNING ON THE BASIS OF DATA ON FILE | 2015 |
|
RU2595510C1 |
METHOD FOR INTEGRATION OF PERIPHERAL DEVICES OF SOFTWARE AND HARDWARE SYSTEMS | 2019 |
|
RU2732729C1 |
INFORMATION RETRIEVAL METHOD AND CORPORATE INFORMATION RETRIEVAL SYSTEM | 2019 |
|
RU2729224C2 |
METHOD OF PROCESSING DATA IN HYBRID STORAGE | 2023 |
|
RU2831216C1 |
SYSTEM AND METHOD OF SEARCHING DATA IN DATABASE OF GRAPHS | 2015 |
|
RU2707708C2 |
Authors
Dates
2024-01-11—Published
2022-04-07—Filed