How to create cache for high volume online data
"How to create cache for high volume online data using Elasticsearch and not fail"
4 years ago we decided to create a central cache system for our banking data. The main goals for our system were:
- quick writing and sharing,
- fault tolerance,
- scalability,
- data integrity,
- synchronization across multiple clusters,
- migration without data quality loss.
The system was based on the elastic search engine, SQL database, the Redis server and java. In my presentation I will focus on the technical details of our solution:
- architecture (java, oracle, elastic),
- transmission protocols,
- performance.