Customer Profile

Our Client is a leader in cybersecurity vulnerability management with more than 24,000 customers worldwide.

Problem Statement

Our Client receives more than 30 TB of data every month. The Client’s customers need to access this data for their information security management. Access time is of essence to assess and manage security threats.

Due to the size of the data, it took a customer 1 to 3 hours to access the data it required. This was unacceptable to many customers. They required a response time of less than a second.

The Solution

a. Delixus found that each customer accessed only his own data.. So, the first step was to partition the data by customer.

b. The customers were interested in the latest data. Very rarely did they accessed the old data. So, the second step was to partition the data according to its age.  Only the data for seven days was kept active; the old data was archived.

c. The response time was further reduced by clustering the database.

d. The database configuration was optimized to use the servers efficiently.

e. As a final step, Delixus optimized the query to reduce the response time to less than a second.

Benefits

Our Client was able to retain a large number of customers who were satisfied with the noteworthy improvement in access time from hours to less than 1 second.

Factors of success

Mastery of a host of software packages and database management were required for the successful completion of the project. These included

Databased: PostgreSQL, Citus, MySQL, ElasticSearch, MongoDB, Dynamo DB, Cassandra, SQLite, Hive

Frameworks and tools: Elastic Map Reduce, Citus

Logs and log aggregation: slf4j, Slunk and WatchDog

Deployment: Kubernetes, Docker, Nginx, Jenkins, CI/CD

Maintenance: Performance and metric Dashboards, Correlation Analysis, Incident Management.