SK C&C Developed Big-Data-Based Platform for Integrated Security Log Analysis

 

SK C&C (www.skcc.com) announced on April 8 that the company, in collaboration with its security arm Infosec, had developed a big-data-based platform for integrated security log analysis.

SK C&C completed overall security performance testing, including hacking detection, searching and tracking, on data traffic generated monthly from over 1,800 websites under Infosec’s security control. The amount of data generated from the Infosec-controlled 1,800 websites is about staggering 3TB, or 15 billion visits per month.

The newly developed platform is Korea’s first security log analysis system built based on open-sourced software, such as Twitter Storm^* and Hadoop.^** The platform features reduced hack detection time through real-time data processing, improved hacking detection accuracy, and reduced hacker search and tracking time.

The real-time data processing technology analyzes log data in real-time – via streaming – without having to store data separately, allowing real-time hack detection. Conventional platforms available for use usually sort, store, and analyze log data separately, requiring seconds or even minutes to detect hacking.

While the vast majority of security firms manage to analyze logs of security systems – IDS, firewall, etc. – separately due to quantitatively increased traffic data, SK C&C’s big-data-based technologies can now perform integrated correlation analysis between various systems. This allows detection of abnormal hacking signs that remain undetectable from conventional separated analysis.  

Moreover, using the big-data technologies SK C&C was able to reduce hack searching and tracking time to within 4 seconds. 

The new open-sourced platform will bring even more benefits when commercialized, such as reduced software license costs, break away from particular vendor dependency, and strengthened IT competitiveness based on software openness. 

Senior Vice President Lee Byung-song, head of CV Innovations Group of SK C&C, said “SK C&C’s integrated security log analysis platform is a big-data-based system, and we will continue to produce tailored services and solutions in various industries, including security using, the big-data technologies that we have accumulated over the years.”

^*Twitter storm is Twitter’s open source processing system intended to analyze big-data in real-time. While, Hadoop is focused on batch processing, Twitter Storm is a system specialized on real-time analysis.

**Hadoop is a free Java Software framework that supports data intensive distributed applications running on large clusters of computers. Hadoop is a system typically known for its parallel processing of large data in a distributed environment.