Big Data and Machine Learning Applications for DevOps and Security
A provider of global satellite imaging and geospatial monitoring data partnered with Synaptiq to support one of their large government agency clients. The agency wanted to bring together hundreds of stovepipe analytics projects, data sources and tools into a common cloud-based framework. This would allow them to more effectively unlock the potential of machine learning and advanced analytics to exploit synergies across the organization.
Synaptiq leveraged its extensive expertise in infrastructure for massive data environments to develop core big data and machine learning capabilities, successfully addressing the agency’s key challenges. Synaptiq built a scalable big data pipeline that aggregates various cloud data sources from hundreds of Amazon resources into a data lake used for time series analysis. The system was built using Amazon Kinesis, Spark EMR, Amazon S3 and CloudFormation.
Synaptiq constructed a deep reinforcement learning system that automatically optimizes cloud resources based on user defined specifications of what's important. The machine figures out "how" by learning optimal control policies. Synaptiq also performed feature engineering, constructed a probabilistic supervised machine learning system that identifies false positive software vulnerabilities from software scanning results, and designed a simple user experience that displays a significantly reduced set of ranked vulnerabilities.
The solutions were demonstrated to the agency and follow on work has begun to prepare the solutions for production and address other agency challenges using machine learning.