This is the story of how we built a highly available data pipeline that processes terabytes of network data every day, making it available to security researchers for security assessment and threat hunting. Building this kind of stuff in the cloud is not that complicated, but if you have to make it near real-time, fault tolerant and 24/7 available, well... that's another story. In this talk, Luciano and Domagoj will tell you how they achieved this ambitious goal and how they missed a few good nights of sleep while trying to do that! Spoiler alert: contains AWS, lambda, elastic search, monitoring, alerting & more!

Luciano Mammino
Luciano is a software engineer born in 1987, the same year that “Super Mario Bros” was released in Europe, which, by chance is his favourite game! Luciano started coding early at the age of 12, hacking away with his father's old i386 armed only with MS-DOS and the QBasic interpreter and he has been professionally a software developer for more than 10 years. Luciano is currently a Solution Architect at Vectra AI in Dublin where he's working on automating the hunt for cyberattackers and speeding-up incident response. Luciano loves the fullstack web, Node.js and Serveless so he co-authored the book "Node.js design patterns", launched fstack.link (a semi automated newsletter for Fullstack developers) and Serverlesslab.com (in-house serverless training).

Domagoj Katavic
Domagoj Katavic (Principal Software engineer at Vectra) is a technology enthusiast coming from Split, Croatia. From his early days, Dom has enjoyed playing with technology, starting with programming robots in PBASIC, to gesture recognition and tinkering with FPGA. Dom has always liked programming so his first job was building a cloud editor Codeanywhere and teaching programming at a local university. Now he is a Software Engineer at Vectra AI where he is making the hunt for cyber attacks as easy and automated as possible.