This is a recently open sourced project at Brex that I'm leading, you can read more about it in our announcement blog post: https://medium.com/brexeng/announcing-substation-188d049d979...
Here are some highlights:
- Serverless data pipeline toolkit and data transformation package written in Go
- Running in production for over a year, processes billions of events per day
- All in cost for all of our data pipelines is ~$30/TB (storage, compute, encryption)
For anyone who wants to get hands on, you can try it out using the quickstart guide: https://github.com/brexhq/substation#quickstart