Posts

Showing posts from 2019

Customer Flexibility

Within Engineering@DataNexus, we work hard to build a data platform that works for all of our customers, regardless of size or complexity. One of our first internal engineering requirements is ubiquity, in that w hether our customers run on AWS, Azure, OpenStack, or bare metal, our platform looks, feels, and behaves the same . To that end, we are huge fans of process automation, whether it's creating VPCs and networks, spinning up or down virtual machines, or configuring our data platform layer. We heavily rely on Ansible to stand up and configure our application and data layer, consisting of elements such as Kafka, Cassandra, Postgres, Elasticsearch, or even operational tasks such as wiring data sources to data destinations. As most folks would expect, we organize each layer into one more or Git repositories. Our Kafka code is responsible for installing and configuring Apache Kafka, as well as both Confluent enterprise and community. That encompasses Apache projects such as Zook

Amazing updates to DataNexus Platfrom v2

Image
Amazing amount of progress on v2.0 of the DataNexus platform. In the past few months we have added: 1. Full idempotent automation of a new or expanding Kafka or Confluent secure cluster deployment across multiple clouds 2. Ability to choose and deploy management including Cruise Control or Control Center 3. Ability to detect drift across configurations of large data clusters with alerting and reporting 4. Ability to build a simple rules file and mask/filter any data in stream without complex coding 5. Separation of Application logic from underlying cloud infrastructure 6. Addition of more connectors for sources and destinations Coming this month is a completely simplified approach to idempotent infrastructure deployments: 1. from new account or existing 2. cross cloud and region support for a single application 3. automated tagging in clouds 4. deployment to bare metal with all cloud features 5. additional security controls at infrastructure layer 6. simplified files for configura