Customer Flexibility

Within Engineering@DataNexus, we work hard to build a data platform that works for all of our customers, regardless of size or complexity. One of our first internal engineering requirements is ubiquity, in that w hether our customers run on AWS, Azure, OpenStack, or bare metal, our platform looks, feels, and behaves the same . To that end, we are huge fans of process automation, whether it's creating VPCs and networks, spinning up or down virtual machines, or configuring our data platform layer. We heavily rely on Ansible to stand up and configure our application and data layer, consisting of elements such as Kafka, Cassandra, Postgres, Elasticsearch, or even operational tasks such as wiring data sources to data destinations. As most folks would expect, we organize each layer into one more or Git repositories. Our Kafka code is responsible for installing and configuring Apache Kafka, as well as both Confluent enterprise and community. That encompasses Apache projects such as Zook

Amazing updates to DataNexus Platfrom v2

Amazing amount of progress on v2.0 of the DataNexus platform. In the past few months we have added: 1. Full idempotent automation of a new or expanding Kafka or Confluent secure cluster deployment across multiple clouds 2. Ability to choose and deploy management including Cruise Control or Control Center 3. Ability to detect drift across configurations of large data clusters with alerting and reporting 4. Ability to build a simple rules file and mask/filter any data in stream without complex coding 5. Separation of Application logic from underlying cloud infrastructure 6. Addition of more connectors for sources and destinations Coming this month is a completely simplified approach to idempotent infrastructure deployments: 1. from new account or existing 2. cross cloud and region support for a single application 3. automated tagging in clouds 4. deployment to bare metal with all cloud features 5. additional security controls at infrastructure layer 6. simplified files for configura

DataTrust Technology

March 27, 2018 by Christopher Keller As promised, here is the technology portion of our identity blockchain. The complete white paper can be found  here . We’re grateful for the feedback we’ve received so far and we’re looking forward to the next phases. Find  me  on twitter at  @cnkeller  and email at  and  Eddie  at  @eddie_satterly  and . The DataTrust technology stack used to implement the Trusted Identity Platform (TIP) is based both upon popular open source packages, as well as the commercial DataNexus platform. Figure 1 shows how multiple entities interact using the distributed ledger platform. Figure 1: DataTrust Platform Overview DataNexus  Platform The underlying platform is a Zero Touch Provisioning (ZTP) enterprise data capture and routing solution which serves as the base technology layer. While extensible to any technology, the standard open source implementation is comprised of: Ansible  and  Python  to handle m

Identity Platform Detailed Use Cases

March 21, 2018 by Christopher Keller Three weeks ago , we announced to the public our blockchain based identity platform. We appreciate the community response and have been hard at work on the formal white paper. Today, we would like to release the first half of the white paper: the detailed use cases. Stayed tuned for the second half, the technical implementation. Introduction In today's business landscape, the platform owns and monetizes the data which constructs the individual identity, making sharing or revocation of that data (right to be forgotten) disadvantageous. The European Union (EU) views data privacy as a fundamental right and has crafted the General Data Protection Regulation (GDPR) to enforce adherence. The Trusted Identity Platform allows for the creation, management, and sharing of identities using a distributed ledger overlaid on the DataNexus routing platform. This gives individuals the ability to to control and monetize who gets their identity da

Data Trust Platform and GDPR Compliance

March 20, 2018 by Christopher Keller We never set out to work at the intersection of compliance and technology; it just sort of happened that way. From developing a world first  Hadoop  solution that was both favorably HIPAA and PCI accredited, to  APRA  approval for public cloud data migration, to our to our latest efforts with our blockchain-based identity trust, we’ve been working in this space a while. So when I read the article on a nightmare data access request (you can find that article  here ), my thoughts naturally went to “hey, we can help with that”. The article points out nine high level requests that would cause compliance issues for companies without established business processes and backed by technology designed to address requests such as these.  We’ll briefly go through each high level item and see whether or not our platform can impact its fulfillment: When using an identity established in our  data trust  platform, it’s relatively straight forward to res

Ephemeral Data Trust

February 28, 2018 by Christopher Keller   Data protection, and more importantly, the right to be forgotten is all the rage. Legislation such as the  General Data Protection Regulation  (GDPR) is imposing significant penalties to organizations that do not implement robust controls around data privacy. At DataNexus, we think about data and its implications every day. Ingestion, transfer, storage, extraction, masking, and removal are all elements of our platform and integral to the problems we solve. We’ve recently began prototyping an approach to a data trust using the mechanics of the distributed ledger and built upon the DataNexus platform. Using the ledger, we openly record the events associated with encryption keys and movement of data collections between entities, such as a consumer and business. Each entity is represented by one or more unique wallet addresses. Within each wallet, we store the encryption keys to the collections owned by that entity and shared with other s

Use Case: Filters for Compliance

February 19, 2018 by Eddie Satterly