A Simple Apache Nifi 3 Node Docker Cluster | Apache Nifi | Part 13

What do you need to consider when running a Nifi cluster? Let’s find out with a simple Apache Nifi 3 node docker cluster.

Support the channel by Subscribing!

PATREON – Would you like to support the channel in other way? Then check it out.

https://www.patreon.com/stevenkoon

TWITTER https://twitter.com/koon_steven

Docker File Used

version: "3" services: zookeeper: hostname: zookeeper container_name: zookeeper image: 'bitnami/zookeeper:latest' environment: - ALLOW_ANONYMOUS_LOGIN=yes networks: - nifinet nifi01: image: apache/nifi:latest ports: - 6980:8080 networks: - nifinet environment: - NIFI_WEB_HTTP_PORT=8080 - NIFI_CLUSTER_IS_NODE=true - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082 - NIFI_ZK_CONNECT_STRING=zookeeper:2181 - NIFI_ELECTION_MAX_WAIT=1 min nifi02: image: apache/nifi:latest ports: - 6979:8080 networks: - nifinet environment: - NIFI_WEB_HTTP_PORT=8080 - NIFI_CLUSTER_IS_NODE=true - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082 - NIFI_ZK_CONNECT_STRING=zookeeper:2181 - NIFI_ELECTION_MAX_WAIT=1 min nifi03: image: apache/nifi:latest ports: - 6978:8080 networks: - nifinet environment: - NIFI_WEB_HTTP_PORT=8080 - NIFI_CLUSTER_IS_NODE=true - NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082 - NIFI_ZK_CONNECT_STRING=zookeeper:2181 - NIFI_ELECTION_MAX_WAIT=1 min networks: nifinet: driver: bridge

Apache NiFi supports powerful and scalable directed graphs of data routing, transformation, and system mediation logic. Some of the high-level capabilities and objectives of Apache NiFi include:

Web-based user interface
    Seamless experience between design, control, feedback, and monitoring
Highly configurable
    Loss tolerant vs guaranteed delivery
    Low latency vs high throughput
    Dynamic prioritization
    Flow can be modified at runtime
    Back pressure
Data Provenance
    Track dataflow from beginning to end
Designed for extension
    Build your own processors and more
    Enables rapid development and effective testing
Secure
    SSL, SSH, HTTPS, encrypted content, etc...
    Multi-tenant authorization and internal authorization/policy management

What is Apache NiFi?

Put simply NiFi was built to automate the flow of data between systems. While the term 'dataflow' is used in a variety of contexts, we use it here to mean the automated and managed flow of information between systems. This problem space has been around ever since enterprises had more than one system, where some of the systems created data and some of the systems consumed data.

Chapters: 0:00 Intro and ... https://www.youtube.com/watch?v=eQ4UlerXJ24