meshIQ Kafka Console Implementation: readme.txt ================================================================================================================ Follow below steps to get started with the meshIQ Kafka Console setup. NOTE: Before running this procedure, please verify: ------------------------------------------------------------------------------------------------------------------------------------------ Prerequisites: Linux System: - Operating System: Linux 64 bit OS - CPU: Minimum 4 virtual CPUs - Memory: Minimum 32GB RAM - File System: Minimum 100 GB free space - Recommended: Dedicated Server Instance NETWORK: Web HTTP Server: PORT=32080(TCP) Node Exporter: PORT=9100(TCP) Kafka JMX Java Agent: PORT=9404(TCP) Kafka Broker node: PORT=9092 Zookeeper node: PORT=2181 zookeeper jmx: PORT=7000(TCP) ========================================================================================================================================== The machine on which the install will be done respects all prerequisites for installation described above. For more information on meshiq kafka console please visit this link: https://customers.meshiq.com/hc/en-us/categories/30093024041363 MKC SETUP STEPS: These steps assume local installation for all related components. KUBERNETES INSTALLATION: 1. Untar/unzip the tar -zxvf mkc_containers_packVx.x.tar.Z package to the any path of your choice ... This package contains the directory "meshiq" as a root directory. SHELL COMMAND: tar -zxvf mkc_containers_packV1.2.tar.Z 2. TO START THE HELMCHARTS DEPLOYMENT: NOTE: Deploy zookeeper first, before deploying Kafka. TO DEPLOY mKC (meshiq kafka console): NOTE: mKC deployment requires Kafka for its internal usages and uses it for data storage internally, helm install will call for two nodes to be available for this purpose one for its internal Kafka deployment and another for mKC components. Its recommended to have both of these services running on one node this can be done by labeling the node. SHELL COMMAND: kubectl label nodes "YourTargetNodeIP" mkc=node SHELL COMMAND: cd /your_deployment_path/meshiq/helm/mkc SHELL COMMAND: helm install meshiq-mkc -f values-sub-kafka.yaml ./ TO DEPLOY ZOOKEEPER: SHELL COMMAND: cd /your_deployment_path/meshiq/helm/zookeeper REVIEW CONFIGURATION: Review values.yaml and README.md files for cluster deployment settings. SHELL COMMAND: helm install zookeeper ./ TO DEPLOY KAFKA: SHELL COMMAND: cd /your_deployment_path/meshiq/helm/kafka REVIEW CONFIGURATION: Review values.yaml and README.md files for cluster deployment settings. SHELL COMMAND: helm install kafka ./ TO DEPLOY SCHEMA-REGISTRY: SHELL COMMAND: cd /your_deployment_path/meshiq/helm/schema-registry REVIEW CONFIGURATION: Review values.yaml and README.md files for cluster deployment settings. SHELL COMMAND: helm install schema-registry ./ 3. CONNECT MKC TO YOUR EXISTING KAFKA INSTALLATION: Access the MKC UI from a web browser, use the following address: http://:32080 NOTE: IF YOU TRY TO GET TO THE WEB UI IMMEDIATELY AFTER ALL OF THE PROCESS HAVE BEEN SUCCESSFULLY STARTED, IT IS EXPECTED TO SEE AN 'OWL' ON THE WEB UI, IT WILL MAY TAKE UP-TO 30 SEC FOR THE LOGIN SCREEN TO APPEAR. User Name: user1 (case sensitive) Password: pass (case sensitive) 1. Once logged in, select either explore Kafka or 'Kafka link' from Quick Access menu to register Kafka with your installation. 2. Then type IP address of kafka cluster (kafka-meshiq-hs), if successfully connected it will proceed to next screen. 3. Follow instructions on the screen and to enable Enable JMX and Node statistics, click "Metrics Settings" and select "Use JMX and NODE metric" 4. NOTE: edit zookeeper connection and replace host with "zookeeper-meshiq-0.zookeeper-meshiq-hs" 5. Click on register. SCHEMA-REGISTRY: Once logged in to meshIQ Kafka console to connect to deployed schema-registry follow below steps: 1. Navigate to the left side panel and select "Settings" then select "Extensions" under Cluster settings. 2. Continue and select "Schema registry Cluster Register" located in green background. 3. At this point fill out Schema registry connection settings, name your "Cluster Name" anyname will do. 4. Then select "Add Host" and point it to "schema-registry-meshiq-hs" for host. 5. Also enable "Metrics Settings" for both Schema metrics and Node metrics, Select Add. ------------------------------------------------------------------------------------------------------------------------------ YOU HAVE COMPLETED MKC IMPLEMENTATION. ========================================================================================================================================== USEFUL COMMANDS: kubectl get pods -o wide -w (to check status of running pods) kubectl label nodes "yourNodeIP" mkc=node (to lable a node) kubectl logs "PodName" (to check logs) helm uninstall "name" (to uninstall deployment) helm upgrade "name" (to upgrade deployment) ========================================================================================================================================== OPTIONAL: ==========================================================================================================================================