Monitor Azure HDInsight with Azure Log Analytics
- 3. • EdgenodeSSHWatchdog • GatewayTCPPingWatchdog • SSHTCPPingWatchdog • RStudioWatchdog • CertRolloverWatchdog • JobSubmissionPingWatchdog • OozieWatchdog • DataNodesUpWatchdog • NodeManagersUpWatchdog • ResourceHealthWatchdog • AzureNodeStatusWatchdog • ClusterMALoggingHashWatchdog • ClusterAvailabilityWatchdog • ClusterHealthWatchdog • ……..
- 8. Tool Purpose Ambari Dashboard for monitoring health and status of the Hadoop cluster Yarn UI Monitor Yarn Application and logs Tez View Track and debug the execution of jobs Grafana Workload specific JMX metrics Spark History Server The history server displays both completed and incomplete Spark jobs HMaster UI HBase provides a web-based user interface that you can use to monitor your HBase cluster Visual Studio /VS Code Monitor a Job status in VS with DataLake tools. Spark Remote Job debugging
- 21. Region Servers Hive Resource Manager Livy Zookeepers Hive Server 2 Node Manager Spark HMaster Oozie Timeline Server Jupyter HBase Rest Server WebHCat Executers Apache Phoenix Metastore Drivers RPC Phoenix Query Server
- 22. Oozie Metastore Hive Metastore ADLS Storage (Azure) GW1 Head Node 1 Worker Node Worker Node Worker Node Worker Node Zookeeper1 Edge Node HDI Management IP’s 168.61.49.99 23.99.5.239 168.61.48.131 138.91.141.162 VNET Subnet GW2Head Node 2 Zookeeper2 Zookeeper3
- 23. OMS Agent for Linux HDInsight nodes (Head, Worker , Zookeeper ) FluentD HDInsight plugin 1. Plugin for ‘in_tail’ for all Logs, allows regexp to create JSON object 2. Filter for WARN and above for each Log Type. `grep` filter plugin 3. Output to out_oms_api Type 4. Exec plugin for Metrics HBaseConfigosmconfig Spark Hive Storm Kafka Config Config Config Config Log Analytics(OMS) Service
- 26. HBase log_regionserver_CL metrics_resourcemanager_clustermetrics_CL log_hmaster_CL metrics_regionserver_CL metrics_regionserver_replication_CL metrics_restserver_CL metrics_restserver_jvm_CL metrics_regionserver_ipc_CL metrics_regionserver_wal_CL metrics_regionserver_jvm_CL metrics_regionserver_os_CL metrics_hmaser_os_CL metrics_hmaster_CL metrics_hmaster_assignmentmanager_CL metrics_hmaster_balancer_CL metrics_hmaster_fs_CL metrics_hmaster_ipc_CL metrics_hmaster_jvm_CL Hive hivemetastore_CL log_oozie_CL log_webhcat_CL Kafka log_kafkaserver_CL kafkametrics_CL log_kafkacontroller_CL Spark log_sparkappsexecutors_CL log_sparkappsdrivers_CL log_spark_CL log_livy_CL log_timelineserver_CL log_jupyter_CL VM Perf Syslog Yarn log_nodemanager_CL metrics_resourcemanager_queue_root_CL metrics_resourcemanager_queue_root_joblauncher_ CL metrics_resourcemanager_jvm_CL metrics_resourcemanager_queue_root_default_CL log_resourcemanager_CL log_mrjobsummary_CL