��Storm��ʵս��

2016-12-14 12:12| ��: ��| �鿴: 493716| ��: 0|ԭ��: ��

��ٱ��

�ڴ��ݵ��Hadoop�ǲ��ɺ��ߣ�Ȼ��ʵʱ��ϵ��ӳ٣�Hadoopȴ��Ĳ��㣬Ϊ��ƴ��ʵʱ�Դ��ҵ��˲��ٵĳ��ԣ��Facebook��2011�귢��ġ�Apache Hadoop Goes Realtime at Facebook��н��Hadoop�Ͻ��ʵʱ��ϵͳ��ظĽ��ͬʱ��Puma��վ�û��ʵʱ��Ա��Լ��Ĳ�Ʒ��Ӫ��Ϊ��Ʒѣ�cost-per-click��Yahoo��S4��ʵʱ��㡢Ԥ��û��Թ��Ŀ��ܵĵ��Ϊ��LinkedIn��Kafka��Samza��ʵʱ��͡��͸��ӵļ��صȣ��Storm��Twitter��Դ��ʵʱ��ܣ��ʵʱ��ѧϰ��㡢�ֲ�ʽRPC��ETL�ȳ��

��ݼ��ķ�չ��죬��ӿ�ֵļ��ʢ��ڱ��ογ��У��ǳ��ķ��ϵͳ��Storm��ϵ�ܹ��Լ��ݵ�һЩӦ�á��ʹ��Storm��ʵʱDashboardͳ�ơ��ס��û��ʵʱ�Ƽ��Ӧ�á�ͬʱ��ʵʱ��ݴ��·��ռ��ͼ��㡢�Լ��洢�ȣ�ʵʱ��ߵ��ݡ�

�γ̴��٣�
��һ�Σ�ʵʱ��ƽ̨��
��ʵʱϵͳ��Ҫ��ҵ��ٵ��ս��
��Ҫ��ʵʱƽ̨��Ҫ��͹��ɣ�ʵʱ��ݵ��ռ�ͨ��ݽ��ʽ
ʵʱ��ݺ��Flume��Kafka�Ľ��ܣ��Լ�Storm��ƽ̨��λ�ã�
��ֵ��ͻ��˾ʵʱƽ̨�ļܹ��ڵ��š�һ�ŵ��

�ڶ��Σ�Storm��
Storm�Ļ��Nimbus��Supervisor��Worker�� Executor��Task�Ļ��ܣ�
��Ⱥ��ɣ�ͨ��һ��ϼ�Ⱥ��ι��ɣ�
Storm�Ŀɿ��ԣ�Storm��α�֤�ɿ��Լ��ݵ�׼ȷ�ԣ�
Storm��ݷ��Խ��ܣ�

��Σ�Storm��Ⱥ��
Storm��ܣ�
Storm�Ĳ��Ӳ��Ҫ��
��ZooKeeper
��Storm��ڵ�
��Storm��ز��Լ��Ĳ��ܣ�
��Storm��ؽ��̣�
Storm��ػ��̣�
�ύTopology�Ĺ��̽��ܡ�

��ĿΣ�Storm��Ӧ�õĿ��
Storm��Ӧ�ÿ��͵��Թ��̽��ܣ��
�ύʾ��Topology��Ⱥ��
��Ŀ��뱾�ؿ��ú��
��ʹ��Լ�ע��
��ش��Թ��̣�
��Topology�ύ��̺��

��Σ�Storm Nimbus��Supervisor��
Nimbus��ܽ��ܺ��Nimbus��̷��
Nimbus��ִ�й��
��Executor��㷨��
��ܣ�
Ĭ�ϵ��DefaultScheduler�;��EvenScheduler�߼��
Supervisor��ݽṹ��
Supervisor��ִ�й��⡣

��Σ�Storm Worker��Executor��Task��
Worker��ݽṹ�ͼܹ��
Worker�е��ֽ⣻
��Worker�Ĺ��̣�
Executor�Ĵ��̣�
��Spout��Executor�Ĺ��̺��߼��
��Bolt��Executor�Ĺ��̺��߼��
Task�Ĵ��̣�
Ack��ԭ��ܣ�
Acker Bolt��ʵ�֡�

��߿Σ�Storm��ά�ͼ��
��Ϣ��أ�
��־�ͼ��أ�
Storm UI��NimbusClient��ʹ�ã�
Storm Metric��ʹ�ã�
Storm ZooKeeper��Ŀ¼��͹��ܷ��
Storm Hook��ʹ�á�

�ڰ˿Σ�Storm��չ�Ͷ��ο��
Storm UIԭ��ܽ��ܺ��ݺ��壻
Storm UI�¹��ʵ�֣�
Storm��Thrift�ӿڽ��ܣ�
��Դ��뷽��飻
��CGroup��Դ��ʵ�֣�
ʹ��Docker��Storm��Ⱥ��ܡ�

�ھſΣ�Storm��ʵ��Ӧ�ð��
ʵʱDAU��ʵ�֣�
ʵʱ�û��
ʵʱ��Ի��Ƽ��
��Ͷ�ŵľ�׼��

��ʮ�Σ�Stormʹ�þ��Ż�
ʹ�þ��飻
��Ż��飻
�Զ��metric��ռ�

��ʮһ�Σ��ʵʱƽ̨��
JStorm��ܣ�
Spark Streaming��ܣ�
Heron��ܣ�
Flink��ܣ�
Storm��ʵʱ��ܵĹ��ܺ��ܶԱ�

�ڿ�ʱ�䣺

�γ̽��2017��9��23�տ��Σ��γ̳��ʱ��ԼΪ13�ܡ�

�ڿζ����Linux��һ��֪ʶ��JavaЩ��ʶ��Stormʵʱ��ݷ��ƽ̨��Ȥ��IT�˻�ѧϰ��

�ջ�Ԥ�ڣ��ܶ��Storm��Ⱥ��Ӧ��ݷ��

��ʦ��飺
��´�
��δ��ڵ��ƽ̨�߼��ܹ�ʦ��ڵ��ƽ̨��ϵͳ��ʵʱ��ƽ̨��߷��ȣ��ƶ��صļ��ߺͷ��ڵ��Ӧ�úͷ�չ��Ϊ��˾�ڲ��ݷ��ʦ��BI�Ϳ��ʦ�ṩ��ݷ��ߺ�ƽ̨��ƽ̨��ߴ��PB��ʵʱ��50TB��
��Storm��Ļ��ʵ��

�γ��

��ӱ�Ŀγ��շ��ʽ����շѡ�Լ��ѧϰ��ȡ100Ԫ�̶��շ�+300Ԫ�ݴ�ѧ�ѣ�ѧϰԲ��ȫ���ѧԱ��

�ر�˵��£�
��ſγ̱��ȫ��ѣ�ĳλ��˵��ɹ��ȷ�ķ��ټ��ʶȵ�ѹ��ǵ��ʦ��Ҫ��޴��Ͷ��Ϊ�˷�ֹһЩ��ѧϰ;�а�;��ϣ��˷��˽�ʦ�ĸ��Ϊ��Ǽƻ�ģ��ĳЩ��γ̣�ʹ�á��շѡ��ķ��
�� ʱÿλ��ȡ400Ԫ��100ԪΪ�̶� �շѣ��300��ݴ�ѧ�ѣ��ѧԱ��ȫ��γ�Ҫ�󣬰��ȫ��ͻ��ҵ��ÿ�ܲ��2Сʱ��300Ԫȫ��˻ء��ѧԱδ�ܼ��ֵ��ȫ��е�ѧϰ�ƻ��ᱻ�ۿ��ַ�ʽ��ת��Ϊ��ǿ�ҵ�ѧϰԸ��

�γ��ڿη�ʽ��
1�� ѧϰ��ʽ��ʦ��ѧ��ϡ��̲ģ��õ�Ƭ��Ƶ��ѧԱͨ��ѧϰ��ͬʱͨ��̳��ʦ��ѧԱ��ָ��ѧԱ֮��໥��
2�� ѧϰ��ҵ��ʦÿ�ܲ��漰��ҵ��ѧԱ�谴ʱ��ҵ��
3�� ʦ��ҵ��з��ֵ��⣬��Ը��踨��֪ʶ��
4�� ҵ��飺ͨ��飬��ѧҵ��

��ͼ��ɽ��ݿγ̵�ͼ