Hive
- 1. V.SAKTHIPRIYA II-MSC (IT) NADAR SARASWATHI COLLEGE OF ARTS AND SCIENCE
- 2. APACHE HIVE Apache Hive is a data warehouse system built on top of Hadoop and is used for analyzing structured and semi- structured data. Hive abstracts the complexity of Hadoop MapReduce. Basically, it provides a mechanism to project structure onto the data and perform queries written in HQL (Hive Query Language) that are similar to SQL statements. Internally, these queries or HQL gets converted to map reduce jobs by the Hive compiler.
- 3. INTRODUCTION: Hive QL is the HIVE QUERY LANGUAGE Hive offers no support for row-level inserts, updates, and deletes. Hive does not support transactions. Hive adds extensions to provide better performance in the context of Hadoop and to integrate with custom extensions and even external programs.
- 4. CON.. DDL and DML are the parts of HIVE QL Data Definition Language (DDL) is used for creating, altering and dropping databases, tables, views, functions and indexes. Data manipulation language is used to put data into Hive tables and to extract data to the file system and also how to explore and manipulate data with queries, grouping, filtering, joining etc.
- 5. DATABASES IN HIVE: The Databases in the Hive is essentially just a catalog or namespace of tables. They are very useful for larger clusters with multiple teams and users, as a way of avoiding table name
- 6. CON Hive provides commands such as CREATE DATABASE db name -- to create a database in Hive USE db name -- To use the database in Hive. DROP db name -- To delete the database in Hive. SHOW DATABASE -- to see the list of the DataBase
- 7. TABLES IN HIVE: Hive table is logically made up of the data being stored and the associated metadata describing the layout of the data in the table. The data typically resides in HDFS, although it may reside on any Hadoop file system including the local file system. Hive stores the metadata in a relational database and not in HDFS. The command for creating a table in Hive is have>CREATE TABLE EMP (empid int, ename string, esal double)
- 8. To have, we are having two types of tables Managed tables External tables Managed tables External table
- 9. MANAGED TABLES Managed tables are the one which will be managed in the Hive warehouse i.e. whenever we create a managed table definition, it will be stored under the default location of the Hive warehouse i.e./user/Hive/warehouse. When we drop a managed table, Hive deletes the data in the table Managed tables are less convenient for sharing with other tools.
- 10. SYNTAX FOR CREATING HIVE MANAGED TABLE:- Hive>create table manage- tab (empid, ename string, esal int) row format delimited fields terminated by ‘t’ lines terminated by ‘m’ stored as a text file; How to load the data into managed tables We can load the data in two ways Local Mode HDFS Mode
- 11. In local mode, the syntax is hive>load data local in path’/home/new Batch/input1.txt’ Into table managed-tab;For HDFS mode, the syntax is hive>load data in path’/user/ramesh/Hive/input2.txt’ Into table managed – tab;
- 12. EXTERNAL TABLES: Along with the managed tables, Hive also uses external tables. Whenever the key word ‘external’ comes in the table definition part. A hive will not bother about the table definition, i.e. the external table will not be managed by the Hive warehouse system. Along with the external keyword, we can also mention the ‘location’ in the table definition, where exactly the table definition will get stored.
- 13. SYNTAX:- Hive>create external table external- tab(empid int, ename string, esal double) row format delimited fields Terminated by ‘f’ lines terminated by ‘n’ stored as text file location ‘userRameshHive-external’;A location will be automatically created. Loading data into External Tables:- Loading data from HDFS to Hive>load data in path’/Ramesh/input data.txt’ into table external-tab;
- 14. ALTERING TABLE: Most table properties can be altered with the ALTER TABLE statement, which change metadata about the table but not the table itself. ALTER TABLE modifies table meta data on. Then statements can be used to fix mistakes in schema, move partition locations and do other operations
- 15. RENAMING A TABLE: This statement is used to rename the table Log_messages to log msgs Cmd: ALTER TABLE log _ messages RENAME To logmsgs;
- 16. CHANGING COLUMNS You can rename a column, change its position, type or comment. Syntax: ALTER TABLE log-messages CHANCE COLUMN hms hours-minutes- Seconds INT COMMENT ’The hours, minutes and seconds are part of the times tamp’AFTER Severity; You have to specify the old name, a new name and the type even if the name or type is not changed.
- 17. ADDING COLUMNS You can add new columns to the end of the existing columns, before any partition. Example: ALTER TABLE Log-message ADD COLUMNS(app-name String COMMENT” Application Name” ,session-id long);
- 18. DELETING OR REPLACING COLUMNS: The replace statement can only be used with tables that use one of the native ser De modules are Dynamic Ser De or Metadata Type column set ser De. Ser De determines how records are parsed into columns i.e deserialization and how records columns are stored (serialization)
- 19. PARTITIONS: A table may be partitioned in multiple dimensions. Partitioned are defined at table creation time using the PATITIONED by the clause, which takes a list of column definitions. If we want to search a large amount of data, then we can divide the large data into partitions.
- 20. BUCKETS: There are two reasons why you might want to organize your tables (or partitions) into buckets. The first is to enable more efficient queries. The second reason to bucket a table is to make sampling more efficient.