![Phases of Database Design (2/2)
In physical design, further specifications are provided for storing & accessing the DB. The design is implemented,
populated with actual data, and continuously maintained to reflect the state of the mini-world.
The conceptual design is then translated to a logical design.
[Expressed in a data model implemented in a commercial DBMS. E.g. Relational Data Model]
Requirements are documented in detail & transformed into a conceptual design. E.g. Entity-Relationship model
[Represented & manipulated using computerized tools easily maintained, modified, & transformed into a DB]
2/20/2024
Christalin Nelson | Systems Cluster | SoCS
14 of 48](https://image.slidesharecdn.com/overviewofdatabasesanddatamodelling-1-240220163118-4d389bd2/85/Overview-of-Databases-and-Data-Modelling-1-pdf-14-320.jpg)
Overview of Databases and Data Modelling-1.pdf
- 2. Overview of Databases and Data Modelling (Part – 1) 2/20/2024 Christalin Nelson | Systems Cluster | SoCS
- 3. At a Glance • Introduction • An Example • Characteristics of the Database Approach • Actors on the Scene • Workers behind the Scene • Advantages of Using the DBMS Approach • A Brief History of Database Applications • When Not to Use a DBMS 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 3 of 48
- 4. DBMS: A daily necessity • Essential Component of interaction through various applications – Traditional database applications • Store textual or numeric information – Multimedia databases • Store images, audio clips, and video streams digitally – Geographic information systems (GIS) • Store and analyze maps, weather data, and satellite images – Data warehouses and OLAP systems • Extract and analyze useful business information from very large databases • Support decision making – Real-time and active database technology • Control industrial and manufacturing processes – WWW • Internet Search 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 4 of 48
- 5. Introduction (1/5) • Data – Known facts that can be recorded • Database (DB) – Collection of related data that have implicit meaning – Properties • Built for a specific purpose – intended for a user group directly or via applications • Logically coherent collection of data with inherent meaning • Represents some aspect of the real world Has some source from which data is derived Has some degree of interaction with events in real world Has an audience that is actively interacting 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 5 of 48
- 6. Introduction (2/5) • DBMS – A general-purpose software system (collection of programs) that facilitates • Database Definition – Specify data types, structures, and data constraints • Database Construction – Store data on some storage medium controlled by DBMS • Database Manipulation – Retrieve and update, report generation • Database Sharing – Permit multiple users and programs to access simultaneously – Note • The used DBMS can also be a special purpose system (manually generated) 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 6 of 48
- 7. Introduction (3/5) • Simplified DB system environment – Application program • Accesses DB by sending queries to DBMS – Query • Interaction with DB • This causes some data to be retrieved/updated – Transaction • Causes data R/W into DB – Meta-data • Stored as DB catalog or dictionary 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 7 of 48
- 8. Introduction (4/5) • DBMS provides protection – System Protection • Against software and hardware malfunction or crash – Security Protection • Against unauthorized or malicious access • DBMS can maintain the DB system – Evolve as requirements change over time (life cycle of DB) 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 8 of 48
- 9. Introduction (5/5) • Example of a large commercial database? – Amazon Aurora (RDB engine) Amazon.com Storage: 2TB worth Stored on: 200 different servers Visitors per day: about 15million People reqd. to keep DB up-to-date: about 100 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 9 of 48
- 10. Example: University DB (1/3) • Purpose – To maintain information concerning students, courses, and grades in a university environment • Data Records – A record consists of data elements of different data types – Stored in five files – Records are related • File (in Conceptual level) – Collection of ordered/unordered data records of the same type 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 10 of 48
- 11. Database Structure Christalin Nelson | Systems Cluster | SoCS 11 of 48
- 12. Example: University DB (3/3) • Manipulation involves querying and updating – Examples of queries: • Retrieve the transcript • List the names of students who took the section of the ‘Database’ course offered in fall 2008 and their grades in that section • List the prerequisites of the ‘Database’ course – Examples of updates: • Change the class of ‘Smith’ to 2 • Create a new section for the ‘Database’ course for this semester • Enter a grade of ‘A’ for ‘Smith’ in the ‘Database’ section of last semester • Note: – The above should be specified in Query language of DBMS before processing 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 12 of 48
- 13. Phases of Database Design (1/2) • Information System – Consists of various computers, storage systems, application software, and DBs • i.e. DB is considered as a part of a larger undertaking – IT department within a company, designs and maintains an Information System • Phases in designing new DB or Application with existing DB Requirements specification and analysis Conceptual design Logical design Physical design 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 13 of 48
- 14. Phases of Database Design (2/2) In physical design, further specifications are provided for storing & accessing the DB. The design is implemented, populated with actual data, and continuously maintained to reflect the state of the mini-world. The conceptual design is then translated to a logical design. [Expressed in a data model implemented in a commercial DBMS. E.g. Relational Data Model] Requirements are documented in detail & transformed into a conceptual design. E.g. Entity-Relationship model [Represented & manipulated using computerized tools easily maintained, modified, & transformed into a DB] 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 14 of 48
- 15. Traditional vs. Database Approach • Traditional File processing – Each user defines & implements the files needed for a specific software application as part of programming the application – Example: Consider 2 users (Grade reporting officer & Account Officer) interested in student data • Pros: Each user maintains separate files and programs to manipulate these files because each requires some data not available from the other user’s files. • Cons: Redundancy in defining and storing data results in wasted storage space and in redundant efforts to maintain common up-to-date data. • Database approach – Maintain a single data repository that is defined once & accessed by various users 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 15 of 48
- 16. Characteristics of Database approach (1/10) • Main characteristics of database approach – Self-describing nature of a DB system – Insulation between programs and data, and data abstraction – Support of multiple views of the data – Sharing of data & multiuser transaction processing 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 16 of 48
- 17. Characteristics of Database approach (2/10) • Self-Describing Nature of a Database System – Contains DB + Metadata (stored in Catalog) • Meta-data: File structure, Type/Storage format of each data item, & data constraints – Specified by the DB designer before creating the actual DB – Database catalog used by • DBMS software – General purpose DBMS (not written for a specific application) refers to Catalog » Works for any number of DB applications • DB users (need information about DB structure) – Vs. Traditional File Processing (TFP) • DBMS software can access diverse DBs as long as the definition is stored in the Catalog • TFP software can access only specific DBs. The definitions are coded in each program 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 17 of 48
- 18. Database Catalog for the University DB Christalin Nelson | Systems Cluster | SoCS 18 of 48
- 19. Characteristics of Database approach (4/10) • Insulation between Programs and Data – Program-data independence • Structure of data files is stored in DBMS catalog separately from access programs • Vs. TFP: Any change to the structure should be reflected in the corresponding programs – Program-operation independence • In OODBMS & ORDBMS, users can define operations on data as part of DB definitions • An operation (or functions) is specified in two parts – Interface (or signature): includes Operation name – Data type of its arguments (or parameters) • Implementation of the operation is specified separately and can be changed without affecting the interface – i.e. User application programs can oregardless of how the operations are implementedperate on data by invoking these operations through their names and arguments, 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 19 of 48
- 20. • Data abstraction – Allows program-data independence & program-operation independence – DBMS provides users with a Conceptual data representation • i.e Does not include details of how data is stored or how operations are implemented • Internal implementation of a file may be defined by its – Record length – no. of characters in each record (in bytes) – Each data item may be specified by its starting byte within a record and its length (in bytes) – Data model • Type of data abstraction used to provide conceptual representation Characteristics of Database approach (5/10) Internal Storage format for a STUDENT Record based on DB Catalog Abstract Operation? E.g. Calculate_GPA 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 20 of 48
- 21. Characteristics of Database approach (6/10) • Support of Multiple Views of the Data – View • Subset of the database • Contains virtual data derived from the DB files but is not explicitly stored – Multiuser DBMS • Allow multiple users to access the database at the same time. This is essential if data for multiple applications is to be integrated and maintained in a single database. • Users have a variety of distinct applications • Must provide facilities for defining multiple views 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 21 of 48
- 22. Two views derived from the database Christalin Nelson | Systems Cluster | SoCS 2/20/2024 22 of 48
- 23. Characteristics of Database approach (8/10) • Sharing of Data & Multiuser Transaction Processing (1/3) – Multiuser DBMS? – Concurrency control software • Ensure that several users trying to update the same data do so in a controlled manner so that the result of the updates is correct • Multiuser DBMS ensures that concurrent transactions operate correctly and efficiently – Online transaction Processing (OLTP) application • E.g. Several reservation agents try to assign a seat on an airline flight 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 23 of 48
- 24. Characteristics of Database approach (9/10) • Sharing of Data & Multiuser Transaction Processing (2/3) – Transaction • Central to many database applications • Executing program or process that includes one or more database • ACID properties of Transaction – Atomicity property – Consistency property – Isolation property – Durability property 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 24 of 48
- 25. Characteristics of Database approach (10/10) • Sharing of Data & Multiuser Transaction Processing (3/3) – Atomicity property • All changes to data are performed as if they are a single operation. i.e. Perform all or none of the changes. – Consistency property • Data is in a consistent state when a transaction starts and when it ends. – Isolation property • The intermediate state of a transaction is invisible to other transactions. As a result, transactions that run concurrently appear to be serialized. – Durability property • After a transaction completes, changes to data persist and are not undone, even in the event of a system failure. During fund transfer from one account to another, the atomicity property ensures that, if a debit is made successfully from one account, the corresponding credit is made to the other account. During fund transfer from one account to another, the consistency property ensures that the total value of funds in both the accounts is the same at the start and end of each transaction. During fund transfer from one account to another, the isolation property ensures that another transaction sees the transferred funds in one account or the other, but not in both, nor in neither. During fund transfer from one account to another, the durability property ensures that the changes made to each account will not be reversed. 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 25 of 48
- 26. Actors on the Scene (1/5) • Database administrators (DBA) – Authorize access to the database – Coordinate & Monitor DB usage – Acquire resources (software and hardware) – Accountable for problems (such as security breaches and poor system response time) – Note: • In large organizations, the DBA is assisted by a staff that carries out these functions 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 26 of 48
- 27. Actors on the Scene (2/5) • Database designers – Identify the data to be stored – Choose appropriate structures to represent and store this data – Communicate with all prospective DB users to understand their requirements & create a design that meets these requirements – Develop DB views that meet the data and processing requirements of user groups. Each view is then analyzed and integrated with the views of other user groups – Note: • The designers are on the staff of the DBA and may be assigned other staff responsibilities after the database design is completed. 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 27 of 48
- 28. Actors on the Scene (3/5) • End users (1/2) – People whose jobs require DB access (querying, updating, and generating reports) – Types • Casual end users – Occasional access to different information – Use sophisticated query language to specify their requests – Typically middle/high-level managers or other occasional browsers • Naive or parametric end users – Make up a sizable portion of DB end users who constantly query and update the DB – Use standard types of queries and updates called canned transactions (transactions that were carefully programmed and tested) – E.g. Bank Tellers, Reservation agents 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 28 of 48
- 29. Actors on the Scene (4/5) • End users (2/2) – Types (contd.) • Sophisticated end users – Thoroughly familiarize themselves with the facilities of the DBMS – Implement own applications to meet their complex requirements – E.g. Engineers, scientists, business analysts • Standalone users – Maintain personal DBs by using ready-made program packages that provide easy-to-use menu- based or graphics-based interfaces Are all proficient to use all facilities of the DBMS? 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 29 of 48
- 30. Actors on the Scene (5/5) • Software Developers or Engineers – System analysts • Determine requirements of end users (especially for naive or parametric users) and develop specifications – Application programmers • Implement the specifications as programs, test, debug, document & maintain canned transactions 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 30 of 48
- 31. Workers behind the Scene (1/2) • DBMS system designers and implementers – Design and implement the DBMS modules and interfaces as a software package • E.g. Implementing catalog, query language processing, interface processing, accessing and buffering data, controlling concurrency, and handling data recovery and security • Interface with System Software (OS, Compilers, etc.) for programming languages • Tool developers – Design and implement tools • Tools are optional packages that are often purchased separately – Include packages for database design, performance monitoring, natural language or graphical interfaces, prototyping, simulation, and test data generation • Note: – In many cases, independent software vendors develop and market these tools 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 31 of 48
- 32. Workers behind the Scene (2/2) • Operators and maintenance personnel – Responsible for running and maintenance of hardware and software environment for database system 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 32 of 48
- 33. Advantages of Using the DBMS Approach (1/9) • Controlled redundancy – How is it with TFP? • Disadvantages – Duplication of effort – Wastage of storage space – Inconsistency – Data normalization – Denormalization • Combine data from multiple tables into single table that can be queried faster (Increased performance) Which one is Consistent? 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 33 of 48
- 34. Advantages of Using the DBMS Approach (2/9) • Restrict unauthorized access – Access type should be controlled • i.e. Few can retrieve, Few can retrieve & update – Security and authorization subsystem • DBA create accounts and specify account restrictions • DBMS auto enforces these restrictions – Privileged software • Security and authorization subsystem used by DBA’s staff • Canned transactions used by Parametric users 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 34 of 48
- 35. Advantages of Using the DBMS Approach (3/9) • Provide persistent storage for program objects & data structures – Data values & structures can be stored in files before being discarded – Retrieved & formatted to program’s requirements – Feature of OODBMS (compatible with C++, Java, etc.) • Vs. traditional DBMS: Impedance mismatch problem (incompatible with programming languages) 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 35 of 48
- 36. Advantages of Using the DBMS Approach (4/9) • Provide storage structures & search techniques for efficient query processing – DBMS provides specialized data structures that speed-up disk search • Index: Auxiliary File based on tree or hash data structures & are suitably modified – Choice of indices is part of physical DB design & tuning (one of DBA staff’s responsibility) – Buffering or caching module • DBMS performs its own data buffering (though OS is responsible) • DB is stored in disks. Buffering stores a part of DB in main memory to process DB records needed by a particular query – Query processing & Optimization module • Responsible for choosing an efficient query execution plan for each query based on the existing storage structures • Provide Backup and Recovery – Backup & Recovery Subsystem • Responsible for recovery from hardware/software failures 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 36 of 48
- 37. Advantages of Using the DBMS Approach (5/9) • Provide multiuser Interfaces – Example • Query languages for casual users • Forms and command codes for parametric users • Programming language interfaces for application programmers • Menu-driven & Natural language interfaces for standalone users – Note: Forms-style & menu-driven interfaces are commonly known as GUIs • Represent complex relationships among data – Numerous varieties of data that are interrelated in many ways • Example: Record for ‘Brown’ in STUDENT file is related to 4 records in GRADE_REPORT file 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 37 of 48
- 38. Advantages of Using the DBMS Approach (6/9) • Enforce Integrity constraints for data (1/2) – DBMS should define & enforce constraints – Responsibility of DB designers to identify integrity constraints during DB design – Derived from the meaning or semantics of data & of mini-world it represents – Examples • Specify data type for each data item – Example: Value of Name must be a string of no more than 30 alphabetic characters • Relate record in one file to records in other files (Referential integrity constraint) – Example: Every section record must be related to a course record • Specify uniqueness on data item values (Key or uniqueness constraint) – Example: Every course record must have a unique value for Course_number 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 38 of 48
- 39. Advantages of Using the DBMS Approach (7/9) • Enforce integrity constraints for data (2/2) – Enforcement • Specified to DBMS & auto enforced • Checked at time of data entry or by update programs (Business rules) • A data item may be entered erroneously & still satisfy specified integrity constraints. Such data entry errors can only be discovered manually and corrected later by DB updation – Example: If a student receives a grade of ‘A’ but a grade of ‘C’ is entered, the DBMS cannot discover this error automatically because ‘C’ is a valid value for the Grade data type. A grade of ‘Z’ would be rejected automatically by DBMS because ‘Z’ is not a valid value for Grade data type – Inherent rules of the data model describe the validity • Example: In ER model, a relationship must involve at least two entities 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 39 of 48
- 40. Advantages of Using the DBMS Approach (8/9) • Permitting inferencing and actions using rules – Deductive DB systems • Define deduction rules that inference new information from the stored DB facts • Example – Application have complex rules to find students on probation. These can be specified declaratively as rules, which when compiled & maintained by DBMS can determine all students on probation • Vs. traditional DBMS: Explicit procedural program code is required to support such applications – Trigger • Rule activated by updates to the table, which results in performing some additional operations to some other tables, sending messages, and so on. – Stored procedures • More involved procedures which are part of DB definition to enforce rules. Invoked when certain conditions are met. – Active DBMS • Provide active or ECA (Event-Condition-Action) rules that can automatically initiate actions when certain events and conditions occur 2/20/2024 Christalin Nelson | Systems Cluster | SoCS Rule Change! (Change code or rules) – Which one is easy? 40 of 48
- 41. Advantages of Using the DBMS Approach (9/9) • Additional implications of using the database approach – Potential for enforcing standards • Vs. User group having control of its own files – Reduced application development time • 0.17 to 0.25 of time taken for traditional file system – Flexibility • Change in DB structure without affecting data & existing application programs – Availability of up-to-date information • Concurrency control & recovery subsystems – Economies of scale • Reduction in overall cost of operation & management 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 41 of 48
- 42. A Brief History of Database Applications (1/5) • Early database applications using hierarchical and network systems – Large numbers of records of similar structure, many record types & interrelationships – Implemented on large expensive Mainframe computers (mid 1960s to 1980s) – Paradigms • Hierarchical systems, Network model-based systems, Inverted File system – Cons • Laborious to reorganize DB when application’s requirements changed • Provided only programming language (PL) interfaces 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 42 of 48
- 43. A Brief History of Database Applications (2/5) • Providing data abstraction and application flexibility with relational databases – Separates physical storage of data from its conceptual representation – Provides a mathematical foundation for data representation and querying – High level Query language instead of PL interface (late 1970s • Flexibility to quick creation of new queries and DB reorganization – Exist in almost all types of computers 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 43 of 48
- 44. A Brief History of Database Applications (3/5) • Object-oriented applications and the need for more complex databases – OO programming languages (1980s) & later OODB – General data structures, store complex structured objects, incorporated OO concepts – Used only in specialized applications • E.g. Engineering design, multimedia publishing, and manufacturing systems • Interchanging data on the Web for e-commerce using XML – e-commerce applications (1980s) uses DBMS – Web publishing language to create documents (like HTML) • Documents are linked through hyperlinks – Extended markup language (XML) - Primary standard for interchanging data among various types of databases and Web pages 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 44 of 48
- 45. A Brief History of Database Applications (4/5) • Extending database capabilities for new applications – Unsuitability of the RDBs • Requirement of new data types, more complex data structures,, new operations & query language constructs required to manipulate new data types, new storage & indexing structures – Extensions to better support specialized requirements for applications • Scientific applications, Storage and retrieval of images/videos, Data mining applications, Spatial applications, Time series applications – Web-enabled applications & DB backends (one/more DBs from different vendors & different models) • Enterprise resource planning (ERP) • Customer relationship management (CRM) 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 45 of 48
- 46. A Brief History of Database Applications (5/5) • Databases versus information retrieval – Information retrieval (IR) • Deals with books, manuscripts, and various forms of library-based articles 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 46 of 48
- 47. When Not to Use a DBMS • DBMS involves overhead costs (vs. TFP) – High initial investment – Security, Concurrency control, recovery & integrity functions • More desirable to use regular files for – Simple, well-defined DB applications not expected to change at all – Stringent, real-time requirements that may not be met because of DBMS overhead – Embedded systems with limited storage capacity – No multiple-user access to data 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 47 of 48
- 48. 2/20/2024 Christalin Nelson | Systems Cluster | SoCS 48