Statistical Databases
- 1. Statistical Databases By Samir Rana
- 2. Database used for statistical analysis.
- 3. ● Researchers have access to statistics, but not the records inside. ● Access control and limited query keeps prying eyes off from sensitive data ● Basic ops Limited to : Count, Sum, Mean, std deviation etc
- 4. Requirements Statistical Databases ● Data Model ● Query Language ● Integrity Constraints ● Recovery ● Physical DB organization ● Data analysis requirements
- 5. Data Model 1. Due to Multidimensionality of SD, Relational Model is not suitable. 2. New data structures and operations are needed, such as data cube operator and aggregate Data structures Different Data model proposed: SUBJECT , Semantic Association Models, GRASS, Conceptual Statistical Model, Statistical Object Representation Model
- 6. Query Language ● powerful and easy-to-use query languages to define and manipulate statistical data. ● Evaluation Criteria of Statistical query languages : ○ data and metadata definition, ○ data manipulation, ○ interface to statistical packages, ○ the expressive power of the language.
- 7. Query Languages ● SDBMS built on top of CDBMS: ○ GRAFSTAT on DB2(SQL/DS), ○ STRAND on INGRES ● Generalized Interface system that links together available CDBMS, statistical packages and graphics software ○ SIBYL , GPI and PEPIN-SICLA ● Separately developed SDBMS: ○ RAPID, CAS SDB, ABE, SIR/SQL , GENISYS, CANTOR. ○ SIR/DBMS, TPL, TPLDCS, BROWSE. ● SDBMS with graphical user interfaces: ○ SUBJECT , GUIDE , ABE , STBE, SEEDS online code book. ● Formal Extensions of Relational Model: ○ SSDL ● Natural language based user interface: ○ LIDS 86 . ● Query languages which calculate aggregates from temporal data: ○ QUEL , HQUEL , TBE.
- 8. Tree Based Statistics Access Method(TBSAM) ● Calculate set-of-aggregates of all data items such that boolean qualification ● Based on the B+ tree, and it exploits all the benefits of a B+ trees dynamic nature. ● Aim is the efficient retrieval of a tuple, given the value of its index attribute. ● Dynamic index, and thus can support insertion/deletion/modification of tuples in relation. ● Various types of statistical queries can be facilitated : ○ descriptive statistics, ○ order statistics, ○ statistical sampling types of queries.
- 9. Processing and Optimization ● large portion of statistical data are either spatial or temporal data. ● pure tables of relational databases are not capable of efficientlystoring or helping retrieving such data. ● algorithms reorder the operations to be performed → build the optimal or suboptimal query processing tree → depending on the physical data storage structures, chooses the best possible strategy to query data. Operations on Temporal data ● Temporal theta join : the conjunction of two sets or predicates, the time join predicate and the non-time join predicate. ● TE-join : two tuples (or rows) in two join relations (tables) are joined if their time intervals intersect. ● T-join : causes the concatenation of tuples from the operand relations only if their time intervals intersect.
- 10. Security ● Easy to infer, The contents of Specific Records from Statistical Data ● Conflict of Providing Statistics and securing individual records gives rise to Inference Control. ● Type of Inference Control ○ Query set Restriction ○ Data perturbation ○ Output Perturbation ○ Conceptual Approach Evaluation of Effectiveness of Inference control: ● Security ● Robustness ● Bias ● Precision ● Consistency ● Cost
- 11. Other Applications ● Data Visualization: ○ A point in multidimensional space. ○ Can be used as a basis to build an interactive data visualization system. ○ User can browse in the multidimensional space. ● Statistical expert systems: ○ a program which can act in the role of an expert statistical consultant. ○ give expert advice on how to design a study, what data to collect to answer the research questions, and how to analyze the data collected.
- 12. Conclusion ● Applications that collect vast amounts of data, and require interactive real-time analysis capabilities on it, is on the rise. ● the standard approach of statistical analysis to load part of the data from a file or database into a statistical package, and then performing analysis on it will not work due to efficiency reasons. ● The overall goal of research in statistical database management has been to make this analysis an integral part of the data management system itself. ● The focus of the research community has been on developing techniques to make this happen.
- 13. Thank you