Open Source Python Data Management Systems

This is an ETL software which loads data from DBF/XBase files into MySQL. This utility has command line interface, designed to work without user interaction.

We're excited to announce the launch of a new open-source product, data-diff that makes comparing datasets across databases fast at any scale. data-diff automates data quality checks for data replication and migration. In modern data platforms, data is constantly moving between systems, and at the modern data volume and complexity, systems go out of sync all the time. Until now, there has not been any tooling to ensure that when the data is correctly copied. Replicating data at scale, across hundreds of tables, with low latency and at a reasonable infrastructure cost is a hard problem, and most data teams we’ve talked to, have faced data quality issues in their replication processes. The hard truth is that the quality of the replication is the quality of the data. Since copying entire datasets in batch is often infeasible at the modern data scale, businesses rely on the Change Data Capture (CDC) approach of replicating data using a continuous stream of updates.

SnapLogic is an Open Source Data Integration framework that combines the power of state-of-the-art dynamic programming languages with standard Web interfaces to solve today's most pressing problems in data integration.

A group a subprojects for Data Cleaning projects, mainly as a step of a Data Mining Project. Visit www.datacleaningopensource.com to review our current applications or if you want to add yours. NOTE: PROGRAMMING SKILLS ARE REQUIRED.

A collection of tools for working with the comparative data analysis ontology including import/export facilities for common phylogenetic file formats, and also a triple-store framework.

Spools data for a given query or table from 14 databases. Windows command-line application.

Touch and go Windows command line data migration tool for 14 databases: 1. Sybase ASE 2. Informix Innovator C 3. Sybase SQL Anywhere 4. DB2 UDB 5. SQLServer 6. MariaDB 7. Sybase IQ 8. PostgreSQL 9. MySQL 10. Informix IDS 11. TimesTen 12. Oracle 13. SQL Lite 14. Exadata

#SaveUkraine #StopRussia #FreeUkraine #StopPutin #CrimeaIsUkraine #UnitedForUkraine #RussiaInvadedUkraine Exports/Spools scalar data on disk for a given Oracle table. Turbo mode spools 5x faster.

#SaveUkraine #StopRussia #FreeUkraine #StopPutin #CrimeaIsUkraine #UnitedForUkraine #RussiaInvadedUkraine Spools/extracts/dump table or query data from SQL Server 2015, 2008,2012. Serial spool creates single dump file. Turbo mode offers 5x spool performance improvement. Sharded turbo more creates multiple files.

Migrate your data from SQLServer to Oracle without creating single dump file. Input is a SQLServer query file defining dataset you want to copy to Oracle. Target table has to exist for copy to go through. Turbo mode offers 5x copy performance improvement.

FLAG was designed to simplify the process of log file analysis and forensic investigations. FLAG facilitates efficient analysis of large quantities of data within an interactive environment. PyFlag is the reimplementation of FLAG in Python.

<This project has been completely rewrote and transformed into a new one: https://sourceforge.net/projects/deday/. 2013/06/26> The graphic user interface for DEDA (DEmography Data Analysis), a scientific software package fitting survivalship data to a number of distributions using maximum likelihood (ML) method. Currently, Weibull (2p), Gompertz and Gompertz-Makeham are supported. IMPORTANT NOTICE: Only the GUI is provided here. In order to perform the analysis, one also need the DEDA computation core program. Please email a request to me, entitled: 'Request for DEDA computation core', if you wish to have a copy.

A lightweight, browsing-based, 100% Python, federated data integration framework. Users may create custom schemas for disparate sources, query and expand results across sources to find related data; for use in fields such as bioinformatics and datamining

A tool that parses SQL Select statements and generates a diagram. The diagram shows parts of the underlying SQL directly in the diagram. For example x=30 , GROUP BY (year), HAVING MIN(age) > 18. It is easy to see cartesian joins and/or loops.

idyuts is \"I Dare You to Use This Shell\"; a pre-hibernate approach to replacing an ORM written with jython functors into a pure-Java language command pattern. The \"pipeline codegen artifacts\" are simple IoC templates, and trivial to adapt

#FreeUkraine #SaveUkraine #StopRussia #StopPutin #CrimeaIsUkraine #UnitedForUkraine #RussiaInvadedUkraine UI written using wxPython. Allows you, to copy tables between Oracle databases using drag-n-drop interface. AKA filezilla, but for tables, not files.

To create a framework to extract Web data and store in local RDBMS, to generate assessment reports on quality of the data being extract, and to publish the quality reports on the Web.

Python based webservice to manage and display information from some fiber-GISes.

TODO

pyMantis is a data-management system for (systems) biology build on the web2py framework. It features: tree based file explorer, relational db table wizzard with automated creation of user interfaces, internal and external access management, wiki, ..