Mongosync Insights version 0.9.1.15

[BigMarcio]

Mongosync Insights version 0.9.1.15
Remove the connectivity information from Monitoring UI
Document the tested Migration Verifier and Mongosync versions
Reorder the Monitoring form to show priorities
Improve MI logic to identify mongosync log and metric files
Validate progress endpoint URL
Add endpoint port validation
Allow to copy Mongosync Progress table content
Add a new Error pattern
Add a search by date and time into Log Vierwer
Add automated unit tests
Validate the Verifier Metadata version for compability
Fix Verifier mismatches to show what is missing
Add Migration Verifier monitoring through endpoint
Change Migration Verifier Monitoring UI to use the same look and feel from Migration Monitoring

[Copilot]

Pull request overview

This PR bumps Mongosync Insights to v0.9.1.15 and expands both the Log Analyzer and Live Monitoring experiences (including standalone Migration Verifier monitoring), along with adding a dedicated automated test suite and CI workflow for the migration/mongosync_insights subtree.

Changes:

• Enhances Live Monitoring UI/UX (data-source badges, reordered inputs, verifier dashboard polling split into progress/summary/metadata).
• Improves Log Analyzer capabilities (better log/metrics file recognition, archive handling, log viewer datetime range search, progress table copy-as-Markdown).
• Adds a pytest suite + a GitHub Actions workflow to run tests on PRs.

Reviewed changes

Copilot reviewed 56 out of 63 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
migration/README.md	Updates top-level migration docs to reflect verifier monitoring on /live/.
migration/mongosync_insights/tests/conftest.py	Adds shared pytest fixtures (app client, templates, sample data).
migration/mongosync_insights/tests/fixtures/sample_mongosync.log	Adds committed log fixture for e2e upload tests.
migration/mongosync_insights/tests/fixtures/sample_mongosync_metrics.log	Adds committed metrics fixture for e2e upload tests.
migration/mongosync_insights/tests/test_app_config.py	Adds extensive tests for config/env parsing, endpoints, verifier settings, and helpers.
migration/mongosync_insights/tests/test_connection_cache.py	Updates cache-clearing tests for new verifier metadata client cache.
migration/mongosync_insights/tests/test_connection_validator.py	Adds tests for connection-string sanitization.
migration/mongosync_insights/tests/test_create_app.py	Adds app-factory tests and verifies blueprint registration.
migration/mongosync_insights/tests/test_data_sources.py	Adds tests for data-source badge payload builder.
migration/mongosync_insights/tests/test_file_decompressor.py	Adds tests for archive decompression + macOS metadata skipping.
migration/mongosync_insights/tests/test_index_build_destination_cache.py	Adds tests for destination index-build cache behavior.
migration/mongosync_insights/tests/test_live_metadata_status.py	Adds tests for live metadata helpers and gating logic.
migration/mongosync_insights/tests/test_live_monitoring.py	Adds tests for live monitoring fetch/build behavior and timeouts.
migration/mongosync_insights/tests/test_live_routes.py	Adds tests for Flask live routes and verifier endpoints.
migration/mongosync_insights/tests/test_log_store.py	Adds tests for SQLite log store insert/search utilities.
migration/mongosync_insights/tests/test_log_store_registry.py	Adds tests for store registry TTL + maintenance runner.
migration/mongosync_insights/tests/test_log_time.py	Adds tests for datetime parsing and search-bound normalization.
migration/mongosync_insights/tests/test_logs_metrics.py	Adds tests for log/metrics parsing helpers and MIME detection.
migration/mongosync_insights/tests/test_logs_routes.py	Adds tests for logs blueprint routes incl. timestamp-range search.
migration/mongosync_insights/tests/test_otel_metrics.py	Adds tests for Prometheus/otel metrics parsing & collector behavior.
migration/mongosync_insights/tests/test_plot_theme.py	Adds tests for Plotly theme helpers and template registration.
migration/mongosync_insights/tests/test_session_support.py	Adds tests for session cookie/store update/create behavior.
migration/mongosync_insights/tests/test_snapshot_store.py	Adds tests for snapshot save/load/list/delete/cleanup lifecycle.
migration/mongosync_insights/tests/test_store_paths.py	Removes route tests from this file (moved to dedicated logs/live route test modules).
migration/mongosync_insights/tests/test_upload_fixtures.py	Adds end-to-end upload tests using committed fixtures.
migration/mongosync_insights/tests/test_utils.py	Adds tests for formatting helpers in lib/utils.py.
migration/mongosync_insights/templates/base.html	Adds Home sidebar link and introduces template blocks to suppress upload/settings UI on some pages.
migration/mongosync_insights/templates/live/home.html	Reorders form inputs (endpoint first), adds verifier endpoint fields and client-side validation.
migration/mongosync_insights/templates/metrics.html	Disables upload UI/sidebars for monitoring page layout.
migration/mongosync_insights/templates/upload_results.html	Adds log viewer datetime range filters and progress-table copy-as-Markdown overlay/button logic.
migration/mongosync_insights/templates/verifier_metrics.html	Rebuilds verifier page to use shared live-monitor styling and split polling JS.
migration/mongosync_insights/static/css/mi-live-monitor.css	Adds badge styling and verifier dashboard layout tweaks.
migration/mongosync_insights/static/css/mi-upload-results.css	Adds styles for chart copy badge and log viewer datetime UI layout.
migration/mongosync_insights/static/js/mi-live-monitor.js	Removes connectivity card; adds data-source badges to toolbar and error/empty states.
migration/mongosync_insights/static/js/mi-verifier-monitor.js	Adds standalone verifier dashboard renderer (DOM-based).
migration/mongosync_insights/static/js/mi-verifier-poll.js	Adds independent polling for verifier progress/summary/metadata slices.
migration/mongosync_insights/requirements-dev.txt	Adds pytest dev dependency.
migration/mongosync_insights/README.md	Documents prerequisites, compatibility notes, dev/test instructions, and updated workflow descriptions.
migration/mongosync_insights/PACKAGING.md	Updates packaged env example to include verifier settings and new timeouts/error patterns file.
migration/mongosync_insights/MIGRATION_MONITORING.md	Updates monitoring docs, verifier split polling, and clarified data-source priority.
migration/mongosync_insights/LOG_ANALYZER.md	Documents filename classification rules, log viewer datetime search, and snapshot TTL behavior.
migration/mongosync_insights/LOG_VERBOSITY.md	Documents progress-table “Copy as Markdown” behavior.
migration/mongosync_insights/CONFIGURATION.md	Adds verifier-specific env vars, timeouts, and updated refresh semantics.
migration/mongosync_insights/CONNECTION_STRING.md	Updates security notes and adds verifier connection-string behavior.
migration/mongosync_insights/lib/app_config.py	Version bump + new endpoint validation, verifier settings/timeouts, file classification rules, and verifier metadata client support.
migration/mongosync_insights/lib/data_sources.py	Introduces data-source badge payload builder.
migration/mongosync_insights/lib/error_patterns.json	Adds a new embedded verifier CollectionUUIDMismatch retry pattern.
migration/mongosync_insights/lib/file_decompressor.py	Skips macOS archive metadata members and applies skipping to classified archive decompressors.
migration/mongosync_insights/lib/live_metadata_status.py	Switches to new mongosync internal DB auto-detection helper.
migration/mongosync_insights/lib/live_monitoring.py	Adds configurable timeouts and returns data-source badge info instead of connectivity info.
migration/mongosync_insights/lib/live_verifier_metadata.py	Uses new embedded verifier DB name variables for persistence DB access.
migration/mongosync_insights/lib/log_time.py	Adds helpers for parsing and normalizing datetime bounds for log timestamp searching.
migration/mongosync_insights/lib/logs_metrics.py	Improves archive/file-type handling, adds “unrecognized file” error, and passes progress-table data to template.
migration/mongosync_insights/blueprints/live.py	Adds verifier endpoint form inputs + routes for progress/summary/metadata slices; improves endpoint validation.
migration/mongosync_insights/blueprints/logs.py	Adds timestamp-range search parameters with validation.
.github/workflows/mongosync-insights-tests.yml	Adds CI workflow for running pytest under migration/mongosync_insights/.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.