aboutsummaryrefslogtreecommitdiff
path: root/webmentions_ssg/tasks/scanner.py (follow)
Commit message (Collapse)AuthorAge
* refactor(core)!: standardize application terminologyDennis Fink7 days
| | | | | | | | | | | | | Rename configuration keys, task helpers, forms, network utilities, and view endpoints to make their responsibilities and domains more explicit. Align source-management routes and templates with the new terminology and rename the target-matching module and related task abstractions consistently. BREAKING CHANGE: configuration keys have been renamed and sent-source routes move from `/sent` to `/sources`, with corresponding blueprint endpoint changes.
* docs(core): standardize reST docstringsDennis Fink8 days
| | | | | | | | | Expand function and method documentation with consistent reST parameter, return, and exception fields while clarifying behavior where additional context is useful. Normalize test and migration docstrings to the same documentation conventions.
* refactor(core): split task workflows into focused helpersDennis Fink9 days
| | | | | | | | | Decompose receiver verification, source scanning, and Webmention delivery into explicit claim, check, reconciliation, and outcome-recording stages. Extract Huey backend parsing helpers and add focused tests for the new task boundaries while preserving existing processing semantics.
* refactor(sender): rename locked task helpersDennis Fink9 days
| | | | | | Rename the non-Huey scanner and sender helpers to `_scan_locked` and `_send_locked` so their names reflect that they run only while holding task locks.
* refactor(core): clarify Webmention mutation APIsDennis Fink9 days
| | | | | | | | Make in-place reset and reopen helpers return `None` so callers no longer perform pointless self-reassignment. Split explicit resend behavior into its own method instead of hiding it behind a boolean flag on `reopen`.
* refactor(core): centralize pending revision logicDennis Fink9 days
| | | | | | | | Make `SentWebmention.pending` a hybrid property so the same definition works in Python and SQL expressions. Reuse it across sender and scanner guards to remove duplicated revision checks.
* fix(core): decouple task imports from app initializationDennis Fink10 days
| | | | | | | | | | Keep stable lazy task handles that register against the active Huey backend so modules can be imported before app creation and survive backend reinitialization. Separate package metadata and the shared blueprint from application startup to remove import-order dependencies and keep build-time version lookup lightweight.
* refactor(core): use lowercase extension namesDennis Fink10 days
| | | | | | Rename shared Flask extension singletons to conventional lowercase names and update imports throughout the application and tests to remove redundant aliases.
* chore(core): enable stricter Ruff lint rulesDennis Fink10 days
| | | | | | | | Expand Ruff checks for bugbear, complexity, pathlib, return statements, simplifications, upgrades, and docstring style. Apply the resulting behavior-neutral cleanup across application and test code.
* refactor(core): simplify task control flowDennis Fink10 days
| | | | | | Replace unnecessary pattern matching and temporary assignments with direct conditionals and inline configuration access without changing task behavior.
* refactor(core): split modules by responsibilityDennis Fink10 days
| | | | | | | | Break up monolithic views, task helpers, networking, CLI, and application extension setup so each module owns a focused concern. Reorganize the test suite around the new boundaries while preserving existing Webmention behavior and coverage.
* fix(sender): prevent source scanner path escapesDennis Fink2026-10-31
| | | | | | | | | Reject symlinked source files whose resolved targets leave the configured source directory so scans cannot read unintended files from the host filesystem. Normalize and validate ignored hostname patterns at startup and again at scan time to keep scanner filtering predictable.
* fix(sender): validate missing source directoriesDennis Fink2026-10-30
| | | | | | Resolve configured source paths without requiring them to exist first so missing directories produce the scanner's explicit directory validation error instead of leaking a lower-level path resolution exception.
* fix(sender): stabilize source content fingerprintsDennis Fink2026-10-30
| | | | | | | | | Hash normalized reader-facing content and outgoing targets instead of raw h-entry markup so generated attributes and relative timestamps do not trigger spurious source revisions or repeat Webmentions. Preserve meaningful changes such as image updates and reaction target changes while ignoring target ordering and other markup noise.
* fix(sender): retry manual scans on lock contentionDennis Fink2026-10-30
| | | | | | | | Ensure requested rescans are not silently dropped when a periodic scan already holds the scanner lock. Defer the manual task and retry it after the active scan finishes while still propagating unrelated scan failures.
* feat(sender): add persistent retry backoffDennis Fink2026-10-30
| | | | | | | | | | Persist retry state per source revision so temporary delivery failures survive worker restarts and duplicate queued tasks cannot bypass the retry schedule. Use capped exponential backoff, stop after a configurable attempt limit, and reset the retry budget when a source changes or an admin resends manually.
* fix(sender): ignore inconclusive empty scansDennis Fink2026-10-30
| | | | | | | | Treat scans with no successfully processed source pages as inconclusive so a temporary empty or partially rebuilt output directory cannot mark every source deleted or trigger removal Webmentions. Defer pending sends until at least one source is scanned successfully.
* fix(sender): prevent duplicate canonical source URLsDennis Fink2026-10-30
| | | | | | | | | Keep persisted sources, including deleted ones, as owners of their canonical URLs so newly scanned aliases cannot claim the same source identity. Skip duplicate canonical paths without blocking unrelated sources from being saved or queued for delivery.
* fix(sender): isolate source parser failuresDennis Fink2026-10-30
| | | | | | | | | Prevent malformed pages or unexpected third-party parser exceptions from aborting an entire source scan or causing previously known sources to be treated as deleted. Keep persistence failures visible so database errors still fail the scan.
* fix(core): resolve static type checking issuesDennis Fink2026-10-27
|
* refactor(core): simplify configuration keysDennis Fink2026-10-25
| | | | | | Remove the WEBMENTIONS_SSG prefix from application-specific configuration settings and update validators, tasks, and tests to use the shorter names.
* test(core): align tests with module ownershipDennis Fink2026-08-21
| | | | | | | | | Move direct validator, model, and Huey extension coverage into dedicated test modules and mock dependencies where behavior belongs to another module. Expand Huey extension typing and coverage while removing duplicate assertions from scanner, sender, form, and view tests.
* chore(core): standardize SPDX copyright headersDennis Fink2026-08-20
| | | | | | | | Add SPDX copyright and license headers to source, test, migration, template, and static files throughout the project. Include the maintainer email in copyright notices and fill in the BSD license copyright holder.
* refactor(core): improve typing and test coverageDennis Fink2026-08-19
| | | | | | | | | Restructure database and user CLI commands, make passwords write-only model properties, and use the UTC datetime constant throughout the application. Add PEP 287-style documentation and expand receiver, scanner, sender, and URL security tests to cover error paths and edge cases.
* feat(sender): add manual source rescan actionDennis Fink2026-08-17
| | | | | | | | Add an authenticated action to queue a source rescan from the sent Webmentions overview. Extract the reusable confirmation modal and its JavaScript so received and sent admin actions share the same implementation.
* feat(sender): add outgoing Webmention pipelineDennis Fink2026-08-15
Scan generated h-entry documents for Webmention targets and track source revisions and delivery state in the database. Discover target endpoints, send Webmentions asynchronously, and reconcile updated, removed, restored, and deleted sources across scans. Add authenticated views for inspecting sent Webmentions and make the scanner schedule configurable.