Commits

Git-Historie von /var/www/meran-kauf · Branch main · 110 Commits. Neueste zuerst, Zeiten in Europe/Berlin. Regel: nach jedem Feature ein Commit auf main (siehe AGENTS.md).

HashDatumAutorNachricht
3df3bf8 Meran-Kauf Disable new-listings email via MERAN_NOTIFY_EMAIL_ENABLED (scrape unchanged); meran-scrape-ctl email on|off
e02a816 Meran-Kauf docs: scraper+email on/off switch (meran-scrape-ctl), cutoff 2027-01-01 Madeira; undo 2026-11-08 pause note
c8eb564 Meran-Kauf docs: pause Meran scraper + new-listings email (timer disabled, service gated by flag) with reactivation steps
da395f7 Meran-Kauf docs: A/B-only duplicate rules
6e93bf0 Meran-Kauf Duplicate rules: only same-title (A) and same-price+size (B); fuzzy checks removed; dismissals persist in DB
1cf6c27 Meran-Kauf docs: duplicates view Delete + deleted-never-recalculated rules
181a17b Meran-Kauf Duplicates view: Delete in right-click card menu (soft delete); deleted listings never recalculated as duplicates
4ddff4d Muse
SSR listing rows: add the Duplikate jump button (twin of buildListingRow)
Commit 6d15aaf added the prominent amber Duplikate (n) deep-link button
only to the JS infinite-scroll row builder, so initially server-rendered
rows never showed it. Add the same button to the listing_row macro in
app/templates/_listing_row.html (n = dup_candidate_ids + merged_siblings,
same /?status=duplicates&dup=A&focus=<id> link, data-ref dup-jump-<id>).
Docs: memory/duplicates.md
ba28795 Muse
Delete button: soft delete via listings.deleted_at (row kept, hidden from table views)
The row context menu Delete no longer hard-deletes. It now sets
listings.deleted_at on that one listing id (not relevant anymore):
- new DB column deleted_at (migration, NULL = visible)
- db.soft_delete_listing(id): marks one id, leaves merged siblings and
  duplicate candidate/dismissal data untouched
- _listings_where() always excludes deleted_at IS NOT NULL -> covers the
  initial server-rendered table, /api/listings infinite scroll, search and
  the cross-filter segment
- counts_by_status / count_by_status (filter chips, neu badge) exclude
  soft-deleted rows as well
- DELETE /api/listings/<id> returns {ok, id, deleted_at} (404 if missing)
- frontend: confirm dialog now says the row stays in the DB; removed the
  misleading (N dups) suffix from the Delete button (data-ref stable)
- re-scraping the same URL does not clear deleted_at
Docs: AGENTS.md + memory/soft-delete.md
6d15aaf Meran-Kauf Listings table: prominent Duplikate jump button with count, deep-links into duplicates view
594014d Meran-Kauf Notiz column: moved right after Quelle, min-width 140px
a0d86b4 Meran-Kauf
Revert "Search: full-width prominent row above filter pills"
This reverts commit 15aceac798308d630544df983c50fc3b52bc7eab.
15aceac Meran-Kauf Search: full-width prominent row above filter pills
b3815b7 Meran-Kauf Search field on its own row above the Filter chips
a09d356 Meran-Kauf Table width toggle: 1400px vs full window width (persisted)
2c3e03d Meran-Kauf Table: full-bleed #listings-table-wrap to use full viewport width
b0b2a7d Meran-Kauf Table: min-width 1400px, still stretches to full window width
39b6402 Meran-Kauf Table layout: title min 130px, source column max 110px with ellipsis
13b51e6 Meran-Kauf
Compact url-cluster popup to merge popup height
Header row: title + status + Speichern button. Ziel and Name side by side in one row. URLs textarea (rows=2) carries the former label text as placeholder; the separate label span is removed.
d53bec0 Meran-Kauf
Fix: populate Ziel dropdown on page load
The url-cluster popup starts visible, but loadUcTargets() only ran when
reopening it via the pill, leaving the Ziel select empty on first view.
6bbe923 Meran-Kauf
Cluster table view: show clusters only, drop Kein Cluster group
Unclustered listings are hidden in the grouped view; they remain visible
in the regular table, where the per-row Cluster dropdown can remove a
listing from its cluster via the empty option.
6ca4e0c Meran-Kauf Docs: cluster table view spans all statuses
df7afbf Meran-Kauf
Cluster table view: always span all statuses
The grouped cluster view is not a status filter: it loads listings with
status=ALL so clustered listings are visible without switching to
Status > All. A small info row at the top states the all-status scope.
2a1a5da Meran-Kauf
Popups side by side: url-cluster-popup left, merge-popup right
layoutBottomPopups() now centers the pair horizontally at the bottom
instead of stacking vertically; both divs stay separate.
03efcf7 Meran-Kauf
Cluster table view: group listings by cluster in #listings-table
- btn-url-clusters now toggles a grouped table view: all results of the
  current filter, one cluster below the other, clusters sorted A-Z,
  unclustered listings in a trailing Kein Cluster group. Same cluster =
  slight grouping via alternating row tints. Rows are draggable between
  groups (POST /api/url-clusters/move); group headers carry a delete
  button; members without a listing show as muted orphan rows.
- Regular table: new trailing Cluster column with a per-row assignment
  dropdown (saves immediately); in cluster view the column shows the name.
- Backend: db._attach_cluster_info adds cluster_id/name to list_listings
  rows and listing_to_api_dict; rows carry data-url for the move API.
- The old url-clusters-panel is replaced by this view (its delete/reassign
  functions move to group headers, drag & drop and the row dropdown).
54a7b1a Meran-Kauf Docs: note popup stacking in url-clusters memory
8cee37f Meran-Kauf
Stack bottom popups vertically, compact both
- merge-popup was hidden behind url-cluster-popup (both fixed at the
  same bottom-center slot). New layoutBottomPopups() stacks the two
  separate divs vertically by measured height: merge-popup keeps its
  familiar bottom spot, url-cluster-popup sits above it; reopen pills
  join the stack when their popup is closed. Re-runs on show/hide,
  status-text changes and window resize.
- Compact both popups to save vertical space (smaller padding, gaps,
  font; cluster URL textarea 5 -> 3 rows).
0cb7863 Meran-Kauf
Manual URL clusters: bottom popup, filters button, reassign UI
- New tables url_clusters + url_cluster_members (UNIQUE url_key: one URL
  belongs to at most one cluster). No automation, purely user-curated.
- Bottom popup (teal, like merge-popup): paste URLs -> new cluster or add
  to existing one; optional name (defaults to Cluster N).
- Filters button URL-Cluster toggles an inline panel listing all clusters
  with member titles/sources, open links, per-cluster delete.
- Per-member reassign select moves a URL to another cluster or removes it.
- API: GET/POST /api/url-clusters, POST /api/url-clusters/move,
  DELETE /api/url-clusters/<id>.
- Docs: memory/url-clusters.md, AGENTS.md data-refs + index.
- Also restores the offline URL check, background scrape polling and
  not-seen marker code paths that the previous commit had overwritten
  with stale file copies (working tree now equals 6377943 + this feature).
9827757 Meran-Kauf
Manual URL clusters: bottom popup, filters button, reassign UI
- New tables url_clusters + url_cluster_members (UNIQUE url_key: one URL
  belongs to at most one cluster). No automation, purely user-curated.
- Bottom popup (teal, like merge-popup): paste URLs -> new cluster or add
  to existing one; optional name (defaults to Cluster N).
- Filters button URL-Cluster toggles an inline panel listing all clusters
  with member titles/sources, open links, per-cluster delete.
- Per-member reassign select moves a URL to another cluster or removes it.
- API: GET/POST /api/url-clusters, POST /api/url-clusters/move,
  DELETE /api/url-clusters/<id>.
- Docs: memory/url-clusters.md, AGENTS.md data-refs + index.
6377943 Meran-Kauf
Add not-seen marker for listings missing from the latest successful run
db._attach_seen_state sets not_seen_since when last_seen_at is older than the
start of the latest scrape run of the listing source that finished without
error (aborted/failed/paused runs never count; paused idealista never gets
the marker). Grey badge "nicht mehr gesehen seit DD.MM." with tooltip in the
Titel meta line (SSR + JS twin), also exposed in the API dict. State badges
wrap so they do not widen the Titel column. Docs + data-refs updated.
8391aeb Meran-Kauf
Add offline URL check for listings (offline badge, URLs prüfen button)
New nullable columns listings.offline_at, url_checked_at, url_http_status.
app/url_check.py gently re-checks stored URLs of listings not seen in the
latest successful run of their source (immoweb, immoreal, wohnen; never
idealista): 2.5-5 s delay, timeouts, stops a source on 403/429. 404/410 or a
redirect to a not-found page sets offline_at; 200 clears it. Runs after every
scrape and manually via POST /api/url-check (+ /status). Funnel status is not
changed. Red offline badge with tooltip in the Titel meta line.
Docs: memory/offline-listings.md, index + data-refs updated.
1ee1e39 Meran-Kauf
Run manual scrape in background with status polling
POST /api/scrape now starts meran-kauf-scrape.service via a narrow sudoers
rule (fallback: detached run_scrape.py) and returns immediately. A flock on
/var/lib/meran-kauf/scrape.lock prevents overlapping runs. New GET
/api/scrape/status (lock/unit state, current source, runs since trigger,
last summary); the UI shows "Scrape läuft…" and polls until done.
Unfinished scrape_runs older than 30 min are marked aborted with error_text.
gunicorn runs with --no-control-socket (fixes /var/www/.gunicorn error).
330f894 Meran-Kauf
Selective duplicate merge: selection mode, per-card menu, pair-level dismissals, undismiss UI
- "Auswahl bearbeiten" selection mode per candidate group: checkboxes,
  primary dropdown limited to the selection, enrich toggle, sticky action bar.
- POST /api/duplicates/selective-merge merges only selected listings and
  dismisses every other pair among the group (incl. kept-vs-kept pairs),
  so a wrong cluster cannot reform. Returns dismissed_pairs for Undo.
- Dismissals stay strictly pair-level: never suppress unrelated pairs.
- Per-card right-click menu: select/deselect, mark unique in this group,
  choose primary, open in table (row flash).
- Dismissed-pairs section with per-pair "Wieder öffnen"; toast with Undo
  after merge/dismiss. GET /api/duplicates/dismissals.
- POST /api/duplicates/dismiss also accepts explicit {pairs:[[a,b],...]}.
- Stable data-refs for all new controls; docs updated.
89b6afa Meran-Kauf One result per unique URL: url_key identity, upsert fallback, backfill dedupe
9c5db67 Meran-Kauf Row context menu: Delete (listing + merged group) and Tag hover submenu
35f0fa4 Meran-Kauf Tags: user-defined tag system (settings CRUD, title-column pills, tag sub-filter under status funnel)
ccf3eb4 Meran-Kauf Duplicates: primary status syncs to all merged siblings (status API + merge); one-time backfill of 23 rows
14b8340 meran-bot Listings table: add Rolle column (Primär / Sekundär badge)
cf32908 meran-bot Duplicates: bottom-center quick-merge popup (primary ID + other IDs)
b5232b2 Meran-Kauf Docs: VPS access rule (direct ssh netcup primary, Mac netcup_db_server fallback)
d44ec43 meran-bot Notiz: DB column + PATCH API + Notiz column with inline editor
9b9bdbb Meran-Kauf memory/PROJECT.md: language rule is always English (docs, code comments, commit messages, agent replies); replaces the user-writes-German note
1443465 Meran-Kauf Settings > Agents / Memory .md (/settings?section=docs): AGENTS.md + memory/PROJECT.md with copyable absolute path, rendered/raw toggle; fixed file whitelist (no path param), escape-first Markdown renderer
f818ff5 Meran-Kauf memory/PROJECT.md: handover overview for new agents (purpose, hosting, sources, data model, core logic, ops, workflow, open issues, contacts, memory index); AGENTS.md: prominent Start-here link + index entry
63923c6 Meran-Kauf Fix: status matrix no longer clipped by table wrapper (position:fixed from STATUS cell rect, opens down per e316aa2 or flips up, clamped to viewport, repositions on scroll/resize, hidden while gallery preview is open)
2fc7781 Meran-Kauf Settings > Commits: git history view (/settings?section=commits, git log with timeout + env-only safe.directory, 200/page + Mehr laden); AGENTS.md: prominent commit-on-main rule, no branches, no history rewrite
4bc537d Meran-Kauf memory/daily-scrape.md: note notify recipient address
288b4f6 Meran-Kauf Listings: header click-to-sort (?sort=&dir=, asc/desc/default, NULLs last, server-side; kept in chips, infinite scroll, search); part B
679472c Meran-Kauf Listings thead: hover badge with real DB column name + copy button (col-dbname-{key}, col-dbname-copy-{key}); no layout shift; part A (no sorting)
091e399 Meran-Kauf Objekttyp: listings.objekttyp ENUM(Wohnung,Haus,Grundstück) NULL, PATCH /api/listings/{id}/objekttyp, Typ dropdown column (objekttyp-select-{id}), merge carries objekttyp; table still fits 1400px
3e2cdf3 Meran-Kauf dup-popup: light grey background, keep amber border/stripe/button (size unchanged)
ae426e0 Meran-Kauf dup-popup: move to bottom-right, amber accent (size unchanged)
bc477a7 Meran-Kauf Titel column: merged-duplicate count button + hover popover (dup-count-btn-{id}, dup-popover-{id}) with in-app deep link to Duplikate group
f4f8746 Meran-Kauf Header: Quellen toggle button next to Drive link opens sources panel as full-width row; remove aggregator subtitle
bd6bf2e Meran-Kauf Layout: #content-wrap (filters + merge panel + listings/duplicates) driven by --content-width (1400px); settings via --settings-width
f0f79f0 Meran-Kauf Listings table: Wohnfläche, Gesamtfläche, Grundstück directly right of Preis
d33e61b Meran-Kauf Drive column: listings.drive_url, PATCH /api/listings/{id}/drive, inline edit in table + Duplikate cards, merge carries drive_url
763273c Meran-Kauf Header: Drive: Kauf link next to VPS path (main + settings)
23b1679 Meran-Kauf Immoreal detail areas: Wohnfläche/Verkaufsfläche/Grundstücksfläche via SOURCE_FIELD_MAP, scraper detail fetch for new+stale, backfill --source immoreal
73cc68c Meran-Kauf Areas: Immoweb detail parse (Nettofläche=Wohnfläche, Gesamtfläche, Grundstück numeric), SOURCE_FIELD_MAP, Grundstück/Gesamtfläche columns, backfill CLI
e7fc550 Meran-Kauf Duplikate: section filters A-E in toolbar (Alle / Titel+kein Preis / Titel+gleicher Preis / Titel+gleiche Wohnfläche / Titel), counts, URL+localStorage, price chips
456a96a Meran-Kauf Duplikate: same-title + same-Wohnfläche rules (incl. same source), reason chips, grouped candidates, first_seen_at, group dismiss, Neue-Duplikate popup
751ea0c Meran-Kauf feat(status): auto rule for new listings (Wohnfläche < 120 m² -> neu (unter 120 m²)) + one-off migration
b915d9b Meran-Kauf fix(status): validate against saved funnel (not hard-coded defaults); show labels in Duplikate
9446963 Meran-Kauf feat(settings): show example image path for listing #214 in Speicher
29b4e30 Meran-Kauf fix(images): thumb_pending stat excludes gone rows
27df728 Meran-Kauf feat(images): 10 req/s downloader (6 workers, shared rate limiter), configurable rate shown in Speicher
bc2b25e Meran-Kauf fix(images): treat portal SVG placeholders as gone; use local full when portal thumb is dead
9b328c6 Meran-Kauf feat: local storage of all listing images (listing_images, /media/, gentle downloader, Settings > Speicher)
5287aa8 Meran-Kauf fix: remove stray Jinja junk prefixing status button labels with ">"
c5d88b1 Meran-Kauf
feat: daily DB+code backups with Settings panel
Store dumps under /var/backups/meran-kauf (7 daily + 6 monthly retention).
systemd timer 03:15 Berlin; Settings Backups lists runs and can trigger one.
f38d451 Meran-Kauf
feat: Settings shell with configurable status funnel
Persist funnel (id/label/color/order) in app_settings. Filters, status
matrix, and right-click menu share one config; Settings UI to edit.
edd93c3 Meran-Kauf
feat: add gesendet/waiting and OUT (war in kontakt) statuses
Extend workflow matrix, chips, Duplikate select, merge priority, and
CSS. DB stores display labels; status_ref_slug for data-refs; VARCHAR(64).
2132886 Meran-Kauf
feat: left/right arrows on gallery hover preview
Add prev/next buttons on the preview stage (gallery-prev/next data-refs).
Same wrap-around stepGallery ±1 as click / right-click; update hint text.
907726e Meran-Kauf
style: make Duplikate primary member card visually obvious
Highlight checked .dup-member with primary border/glow; mute siblings;
always show prominent Primär badge via :has(input:checked).
f28d6ae Meran-Kauf
feat: Keine Dublette dismiss for duplicate candidates
Persist dismissed pairs in duplicate_dismissals (a < b) so refresh
and scrapes never re-flag them. Candidate UI gets secondary button;
POST /api/duplicates/dismiss (+ optional undismiss).
79567f8 Andreas Maaps
feat: daily random scrape timer + new-listings email notify
Replace hourly scrape timer with once-daily random window
06:00–midnight Europe/Berlin. Email grok@liftt.de when upsert
adds listings (NOTIFY_* env, never committed).
692c66c Andreas Maaps
Search across all statuses with in-filter + other segments
When q is set on a specific status chip, /api/listings returns
in-filter matches plus other_items from other statuses (cap 100).
UI shows „In diesem Filter“ then „In anderen Filtern“; empty q and
ALL keep prior single-list behaviour.
abe3ae0 Meran-Kauf
feat: server-side listings search (ID, title, price, m², URL)
Add debounced #listings-search near filters; /api/listings and index
accept q/search and filter canonical rows (sibling URL/id still finds
parent). Duplikate view filters groups client-side.
0cc14b4 Meran-Kauf
style: match #filters chips to status-matrix colors
Reuse .status-btn palette (merken yellow, kontaktieren green,
in kontakt blue, OUT red, etc.) for status filter chips via
data-status; Duplikate chip stays its own readable style.
db42412 agent feat: status dropdown per listing in Duplikate compare STATUS row
31570ed Meran-Kauf
feat: propagate non-NEU status onto primary on merge / Primär wechseln
When merging or changing primary, if the chosen primary is still neu and
any sibling has a workflow status (merken/kontaktieren/in kontakt/gesehen/OUT),
copy that status onto the primary so user workflow is not lost.
c83cde3 Meran-Kauf
feat: rich Duplikate compare, enrich-on-merge, re-primary, sources status
Show full scraped fields side-by-side on Duplikate (highlight empty vs filled).
On merge, optionally copy empty primary scalars/images from siblings (default ON).
Allow changing primary on already-merged groups; collapse merged groups by default.
Move sources panel into header with aktiv/pausiert badges (Idealista DataDome).
3c707d1 Meran-Kauf
feat: show listing thumbs on Duplikate pairs for visual compare
Reuse main-table gallery thumbs + hover preview (full-res only on hover)
so candidate/merged members can be compared side by side before merge.
140d46c Meran-Kauf
feat: cross-source duplicate candidates + manual merge
Flag cross-portal listing pairs (price ±8%, size ±8m²/±8%, soft title
match) without auto-merging. Store canonical vs merged_into_id so scrapers
keep upserting every portal row while normal filters show each apartment
once. Add Duplikate UI, ID column, manual merge-by-ID with primary picker,
ALL sibling markers, and /api/duplicates merge/unmerge/refresh.
4902dcf Meran-Kauf
feat: add Wohnen in Südtirol scrape source
AJAX getObjects list scrape for Meran Wohnung Kauf (ab 150 m²),
ENUM+UI Quelle/Bild thumbs (315x200c→1200x900f), Idealista stays paused.
d3835c0 Meran-Kauf feat: infinite scroll listings (20 per page)
fd067df Meran-Kauf docs: AGENTS note on careful image loading in Bild column
23c08d2 Meran-Kauf perf: small Bild thumbs, defer full-res hover images
d7cf013 Meran-Kauf feat: Immoreal Meran scrape, Quelle column, lazy galleries
bd47ec1 Meran-Kauf docs: memory/ topics, gentle scraping rules, AGENTS index
858bbec Meran-Kauf docs: Idealista scraper pause notes and gentle-scrape guidance
a49d396 Meran-Kauf Idealista: clearer DataDome error + optional IDEALISTA_HTTP_PROXY
e316aa2 Meran-Kauf Status menu always opens downward from top of STATUS cell
8adc8da Meran-Kauf Status buttons: slightly larger
27e1599 Meran-Kauf Pin status hover menu to top of STATUS column
6fc8d49 Meran-Kauf Status buttons: light hover feedback for click target
e0da4a4 Meran-Kauf Status buttons: vertical stack overlay; open above when taller than row
c59db5c Meran-Kauf Status buttons: filter order with OUT last, smaller size, narrower column
9c76fd5 Meran-Kauf Status: OUT only on row hover; equal button sizes; color merken/kontaktieren/in kontakt
b35eef5 Meran-Kauf Show data-ref label only on hover in e-mode
5775f60 Meran-Kauf Add data-ref on buttons and main listing-row elements
7df4aea Meran-Kauf Add data-ref IDs for main UI + e-key inspect/copy mode
9e86c50 Meran-Kauf Show listing counts on filter chips
f063ecd Meran-Kauf
Show VPS project path in the app header
Surface /var/www/meran-kauf under the brand block so the deploy location is visible in the UI.
ade26bc Meran-Kauf
Fix hover preview to render at natural size up to 500px
Stop .col-img img from forcing thumbnail dimensions onto the preview; prefer Immoweb _big URLs in the hover view.
b6519e0 Meran-Kauf
Hover image preview with gallery; click opens listing
Thumbnail click opens the source listing URL. Hover shows a fixed max-450px preview where left/right click cycles photos.
466c21f Meran-Kauf
Add in-app listing image gallery with left/right click
Fetch Immoweb detail galleries on demand, store image_urls, and cycle photos with left click (next) and right click (previous) on the thumbnail.
471372f Meran-Kauf
Show status matrix inline only on listing row hover
Hide other status buttons unless the whole result row is hovered; do not reveal them from OUT-button hover alone.
6686bbb Meran-Kauf
Add instant status matrix UI, larger images, and AGENTS.md
Status column moved left with always-visible OUT and hover 2-row matrix for other statuses; images doubled; document git commit workflow for agents.