Commits
Git-Historie von /var/www/meran-kauf · Branch main · 110 Commits.
Neueste zuerst, Zeiten in Europe/Berlin. Regel: nach jedem Feature ein Commit auf main (siehe AGENTS.md).
| Hash | Datum | Autor | Nachricht |
|---|---|---|---|
3df3bf8 |
Meran-Kauf | Disable new-listings email via MERAN_NOTIFY_EMAIL_ENABLED (scrape unchanged); meran-scrape-ctl email on|off | |
e02a816 |
Meran-Kauf | docs: scraper+email on/off switch (meran-scrape-ctl), cutoff 2027-01-01 Madeira; undo 2026-11-08 pause note | |
c8eb564 |
Meran-Kauf | docs: pause Meran scraper + new-listings email (timer disabled, service gated by flag) with reactivation steps | |
da395f7 |
Meran-Kauf | docs: A/B-only duplicate rules | |
6e93bf0 |
Meran-Kauf | Duplicate rules: only same-title (A) and same-price+size (B); fuzzy checks removed; dismissals persist in DB | |
1cf6c27 |
Meran-Kauf | docs: duplicates view Delete + deleted-never-recalculated rules | |
181a17b |
Meran-Kauf | Duplicates view: Delete in right-click card menu (soft delete); deleted listings never recalculated as duplicates | |
4ddff4d |
Muse |
SSR listing rows: add the Duplikate jump button (twin of buildListingRow)Commit 6d15aaf added the prominent amber Duplikate (n) deep-link button only to the JS infinite-scroll row builder, so initially server-rendered rows never showed it. Add the same button to the listing_row macro in app/templates/_listing_row.html (n = dup_candidate_ids + merged_siblings, same /?status=duplicates&dup=A&focus=<id> link, data-ref dup-jump-<id>). Docs: memory/duplicates.md |
|
ba28795 |
Muse |
Delete button: soft delete via listings.deleted_at (row kept, hidden from table views)The row context menu Delete no longer hard-deletes. It now sets
listings.deleted_at on that one listing id (not relevant anymore):
- new DB column deleted_at (migration, NULL = visible)
- db.soft_delete_listing(id): marks one id, leaves merged siblings and
duplicate candidate/dismissal data untouched
- _listings_where() always excludes deleted_at IS NOT NULL -> covers the
initial server-rendered table, /api/listings infinite scroll, search and
the cross-filter segment
- counts_by_status / count_by_status (filter chips, neu badge) exclude
soft-deleted rows as well
- DELETE /api/listings/<id> returns {ok, id, deleted_at} (404 if missing)
- frontend: confirm dialog now says the row stays in the DB; removed the
misleading (N dups) suffix from the Delete button (data-ref stable)
- re-scraping the same URL does not clear deleted_at
Docs: AGENTS.md + memory/soft-delete.md |
|
6d15aaf |
Meran-Kauf | Listings table: prominent Duplikate jump button with count, deep-links into duplicates view | |
594014d |
Meran-Kauf | Notiz column: moved right after Quelle, min-width 140px | |
a0d86b4 |
Meran-Kauf |
Revert "Search: full-width prominent row above filter pills"This reverts commit 15aceac798308d630544df983c50fc3b52bc7eab. |
|
15aceac |
Meran-Kauf | Search: full-width prominent row above filter pills | |
b3815b7 |
Meran-Kauf | Search field on its own row above the Filter chips | |
a09d356 |
Meran-Kauf | Table width toggle: 1400px vs full window width (persisted) | |
2c3e03d |
Meran-Kauf | Table: full-bleed #listings-table-wrap to use full viewport width | |
b0b2a7d |
Meran-Kauf | Table: min-width 1400px, still stretches to full window width | |
39b6402 |
Meran-Kauf | Table layout: title min 130px, source column max 110px with ellipsis | |
13b51e6 |
Meran-Kauf |
Compact url-cluster popup to merge popup heightHeader row: title + status + Speichern button. Ziel and Name side by side in one row. URLs textarea (rows=2) carries the former label text as placeholder; the separate label span is removed. |
|
d53bec0 |
Meran-Kauf |
Fix: populate Ziel dropdown on page loadThe url-cluster popup starts visible, but loadUcTargets() only ran when reopening it via the pill, leaving the Ziel select empty on first view. |
|
6bbe923 |
Meran-Kauf |
Cluster table view: show clusters only, drop Kein Cluster groupUnclustered listings are hidden in the grouped view; they remain visible in the regular table, where the per-row Cluster dropdown can remove a listing from its cluster via the empty option. |
|
6ca4e0c |
Meran-Kauf | Docs: cluster table view spans all statuses | |
df7afbf |
Meran-Kauf |
Cluster table view: always span all statusesThe grouped cluster view is not a status filter: it loads listings with status=ALL so clustered listings are visible without switching to Status > All. A small info row at the top states the all-status scope. |
|
2a1a5da |
Meran-Kauf |
Popups side by side: url-cluster-popup left, merge-popup rightlayoutBottomPopups() now centers the pair horizontally at the bottom instead of stacking vertically; both divs stay separate. |
|
03efcf7 |
Meran-Kauf |
Cluster table view: group listings by cluster in #listings-table- btn-url-clusters now toggles a grouped table view: all results of the current filter, one cluster below the other, clusters sorted A-Z, unclustered listings in a trailing Kein Cluster group. Same cluster = slight grouping via alternating row tints. Rows are draggable between groups (POST /api/url-clusters/move); group headers carry a delete button; members without a listing show as muted orphan rows. - Regular table: new trailing Cluster column with a per-row assignment dropdown (saves immediately); in cluster view the column shows the name. - Backend: db._attach_cluster_info adds cluster_id/name to list_listings rows and listing_to_api_dict; rows carry data-url for the move API. - The old url-clusters-panel is replaced by this view (its delete/reassign functions move to group headers, drag & drop and the row dropdown). |
|
54a7b1a |
Meran-Kauf | Docs: note popup stacking in url-clusters memory | |
8cee37f |
Meran-Kauf |
Stack bottom popups vertically, compact both- merge-popup was hidden behind url-cluster-popup (both fixed at the same bottom-center slot). New layoutBottomPopups() stacks the two separate divs vertically by measured height: merge-popup keeps its familiar bottom spot, url-cluster-popup sits above it; reopen pills join the stack when their popup is closed. Re-runs on show/hide, status-text changes and window resize. - Compact both popups to save vertical space (smaller padding, gaps, font; cluster URL textarea 5 -> 3 rows). |
|
0cb7863 |
Meran-Kauf |
Manual URL clusters: bottom popup, filters button, reassign UI- New tables url_clusters + url_cluster_members (UNIQUE url_key: one URL belongs to at most one cluster). No automation, purely user-curated. - Bottom popup (teal, like merge-popup): paste URLs -> new cluster or add to existing one; optional name (defaults to Cluster N). - Filters button URL-Cluster toggles an inline panel listing all clusters with member titles/sources, open links, per-cluster delete. - Per-member reassign select moves a URL to another cluster or removes it. - API: GET/POST /api/url-clusters, POST /api/url-clusters/move, DELETE /api/url-clusters/<id>. - Docs: memory/url-clusters.md, AGENTS.md data-refs + index. - Also restores the offline URL check, background scrape polling and not-seen marker code paths that the previous commit had overwritten with stale file copies (working tree now equals 6377943 + this feature). |
|
9827757 |
Meran-Kauf |
Manual URL clusters: bottom popup, filters button, reassign UI- New tables url_clusters + url_cluster_members (UNIQUE url_key: one URL belongs to at most one cluster). No automation, purely user-curated. - Bottom popup (teal, like merge-popup): paste URLs -> new cluster or add to existing one; optional name (defaults to Cluster N). - Filters button URL-Cluster toggles an inline panel listing all clusters with member titles/sources, open links, per-cluster delete. - Per-member reassign select moves a URL to another cluster or removes it. - API: GET/POST /api/url-clusters, POST /api/url-clusters/move, DELETE /api/url-clusters/<id>. - Docs: memory/url-clusters.md, AGENTS.md data-refs + index. |
|
6377943 |
Meran-Kauf |
Add not-seen marker for listings missing from the latest successful rundb._attach_seen_state sets not_seen_since when last_seen_at is older than the start of the latest scrape run of the listing source that finished without error (aborted/failed/paused runs never count; paused idealista never gets the marker). Grey badge "nicht mehr gesehen seit DD.MM." with tooltip in the Titel meta line (SSR + JS twin), also exposed in the API dict. State badges wrap so they do not widen the Titel column. Docs + data-refs updated. |
|
8391aeb |
Meran-Kauf |
Add offline URL check for listings (offline badge, URLs prüfen button)New nullable columns listings.offline_at, url_checked_at, url_http_status. app/url_check.py gently re-checks stored URLs of listings not seen in the latest successful run of their source (immoweb, immoreal, wohnen; never idealista): 2.5-5 s delay, timeouts, stops a source on 403/429. 404/410 or a redirect to a not-found page sets offline_at; 200 clears it. Runs after every scrape and manually via POST /api/url-check (+ /status). Funnel status is not changed. Red offline badge with tooltip in the Titel meta line. Docs: memory/offline-listings.md, index + data-refs updated. |
|
1ee1e39 |
Meran-Kauf |
Run manual scrape in background with status pollingPOST /api/scrape now starts meran-kauf-scrape.service via a narrow sudoers rule (fallback: detached run_scrape.py) and returns immediately. A flock on /var/lib/meran-kauf/scrape.lock prevents overlapping runs. New GET /api/scrape/status (lock/unit state, current source, runs since trigger, last summary); the UI shows "Scrape läuft…" and polls until done. Unfinished scrape_runs older than 30 min are marked aborted with error_text. gunicorn runs with --no-control-socket (fixes /var/www/.gunicorn error). |
|
330f894 |
Meran-Kauf |
Selective duplicate merge: selection mode, per-card menu, pair-level dismissals, undismiss UI- "Auswahl bearbeiten" selection mode per candidate group: checkboxes,
primary dropdown limited to the selection, enrich toggle, sticky action bar.
- POST /api/duplicates/selective-merge merges only selected listings and
dismisses every other pair among the group (incl. kept-vs-kept pairs),
so a wrong cluster cannot reform. Returns dismissed_pairs for Undo.
- Dismissals stay strictly pair-level: never suppress unrelated pairs.
- Per-card right-click menu: select/deselect, mark unique in this group,
choose primary, open in table (row flash).
- Dismissed-pairs section with per-pair "Wieder öffnen"; toast with Undo
after merge/dismiss. GET /api/duplicates/dismissals.
- POST /api/duplicates/dismiss also accepts explicit {pairs:[[a,b],...]}.
- Stable data-refs for all new controls; docs updated. |
|
89b6afa |
Meran-Kauf | One result per unique URL: url_key identity, upsert fallback, backfill dedupe | |
9c5db67 |
Meran-Kauf | Row context menu: Delete (listing + merged group) and Tag hover submenu | |
35f0fa4 |
Meran-Kauf | Tags: user-defined tag system (settings CRUD, title-column pills, tag sub-filter under status funnel) | |
ccf3eb4 |
Meran-Kauf | Duplicates: primary status syncs to all merged siblings (status API + merge); one-time backfill of 23 rows | |
14b8340 |
meran-bot | Listings table: add Rolle column (Primär / Sekundär badge) | |
cf32908 |
meran-bot | Duplicates: bottom-center quick-merge popup (primary ID + other IDs) | |
b5232b2 |
Meran-Kauf | Docs: VPS access rule (direct ssh netcup primary, Mac netcup_db_server fallback) | |
d44ec43 |
meran-bot | Notiz: DB column + PATCH API + Notiz column with inline editor | |
9b9bdbb |
Meran-Kauf | memory/PROJECT.md: language rule is always English (docs, code comments, commit messages, agent replies); replaces the user-writes-German note | |
1443465 |
Meran-Kauf | Settings > Agents / Memory .md (/settings?section=docs): AGENTS.md + memory/PROJECT.md with copyable absolute path, rendered/raw toggle; fixed file whitelist (no path param), escape-first Markdown renderer | |
f818ff5 |
Meran-Kauf | memory/PROJECT.md: handover overview for new agents (purpose, hosting, sources, data model, core logic, ops, workflow, open issues, contacts, memory index); AGENTS.md: prominent Start-here link + index entry | |
63923c6 |
Meran-Kauf | Fix: status matrix no longer clipped by table wrapper (position:fixed from STATUS cell rect, opens down per e316aa2 or flips up, clamped to viewport, repositions on scroll/resize, hidden while gallery preview is open) | |
2fc7781 |
Meran-Kauf | Settings > Commits: git history view (/settings?section=commits, git log with timeout + env-only safe.directory, 200/page + Mehr laden); AGENTS.md: prominent commit-on-main rule, no branches, no history rewrite | |
4bc537d |
Meran-Kauf | memory/daily-scrape.md: note notify recipient address | |
288b4f6 |
Meran-Kauf | Listings: header click-to-sort (?sort=&dir=, asc/desc/default, NULLs last, server-side; kept in chips, infinite scroll, search); part B | |
679472c |
Meran-Kauf | Listings thead: hover badge with real DB column name + copy button (col-dbname-{key}, col-dbname-copy-{key}); no layout shift; part A (no sorting) | |
091e399 |
Meran-Kauf | Objekttyp: listings.objekttyp ENUM(Wohnung,Haus,Grundstück) NULL, PATCH /api/listings/{id}/objekttyp, Typ dropdown column (objekttyp-select-{id}), merge carries objekttyp; table still fits 1400px | |
3e2cdf3 |
Meran-Kauf | dup-popup: light grey background, keep amber border/stripe/button (size unchanged) | |
ae426e0 |
Meran-Kauf | dup-popup: move to bottom-right, amber accent (size unchanged) | |
bc477a7 |
Meran-Kauf | Titel column: merged-duplicate count button + hover popover (dup-count-btn-{id}, dup-popover-{id}) with in-app deep link to Duplikate group | |
f4f8746 |
Meran-Kauf | Header: Quellen toggle button next to Drive link opens sources panel as full-width row; remove aggregator subtitle | |
bd6bf2e |
Meran-Kauf | Layout: #content-wrap (filters + merge panel + listings/duplicates) driven by --content-width (1400px); settings via --settings-width | |
f0f79f0 |
Meran-Kauf | Listings table: Wohnfläche, Gesamtfläche, Grundstück directly right of Preis | |
d33e61b |
Meran-Kauf | Drive column: listings.drive_url, PATCH /api/listings/{id}/drive, inline edit in table + Duplikate cards, merge carries drive_url | |
763273c |
Meran-Kauf | Header: Drive: Kauf link next to VPS path (main + settings) | |
23b1679 |
Meran-Kauf | Immoreal detail areas: Wohnfläche/Verkaufsfläche/Grundstücksfläche via SOURCE_FIELD_MAP, scraper detail fetch for new+stale, backfill --source immoreal | |
73cc68c |
Meran-Kauf | Areas: Immoweb detail parse (Nettofläche=Wohnfläche, Gesamtfläche, Grundstück numeric), SOURCE_FIELD_MAP, Grundstück/Gesamtfläche columns, backfill CLI | |
e7fc550 |
Meran-Kauf | Duplikate: section filters A-E in toolbar (Alle / Titel+kein Preis / Titel+gleicher Preis / Titel+gleiche Wohnfläche / Titel), counts, URL+localStorage, price chips | |
456a96a |
Meran-Kauf | Duplikate: same-title + same-Wohnfläche rules (incl. same source), reason chips, grouped candidates, first_seen_at, group dismiss, Neue-Duplikate popup | |
751ea0c |
Meran-Kauf | feat(status): auto rule for new listings (Wohnfläche < 120 m² -> neu (unter 120 m²)) + one-off migration | |
b915d9b |
Meran-Kauf | fix(status): validate against saved funnel (not hard-coded defaults); show labels in Duplikate | |
9446963 |
Meran-Kauf | feat(settings): show example image path for listing #214 in Speicher | |
29b4e30 |
Meran-Kauf | fix(images): thumb_pending stat excludes gone rows | |
27df728 |
Meran-Kauf | feat(images): 10 req/s downloader (6 workers, shared rate limiter), configurable rate shown in Speicher | |
bc2b25e |
Meran-Kauf | fix(images): treat portal SVG placeholders as gone; use local full when portal thumb is dead | |
9b328c6 |
Meran-Kauf | feat: local storage of all listing images (listing_images, /media/, gentle downloader, Settings > Speicher) | |
5287aa8 |
Meran-Kauf | fix: remove stray Jinja junk prefixing status button labels with ">" | |
c5d88b1 |
Meran-Kauf |
feat: daily DB+code backups with Settings panelStore dumps under /var/backups/meran-kauf (7 daily + 6 monthly retention). systemd timer 03:15 Berlin; Settings Backups lists runs and can trigger one. |
|
f38d451 |
Meran-Kauf |
feat: Settings shell with configurable status funnelPersist funnel (id/label/color/order) in app_settings. Filters, status matrix, and right-click menu share one config; Settings UI to edit. |
|
edd93c3 |
Meran-Kauf |
feat: add gesendet/waiting and OUT (war in kontakt) statusesExtend workflow matrix, chips, Duplikate select, merge priority, and CSS. DB stores display labels; status_ref_slug for data-refs; VARCHAR(64). |
|
2132886 |
Meran-Kauf |
feat: left/right arrows on gallery hover previewAdd prev/next buttons on the preview stage (gallery-prev/next data-refs). Same wrap-around stepGallery ±1 as click / right-click; update hint text. |
|
907726e |
Meran-Kauf |
style: make Duplikate primary member card visually obviousHighlight checked .dup-member with primary border/glow; mute siblings; always show prominent Primär badge via :has(input:checked). |
|
f28d6ae |
Meran-Kauf |
feat: Keine Dublette dismiss for duplicate candidatesPersist dismissed pairs in duplicate_dismissals (a < b) so refresh and scrapes never re-flag them. Candidate UI gets secondary button; POST /api/duplicates/dismiss (+ optional undismiss). |
|
79567f8 |
Andreas Maaps |
feat: daily random scrape timer + new-listings email notifyReplace hourly scrape timer with once-daily random window 06:00–midnight Europe/Berlin. Email grok@liftt.de when upsert adds listings (NOTIFY_* env, never committed). |
|
692c66c |
Andreas Maaps |
Search across all statuses with in-filter + other segmentsWhen q is set on a specific status chip, /api/listings returns in-filter matches plus other_items from other statuses (cap 100). UI shows „In diesem Filter“ then „In anderen Filtern“; empty q and ALL keep prior single-list behaviour. |
|
abe3ae0 |
Meran-Kauf |
feat: server-side listings search (ID, title, price, m², URL)Add debounced #listings-search near filters; /api/listings and index accept q/search and filter canonical rows (sibling URL/id still finds parent). Duplikate view filters groups client-side. |
|
0cc14b4 |
Meran-Kauf |
style: match #filters chips to status-matrix colorsReuse .status-btn palette (merken yellow, kontaktieren green, in kontakt blue, OUT red, etc.) for status filter chips via data-status; Duplikate chip stays its own readable style. |
|
db42412 |
agent | feat: status dropdown per listing in Duplikate compare STATUS row | |
31570ed |
Meran-Kauf |
feat: propagate non-NEU status onto primary on merge / Primär wechselnWhen merging or changing primary, if the chosen primary is still neu and any sibling has a workflow status (merken/kontaktieren/in kontakt/gesehen/OUT), copy that status onto the primary so user workflow is not lost. |
|
c83cde3 |
Meran-Kauf |
feat: rich Duplikate compare, enrich-on-merge, re-primary, sources statusShow full scraped fields side-by-side on Duplikate (highlight empty vs filled). On merge, optionally copy empty primary scalars/images from siblings (default ON). Allow changing primary on already-merged groups; collapse merged groups by default. Move sources panel into header with aktiv/pausiert badges (Idealista DataDome). |
|
3c707d1 |
Meran-Kauf |
feat: show listing thumbs on Duplikate pairs for visual compareReuse main-table gallery thumbs + hover preview (full-res only on hover) so candidate/merged members can be compared side by side before merge. |
|
140d46c |
Meran-Kauf |
feat: cross-source duplicate candidates + manual mergeFlag cross-portal listing pairs (price ±8%, size ±8m²/±8%, soft title match) without auto-merging. Store canonical vs merged_into_id so scrapers keep upserting every portal row while normal filters show each apartment once. Add Duplikate UI, ID column, manual merge-by-ID with primary picker, ALL sibling markers, and /api/duplicates merge/unmerge/refresh. |
|
4902dcf |
Meran-Kauf |
feat: add Wohnen in Südtirol scrape sourceAJAX getObjects list scrape for Meran Wohnung Kauf (ab 150 m²), ENUM+UI Quelle/Bild thumbs (315x200c→1200x900f), Idealista stays paused. |
|
d3835c0 |
Meran-Kauf | feat: infinite scroll listings (20 per page) | |
fd067df |
Meran-Kauf | docs: AGENTS note on careful image loading in Bild column | |
23c08d2 |
Meran-Kauf | perf: small Bild thumbs, defer full-res hover images | |
d7cf013 |
Meran-Kauf | feat: Immoreal Meran scrape, Quelle column, lazy galleries | |
bd47ec1 |
Meran-Kauf | docs: memory/ topics, gentle scraping rules, AGENTS index | |
858bbec |
Meran-Kauf | docs: Idealista scraper pause notes and gentle-scrape guidance | |
a49d396 |
Meran-Kauf | Idealista: clearer DataDome error + optional IDEALISTA_HTTP_PROXY | |
e316aa2 |
Meran-Kauf | Status menu always opens downward from top of STATUS cell | |
8adc8da |
Meran-Kauf | Status buttons: slightly larger | |
27e1599 |
Meran-Kauf | Pin status hover menu to top of STATUS column | |
6fc8d49 |
Meran-Kauf | Status buttons: light hover feedback for click target | |
e0da4a4 |
Meran-Kauf | Status buttons: vertical stack overlay; open above when taller than row | |
c59db5c |
Meran-Kauf | Status buttons: filter order with OUT last, smaller size, narrower column | |
9c76fd5 |
Meran-Kauf | Status: OUT only on row hover; equal button sizes; color merken/kontaktieren/in kontakt | |
b35eef5 |
Meran-Kauf | Show data-ref label only on hover in e-mode | |
5775f60 |
Meran-Kauf | Add data-ref on buttons and main listing-row elements | |
7df4aea |
Meran-Kauf | Add data-ref IDs for main UI + e-key inspect/copy mode | |
9e86c50 |
Meran-Kauf | Show listing counts on filter chips | |
f063ecd |
Meran-Kauf |
Show VPS project path in the app headerSurface /var/www/meran-kauf under the brand block so the deploy location is visible in the UI. |
|
ade26bc |
Meran-Kauf |
Fix hover preview to render at natural size up to 500pxStop .col-img img from forcing thumbnail dimensions onto the preview; prefer Immoweb _big URLs in the hover view. |
|
b6519e0 |
Meran-Kauf |
Hover image preview with gallery; click opens listingThumbnail click opens the source listing URL. Hover shows a fixed max-450px preview where left/right click cycles photos. |
|
466c21f |
Meran-Kauf |
Add in-app listing image gallery with left/right clickFetch Immoweb detail galleries on demand, store image_urls, and cycle photos with left click (next) and right click (previous) on the thumbnail. |
|
471372f |
Meran-Kauf |
Show status matrix inline only on listing row hoverHide other status buttons unless the whole result row is hovered; do not reveal them from OUT-button hover alone. |
|
6686bbb |
Meran-Kauf |
Add instant status matrix UI, larger images, and AGENTS.mdStatus column moved left with always-visible OUT and hover 2-row matrix for other statuses; images doubled; document git commit workflow for agents. |